1.环境及设置：
架构图
ceph-node1 192.168.0.232 这个节点同时用作ceph-deploy
ceph-node2 192.168.0.250
ceph-node3 192.168.0.220
操作系统： centos7.4
内核：3.10.0-693.21.1.el7.x86_64
关闭防火墙，selinux
安装ceph-deploy：
cat >/etc/yum.repos.d/ceph.repo
[ceph-noarch]
name=Ceph noarch packages
baseurl=https://download.ceph.com/rpm/el7/noarch
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://download.ceph.com/keys/release.asc
更新操作系统并安装ceph-deploy，其实两台也需要update，不需要安装ceph-deploy
sudo yum update
sudo yum install ceph-deploy
在所有节点创建ceph用户
sudo useradd -d /home/ceph-node -m ceph-node
sudo passwd ceph-node
echo "ceph-node ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/ceph-node
sudo chmod 0440 /etc/sudoers.d/ceph-node
在ceph-deploy节点创建密钥并分发到各节点
ssh-keygen
ssh-copy-id ceph-node@ceph-node2
ssh-copy-id ceph-node@ceph-node1
修改ssh配置文件默认ssh连接是用ceph-node用户
Host ceph-node1
Hostname ceph-node1
User ceph-node
Host ceph-node2
Hostname ceph-node2
User ceph-node
Host ceph-node3
Hostname ceph-node3
User ceph-node
每个节点设置时间同步
yum -y install ntpdate
crontab -e
*/5 * * * * /usr/sbin/ntpdate ntp1.aliyun.com
2.安装ceph
切换到ceph-deploy节点
mkdir /root/my-cluster
cd /root/my-cluster
如果安装过程有错误或者失败可以用以下命令从头再来
ceph-deploy purge {ceph-node} [{ceph-node}]
ceph-deploy purgedata {ceph-node} [{ceph-node}]
ceph-deploy forgetkeys
rm ceph.*
创建集群
ceph-deploy new ceph-node1 ceph-node2 ceph-node3
vim ceph.conf #加入公共网络网段
public network = 192.168.0.0/24
安装ceph
ceph-deploy install --release luminous ceph-node1 ceph-node2 ceph-node3
部署mon生成密钥文件
ceph-deploy mon create-initial
发送配置文件
ceph-deploy admin ceph-node1 ceph-node2 ceph-node3
创建mgr节点，mgr需要高可用所以创建三个节点
ceph-deploy mgr create ceph-node1
ceph-deploy mgr create ceph-node2
ceph-deploy mgr create ceph-node3
创建osd
ceph-deploy osd create --data /dev/sdb ceph-node1
ceph-deploy osd create --data /dev/sdc ceph-node1
ceph-deploy osd create --data /dev/sdd ceph-node1
ceph-deploy osd create --data /dev/sdb ceph-node2
ceph-deploy osd create --data /dev/sdc ceph-node2
ceph-deploy osd create --data /dev/sdd ceph-node2
ceph-deploy osd create --data /dev/sdb ceph-node3
ceph-deploy osd create --data /dev/sdc ceph-node3
ceph-deploy osd create --data /dev/sdd ceph-node3
使用ceph -s查看集群是否健康，osd是否加入节点
状态图
添加额外mon节点，mon节点也需要高可用
ceph-deploy mon add ceph-node2 ceph-node3
至此ceph集群安装完成
3.安装常见错误
- 错误：
[ceph_deploy][ERROR ] RuntimeError: connecting to host: n1 resulted in errors: IOError cannot send (already closed?)
解决方法：
编辑visudo注释下面的
Defaults requiretty
- 错误：
[n2][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
解决方法：
在ceph.conf添加
public network = 192.168.122.0/24
- 错误：
ceph-deploy安装的时候到yum安装那报错
解决：可能是源有问题执行yum -y remove centos-release-ceph
