10 高可用集群
Ceph RGW是一个无状态的http服务,可以使用http反向代理实现高可用。
1、部署第2个rgw服务
[cephadm@node-1 ceph-cluster]$ ceph-deploy rgw deploy node-2
usage: ceph-deploy rgw [-h] {create} ...
ceph-deploy rgw: error: argument subcommand: invalid choice: 'deploy' (choose from 'create')
[cephadm@node-1 ceph-cluster]$ ceph-deploy rgw create node-2
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephadm/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (2.0.1): /bin/ceph-deploy rgw create node-2
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] rgw : [('node-2', 'rgw.node-2')]
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : create
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7fb4e766af38>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function rgw at 0x7fb4e7cb1050>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.rgw][DEBUG ] Deploying rgw, cluster ceph hosts node-2:rgw.node-2
[node-2][DEBUG ] connection detected need for sudo
[node-2][DEBUG ] connected to host: node-2
[node-2][DEBUG ] detect platform information from remote host
[node-2][DEBUG ] detect machine type
[ceph_deploy.rgw][INFO ] Distro info: CentOS Linux 7.9.2009 Core
[ceph_deploy.rgw][DEBUG ] remote host will use systemd
[ceph_deploy.rgw][DEBUG ] deploying rgw bootstrap to node-2
[node-2][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[node-2][WARNIN] rgw keyring does not exist yet, creating one
[node-2][DEBUG ] create a keyring file
[node-2][DEBUG ] create path recursively if it doesn't exist
[node-2][INFO ] Running command: sudo ceph --cluster ceph --name client.bootstrap-rgw --keyring /var/lib/ceph/bootstrap-rgw/ceph.keyring auth get-or-create client.rgw.node-2 osd allow rwx mon allow rw -o /var/lib/ceph/radosgw/ceph-rgw.node-2/keyring
[node-2][INFO ] Running command: sudo systemctl enable ceph-radosgw@rgw.node-2
[node-2][WARNIN] Created symlink from /etc/systemd/system/ceph-radosgw.target.wants/ceph-radosgw@rgw.node-2.service to /usr/lib/systemd/system/ceph-radosgw@.service.
[node-2][INFO ] Running command: sudo systemctl start ceph-radosgw@rgw.node-2
[node-2][INFO ] Running command: sudo systemctl enable ceph.target
[ceph_deploy.rgw][INFO ] The Ceph Object Gateway (RGW) is now running on host node-2 and default port 7480
2、修改监听端口(可选)
#修改配置文件
[cephadm@node-1 ceph-cluster]$ vi ceph.conf
[cephadm@node-1 ceph-cluster]$ cat ceph.conf
[global]
fsid = 9ebc9b51-1406-43cc-bdd2-d560e58d842f
public_network = 64.115.3.0/24
mon_initial_members = node-1, node-2, node-3
mon_host = 64.115.3.101,64.115.3.102,64.115.3.103
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
mon_allow_pool_delete = true
[client.rgw.node-1]
rgw_frontends = "civetweb port=80"
[client.rgw.node-2]
rgw_frontends = "civetweb port=80"
# push配置文件
[cephadm@node-1 ceph-cluster]$ ceph-deploy --overwrite-conf config push node-1 node2
# 重启rgw服务
[root@node-2 ~]# systemctl restart ceph-radosgw.target
后面,可以使用haproxy+keepalived实现。 也可以使用ngninx+keepalived实现。
monitor高可用自动实现,通过查询monmap获取其他的mon成员连接方式。
mds高可用自动实现,通过monitor获取当前active状态的mds服务。