找回密码
 注册
查看: 846|回复: 2

手动方式部署ceph集群以及添加osd

[复制链接]

1

主题

0

回帖

12

积分

管理员

积分
12
QQ
发表于 2022-7-19 11:20:00 | 显示全部楼层 |阅读模式
一、准备
前一篇点击打开链接只部署了一个单机集群。在这一篇里,手动部署一个多机集群:mycluster。我们有三台机器nod1,node2和node3;其中node1可以免密ssh/scp任意其他两台机器。我们的所有工作都在node1上完成。
准备工作包括在各个机器上安装ceph rpm包(见前一篇第1节点击打开链接),并在各个机器上修改下列文件:
/usr/lib/systemd/system/ceph-mon@.service /usr/lib/systemd/system/ceph-osd@.service /usr/lib/systemd/system/ceph-mds@.service /usr/lib/systemd/system/ceph-mgr@.service /usr/lib/systemd/system/ceph-radosgw@.service
修改:
Environment=CLUSTER=ceph <--- 改成CLUSTER=mycluster ExecStart=/usr/bin/... --id %i --setuser ceph --setgroup ceph <--- 删掉--setuser ceph --setgroup ceph二、创建工作目录
在node1创建一个工作目录,后续所有工作都在node1上的这个工作目录中完成;
mkdir /tmp/mk-ceph-cluster cd /tmp/mk-ceph-cluster三、创建配置文件vim mycluster.conf [global] cluster = mycluster fsid = 116d4de8-fd14-491f-811f-c1bdd8fac141 public network = 192.168.100.0/24 cluster network = 192.168.73.0/24 auth cluster required = cephx auth service required = cephx auth client required = cephx osd pool default size = 3 osd pool default min size = 2 osd pool default pg num = 128 osd pool default pgp num = 128 osd pool default crush rule = 0 osd crush chooseleaf type = 1 admin socket = /var/run/ceph/$cluster-$name.asock pid file = /var/run/ceph/$cluster-$name.pid log file = /var/log/ceph/$cluster-$name.log log to syslog = false max open files = 131072 ms bind ipv6 = false [mon] mon initial members = node1,node2,node3 mon host = 192.168.100.131:6789,192.168.100.132:6789,192.168.100.133:6789 ;Yuanguo: the default value of {mon data} is /var/lib/ceph/mon/$cluster-$id, ; we overwrite it. mon data = /var/lib/ceph/mon/$cluster-$name mon clock drift allowed = 10 mon clock drift warn backoff = 30 mon osd full ratio = .95 mon osd nearfull ratio = .85 mon osd down out interval = 600 mon osd report timeout = 300 debug ms = 20 debug mon = 20 debug paxos = 20 debug auth = 20 [mon.node1] host = node1 mon addr = 192.168.100.131:6789 [mon.node2] host = node2 mon addr = 192.168.100.132:6789 [mon.node3] host = node3 mon addr = 192.168.100.133:6789 [mgr] ;Yuanguo: the default value of {mgr data} is /var/lib/ceph/mgr/$cluster-$id, ; we overwrite it. mgr data = /var/lib/ceph/mgr/$cluster-$name [osd] ;Yuanguo: we wish to overwrite {osd data}, but it seems that 'ceph-disk' forces ; to use the default value, so keep the default now; maybe in later versions ; of ceph the limitation will be eliminated. osd data = /var/lib/ceph/osd/$cluster-$id osd recovery max active = 3 osd max backfills = 5 osd max scrubs = 2 osd mkfs type = xfs osd mkfs options xfs = -f -i size=1024 osd mount options xfs = rw,noatime,inode64,logbsize=256k,delaylog filestore max sync interval = 5 osd op threads = 2 debug ms = 100 debug osd = 100
需要说明的是,在这个配置文件中,我们覆盖了一些默认值,比如:{mon data}和{mgr data},但是没有覆盖{osd data},因为ceph-disk貌似强制使用默认值。另外,pid, sock文件被放置在/var/run/ceph/中,以$cluster-$name命名;log文件放置在/var/log/ceph/中,也是以$cluster-$name命名。这些都可以覆盖。
四、生成keyring
在单机部署中点击打开链接,我们说过,有两种操作集群中user及其权限的方式,这里我们使用第一种:先生成keyring文件,然后在创建集群时带入使之生效。
ceph-authtool --create-keyring mycluster.keyring --gen-key -n mon. --cap mon 'allow *' , S% L7 H+ v: C8 ?6 E; z* ^
ceph-authtool --create-keyring mycluster.client.admin.keyring --gen-key -n client.admin --set-uallow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' & M# P+ w, p- ~* K6 K' M
ceph-authtool --create-keyring mycluster.client.bootstrap-osd.keyring --gen-key -n client.bootstrap-osd --cap mon 'allow profile bootstrap-osd' " R  X9 ~: A. h
ceph-authtool --create-keyring mycluster.mgr.node1.keyring --gen-key -n mgr.node1 --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *'
; ~& i! S% F! O5 h1 I0 N3 g+ [ceph-authtool --create-keyring mycluster.mgr.node2.keyring --gen-key -n mgr.node2 --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *'
5 B6 l7 N/ a% d5 o2 Qceph-authtool --create-keyring mycluster.mgr.node3.keyring --gen-key -n mgr.node3 --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *'
# W( v( S" x$ qceph-authtool mycluster.keyring --import-keyring mycluster.client.admin.keyring 6 j# m/ p% C' m: G! q& b
ceph-authtool mycluster.keyring --import-keyring mycluster.client.bootstrap-osd.keyring
' z/ b; b7 n' n$ r5 h2 h, Kceph-authtool mycluster.keyring --import-keyring mycluster.mgr.node1.keyring 9 ?; _) r, z7 [2 }
ceph-authtool mycluster.keyring --import-keyring mycluster.mgr.node2.keyring ; ~0 t" ^7 r7 f. u: w' e5 Z
ceph-authtool mycluster.keyring --import-keyring mycluster.mgr.node3.keyring3 t" M+ d; V3 Q0 U3 m' O
cat mycluster.keyring [mon.] key = AQA525NZsY73ERAAIM1J6wSxglBNma3XAdEcVg== caps mon = "allow *" [client.admin] key = AQBJ25NZznIpEBAAlCdCy+OyUIvxtNq+1DSLqg== auid = 0 caps mds = "allow *" caps mgr = "allow *" caps mon = "allow *" caps osd = "allow *" [client.bootstrap-osd] key = AQBW25NZtl/RBxAACGWafYy1gPWEmx9geCLi6w== caps mon = "allow profile bootstrap-osd" [mgr.node1] key = AQBb25NZ1mIeFhAA/PmRHFY6OgnAMXL1/8pSxw== caps mds = "allow *" caps mon = "allow profile mgr" caps osd = "allow *" [mgr.node2] key = AQBg25NZJ6jyHxAAf2GfBAG5tuNwf9YjkhhEWA== caps mds = "allow *" caps mon = "allow profile mgr" caps osd = "allow *" [mgr.node3] key = AQBl25NZ7h6CJRAAaFiea7hiTrQNVoZysA7n/g== caps mds = "allow *" caps mon = "allow profile mgr" caps osd = "allow *"五、生成monmap
生成monmap并添加3个monitor
monmaptool --create --add node1 192.168.100.131:6789 --add node2 192.168.100.132:6789 --add node3 192.168.100.133:6789 --fsid 116d4de8-fd14-491f-811f-c1bdd8fac141 7 ?5 K7 n" _0 W& j
monmap [plain] view plain copy monmaptool --print monmap monmaptool: monmap file monmap epoch 0 fsid 116d4de8-fd14-491f-811f-c1bdd8fac141 last_changed 2017-08-16 05:45:37.851899 created 2017-08-16 05:45:37.851899 0: 192.168.100.131:6789/0 mon.node1 1: 192.168.100.132:6789/0 mon.node2 2: 192.168.100.133:6789/0 mon.node3六、分发配置文件
keyring和monmap
把第2、3和4步中生成的配置文件,keyring,monmap分发到各个机器。由于mycluster.mgr.nodeX.keyring暂时使用不到,先不分发它们(见第8节)。
cp mycluster.client.admin.keyring mycluster.client.bootstrap-osd.keyring mycluster.keyring mycluster.conf monmap /etc/ceph scp mycluster.client.admin.keyring mycluster.client.bootstrap-osd.keyring mycluster.keyring mycluster.conf monmap node2:/etc/ceph scp mycluster.client.admin.keyring mycluster.client.bootstrap-osd.keyring mycluster.keyring mycluster.conf monmap node3:/etc/ceph七、创建集群1、创建{mon data}目录mkdir /var/lib/ceph/mon/mycluster-mon.node1 9 e1 u, A7 m5 h
ssh node2 mkdir /var/lib/ceph/mon/mycluster-mon.node2
/ z& V: Q/ A# e3 Assh node3 mkdir /var/lib/ceph/mon/mycluster-mon.node3
注意,在配置文件mycluster.conf中,我们把{mon data}设置为/var/lib/ceph/mon/$cluster-$name,而不是默认的/var/lib/ceph/mon/$cluster-$id;& n' P+ ?, m' p+ f4 m$ ^
$cluster-$name展开为mycluster-mon.node1(23);& `$ f$ Y! q) s+ s3 f3 v. A4 n
默认的$cluster-$id展开为mycluster-node1(23);
2、初始化monitorceph-mon --cluster mycluster --mkfs -i node1 --monmap /etc/ceph/monmap --keyring /etc/ceph/mycluster.keyring
8 K. m: v* B4 Lssh node2 ceph-mon --cluster mycluster --mkfs -i node2 --monmap /etc/ceph/monmap --keyring /etc/ceph/mycluster.keyring / X9 [* q  z* v# H# G7 ~5 e+ s
ssh node3 ceph-mon --cluster mycluster --mkfs -i node3 --monmap /etc/ceph/monmap --keyring /etc/ceph/mycluster.keyring0 S) X, G% r6 \! E# o; m
注意,在配置文件mycluster.conf,我们把{mon data}设置为/var/lib/ceph/mon/$cluster-$name,展开为/var/lib/ceph/mon/mycluster-mon.node1(23)。ceph-mon会
$ ~" X# |: r  _! R; Z: ?4 b! |根据–cluster mycluster找到配置文件mycluster.conf,并解析出{mon data},然后在那个目录下进行初始化。
3、touch donetouch /var/lib/ceph/mon/mycluster-mon.node1/done
# \% Q, _+ Q9 R1 issh node2 touch /var/lib/ceph/mon/mycluster-mon.node2/done 0 s4 Z# R. [, v
ssh node3 touch /var/lib/ceph/mon/mycluster-mon.node3/done4、启动monitorssystemctl start ceph-mon@node1
" L9 e% b$ Y7 u: w1 b( A ssh node2 systemctl start [url=mailto:ceph-mon@node2]ceph-mon@node2

; o0 u; M4 P9 S5 ]7 a( Z! q" Rssh node3 systemctl start ceph-mon@node3[/url]- H+ q& X9 `+ J7 c3 Q5 c4 D( F8 b) C
5、检查机器状态ceph --cluster mycluster -s
- s9 b7 R( }0 I/ R! W  g) _cluster: id: 116d4de8-fd14-491f-811f-c1bdd8fac141 health: HEALTH_OK services: mon: 3 daemons, quorum node1,node2,node3 mgr: no daemons active osd: 0 osds: 0 up, 0 in data: pools: 0 pools, 0 pgs objects: 0 objects, 0 bytes usage: 0 kB used, 0 kB / 0 kB avail pgs:八、添加osd
每台集群都有一个/dev/sdb,我们把它们作为osd。
1、删除它们的分区2、prepareceph-disk prepare --cluster mycluster --cluster-uuid 116d4de8-fd14-491f-811f-c1bdd8fac141 --bluestore --block.db /dev/sdb --block.wal /dev/sdb /dev/sdb ssh node2 ceph-disk prepare --cluster mycluster --cluster-uuid 116d4de8-fd14-491f-811f-c1bdd8fac141 --bluestore --block.db /dev/sdb --block.wal /dev/sdb /dev/sdb ssh node3 ceph-disk prepare --cluster mycluster --cluster-uuid 116d4de8-fd14-491f-811f-c1bdd8fac141 /dev/sdb 注意:prepare node3:/dev/sdb时,我们没有加选项:--bluestore --block.db /dev/sdb --block.wal /dev/sdb;后面我们会看它和其他两个有什么不同。3、activateceph-disk activate /dev/sdb1 --activate-key /etc/ceph/mycluster.client.bootstrap-osd.keyring ssh node2 ceph-disk activate /dev/sdb1 --activate-key /etc/ceph/mycluster.client.bootstrap-osd.keyring ssh node3 ceph-disk activate /dev/sdb1 --activate-key /etc/ceph/mycluster.client.bootstrap-osd.keyring
注意:ceph-disk好像有两个问题:
  • 前面说过,它不使用自定义的{osd data},而强制使用默认值 /var/lib/ceph/osd/$cluster-$id9 ?( ^& P- K4 i& w, c/ K
  • 好像不能为一个磁盘指定osd id,而只能依赖它自动生成。虽然ceph-disk prepare有一个选项–osd-id,但是ceph-disk activate并不使用它而是自己生成。当不匹配时,会出现 如下错误:

    ) V- n9 m" d3 U8 v2 q
# ceph-disk activate /dev/sdb1 --activate-key /etc/ceph/mycluster.client.bootstrap-osd.keyring command_with_stdin: Error EEXIST: entity osd.0 exists but key does not match mount_activate: Failed to activate '['ceph', '--cluster', 'mycluster', '--name', 'client.bootstrap-osd', '--keyring', '/etc/ceph/mycluster.client.bootstrap-osd.keyring', '-i', '-', 'osd', 'new', u'ca8aac6a-b442-4b07-8fa6-62ac93b7cd29']' failed with status code 17
从 ‘-i’, ‘-‘可以看出,它只能自动生成osd id;
4、检查osd
在ceph-disk prepare时,node1:/dev/sdb和node2:/dev/sdb一样,都有–bluestore –block.db /dev/sdb –block.wal选项;node3:/dev/sdb不同,没有加这些选项。我们看看有什么不同。
4.1 node1
mount | grep sdb /dev/sdb1 on /var/lib/ceph/osd/mycluster-0 type xfs (rw,noatime,seclabel,attr2,inode64,noquota) ls /var/lib/ceph/osd/mycluster-0/ activate.monmap block block.db_uuid block.wal bluefs fsid kv_backend mkfs_done systemd whoami active block.db block_uuid block.wal_uuid ceph_fsid keyring magic ready type ls -l /var/lib/ceph/osd/mycluster-0/block lrwxrwxrwx. 1 ceph ceph 58 Aug 16 05:52 /var/lib/ceph/osd/mycluster-0/block -> /dev/disk/by-partuuid/a12dd642-b64c-4fef-b9e6-0b45cff40fa9 ls -l /dev/disk/by-partuuid/a12dd642-b64c-4fef-b9e6-0b45cff40fa9 lrwxrwxrwx. 1 root root 10 Aug 16 05:55 /dev/disk/by-partuuid/a12dd642-b64c-4fef-b9e6-0b45cff40fa9 -> ../../sdb2 blkid /dev/sdb2 /dev/sdb2: PARTLABEL="ceph block" PARTUU cat /var/lib/ceph/osd/mycluster-0/block_uuid a12dd642-b64c-4fef-b9e6-0b45cff40fa9 ls -l /var/lib/ceph/osd/mycluster-0/block.db lrwxrwxrwx. 1 ceph ceph 58 Aug 16 05:52 /var/lib/ceph/osd/mycluster-0/block.db -> /dev/disk/by-partuuid/1c107775-45e6-4b79-8a2f-1592f5cb03f2 ls -l /dev/disk/by-partuuid/1c107775-45e6-4b79-8a2f-1592f5cb03f2 lrwxrwxrwx. 1 root root 10 Aug 16 05:55 /dev/disk/by-partuuid/1c107775-45e6-4b79-8a2f-1592f5cb03f2 -> ../../sdb3 blkid /dev/sdb3 /dev/sdb3: PARTLABEL="ceph block.db" PARTUU cat /var/lib/ceph/osd/mycluster-0/block.db_uuid 1c107775-45e6-4b79-8a2f-1592f5cb03f2 ls -l /var/lib/ceph/osd/mycluster-0/block.wal lrwxrwxrwx. 1 ceph ceph 58 Aug 16 05:52 /var/lib/ceph/osd/mycluster-0/block.wal -> /dev/disk/by-partuuid/76055101-b892-4da9-b80a-c1920f24183f ls -l /dev/disk/by-partuuid/76055101-b892-4da9-b80a-c1920f24183f lrwxrwxrwx. 1 root root 10 Aug 16 05:55 /dev/disk/by-partuuid/76055101-b892-4da9-b80a-c1920f24183f -> ../../sdb4 blkid /dev/sdb4 /dev/sdb4: PARTLABEL="ceph block.wal" PARTUU cat /var/lib/ceph/osd/mycluster-0/block.wal_uuid 76055101-b892-4da9-b80a-c1920f24183f
可见,node1(node2)上,/dev/sdb被分为4个分区:
  • /dev/sdb1: metadata
  • /dev/sdb2:the main block device
  • /dev/sdb3: db
  • /dev/sdb4: wal# H; m; j- o* K8 A! S, O2 A( F0 u
具体见:ceph-disk prepare –help
4.2 node3
mount | grep sdb /dev/sdb1 on /var/lib/ceph/osd/mycluster-2 type xfs (rw,noatime,seclabel,attr2,inode64,noquota) ls /var/lib/ceph/osd/mycluster-2 activate.monmap active block block_uuid bluefs ceph_fsid fsid keyring kv_backend magic mkfs_done ready systemd type whoami ls -l /var/lib/ceph/osd/mycluster-2/block lrwxrwxrwx. 1 ceph ceph 58 Aug 16 05:54 /var/lib/ceph/osd/mycluster-2/block -> /dev/disk/by-partuuid/0a70b661-43f5-4562-83e0-cbe6bdbd31fb ls -l /dev/disk/by-partuuid/0a70b661-43f5-4562-83e0-cbe6bdbd31fb lrwxrwxrwx. 1 root root 10 Aug 16 05:56 /dev/disk/by-partuuid/0a70b661-43f5-4562-83e0-cbe6bdbd31fb -> ../../sdb2 blkid /dev/sdb2 /dev/sdb2: PARTLABEL="ceph block" PARTUU cat /var/lib/ceph/osd/mycluster-2/block_uuid 0a70b661-43f5-4562-83e0-cbe6bdbd31fb
可见,在node3上,/dev/sdb被分为2个分区:
  • /dev/sdb1:metadata
  • /dev/sdb2:the main block device;db和wal也在这个分区上。* i. o) i1 r& A" m
具体见:ceph-disk prepare –help
5、检查集群状态ceph --cluster mycluster -s cluster: id: 116d4de8-fd14-491f-811f-c1bdd8fac141 health: HEALTH_WARN no active mgr services: mon: 3 daemons, quorum node1,node2,node3 mgr: no daemons active osd: 3 osds: 3 up, 3 in data: pools: 0 pools, 0 pgs objects: 0 objects, 0 bytes usage: 0 kB used, 0 kB / 0 kB avail pgs:
由于没有添加mgr,集群处于WARN状态。
九、添加mgr1、创建{mgr data}目录mkdir /var/lib/ceph/mgr/mycluster-mgr.node1 ssh node2 mkdir /var/lib/ceph/mgr/mycluster-mgr.node2 ssh node3 mkdir /var/lib/ceph/mgr/mycluster-mgr.node3
注意,和{mon data}类似,在配置文件mycluster.conf中,我们把{mgr data}设置为/var/lib/ceph/mgr/$cluster-$name,而不是默认的/var/lib/ceph/mgr/$cluster-$id。
2、分发mgr的keyringcp mycluster.mgr.node1.keyring /var/lib/ceph/mgr/mycluster-mgr.node1/keyring scp mycluster.mgr.node2.keyring node2:/var/lib/ceph/mgr/mycluster-mgr.node2/keyring scp mycluster.mgr.node3.keyring node3:/var/lib/ceph/mgr/mycluster-mgr.node3/keyring3、启动mgrsystemctl start ceph-mgr@node1 ssh node2 systemctl start ceph-mgr@node2 ssh node3 systemctl start ceph-mgr@node34、检查集群状态ceph --cluster mycluster -s cluster: id: 116d4de8-fd14-491f-811f-c1bdd8fac141 health: HEALTH_OK services: mon: 3 daemons, quorum node1,node2,node3 mgr: node1(active), standbys: node3, node2 osd: 3 osds: 3 up, 3 in data: pools: 0 pools, 0 pgs objects: 0 objects, 0 bytes usage: 5158 MB used, 113 GB / 118 GB avail pgs:
可见,添加mgr之后,集群处于OK状态。

1

主题

0

回帖

12

积分

管理员

积分
12
QQ
 楼主| 发表于 2022-7-20 13:41:38 | 显示全部楼层
部署Ceph mon服务% C' i" n" w8 @
安装Ceph-mon服务程序(所有设备执行)
5 u8 F6 c4 @* u$ D9 ^yum install -y ceph-mon
( a/ M3 q2 y4 x, I7 R7 Y1) \: q# Q7 Y2 A! B: A: f
初始化Mon服务(Ceph 01执行)
- b" B+ }0 {5 @' E( N4 @% W' Z% G& L9 u生成uuid
8 c( K  j. v. N# }  Zuuidgen. a5 f2 e! ^$ Y; q5 @: O( `% b
> 9bf24809-220b-4910-b384-c1f06ea80728" W8 c' @* X) o4 Y* v! H. v+ D0 l9 I# K7 F
1( a" k) j+ G$ w
2
; n; R* Q1 K3 z2 Y. L创建Ceph配置文件' G6 k" m! i3 s7 U+ a. ~9 c& s
cat >> /etc/ceph/ceph.conf <<EOF7 a8 w! G+ }$ l
[global]
* ]0 \$ K: v* a+ c& {3 {) kfsid = 9bf24809-220b-4910-b384-c1f06ea80728
! v4 \9 T! A7 Smon_initial_members = ceph01,ceph02,ceph03
5 P9 d6 Y+ r, K$ Jmon_host = 10.40.65.156,10.40.65.175,10.40.65.129
% q+ b" W- l0 Y7 b: b8 |5 p" [# Dpublic_network = 10.40.65.0/24
# \+ j/ x. [* iauth_cluster_required = cephx
# K: d6 N  ]- G! {: G9 {1 b4 Tauth_service_required = cephx
0 {! f- p: O' u$ m% A) aauth_client_required = cephx
' s3 q- O) r  {; _& x$ Oosd_journal_size = 1024
) U! C5 e4 Z# Sosd_pool_default_size = 3
1 k8 n. I1 m" Fosd_pool_default_min_size = 23 }0 l" o9 h( ?& b0 z
osd_pool_default_pg_num = 64. \6 ]! C/ d7 G6 r. Y
osd_pool_default_pgp_num = 64
$ y+ r$ S0 b% q7 b# bosd_crush_chooseleaf_type = 1- S! c# f. b/ q+ H$ o
EOF$ N. d. q' w  P* j
1: a9 M8 H% ]0 e, F2 A' ]/ f
2# b, p! P8 S% ]9 q. e' k0 n- m
3
( C# z  n: ~( I- Y1 o! r* u% Y4  p4 j/ Q) p4 z
5' g! V4 J+ }" w  F
6
6 l; k5 T' e1 b% E% u4 F& E7
" f4 L4 _/ g9 d3 ?8
! t5 `  i! w( v8 q% ?; l! |' Z  m9& `: ]- V5 X: `1 F) N9 ?
10" z' j) ?( [7 K1 }
11
' X; y+ C. X9 e/ G( |# q12
! `5 \0 L& a9 ]2 V; X3 W13
2 Y" d' `% y" w, p14
% w' e4 V# H- r- P6 s7 |7 Z1 J/ Z157 H: o6 E- [( j/ u" _
16
5 C1 O8 D3 b( \创建集群Monitor密钥。
0 w. C( R& E$ ]/ ?; b4 sceph-authtool --create-keyring /tmp/ceph.mon.keyring --gen-key -n mon. --cap mon 'allow *'
2 w+ N5 Y- d4 N5 P: W; ~! O18 Q" \3 o& t2 J2 u- ?
创建client.admin用户、client.bootstrap-osd用户密钥,添加到集群密钥中。
9 T( S1 z/ G7 V" M1 m( y& ^3 rceph-authtool --create-keyring /etc/ceph/ceph.client.admin.keyring --gen-key -n client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *'
2 t  H/ S% d  T! `+ ^* J# e+ `7 t) Zceph-authtool /tmp/ceph.mon.keyring --import-keyring /etc/ceph/ceph.client.admin.keyring
4 z$ j( B/ j) K% `) P- R1# Q1 B0 m; t, O: [: T3 e
2
/ i  C1 p+ L" O/ Y9 [ceph-authtool --create-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring --gen-key -n client.bootstrap-osd --cap mon 'profile bootstrap-osd' --cap mgr 'allow r'" T( U, R: n# ]- K% ]
ceph-authtool /tmp/ceph.mon.keyring --import-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring
$ v5 q' V+ a( E1% n  r5 B+ x# f$ N: k
2
/ w0 q1 x& i7 c" @1 N使用主机名、主机IP地址、FSID生成monitor map。0 Q3 N1 q; r) W0 q& p; q- Z, O
monmaptool --create --add ceph01 10.40.65.156 --add ceph02 10.40.65.175 --add ceph03 10.40.65.129 --fsid 9bf24809-220b-4910-b384-c1f06ea80728 /tmp/monmap
& a# S: o5 Z( T1 y1 J! ~1% Z: Y; Y9 ^4 Z, Y) k! k
初始化并启动monitor服务
3 m( S4 q8 x' r5 k2 o$ wsudo -u ceph mkdir /var/lib/ceph/mon/ceph-ceph01# I# b0 g9 y4 A5 d. J7 c
chown ceph.ceph -R /var/lib/ceph /etc/ceph /tmp/ceph.mon.keyring /tmp/monmap
. O$ Z. Y7 m2 @& l" ]* Usudo -u ceph ceph-mon --mkfs -i ceph01 --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring; k5 K/ x6 X2 E1 Q
ls /var/lib/ceph/mon/ceph-ceph01/; n1 Q4 [8 E7 O6 J8 {! b4 @
1
) W& l. A+ W6 p; J2% C* t/ W9 I) W& o$ M& G2 F
3
/ h  E7 O6 R# T( b0 u4. ~' D/ m# v7 d: \6 Y* a: ~
systemctl start ceph-mon@ceph01
2 D9 k2 U5 }" osystemctl enable ceph-mon@ceph01
3 d2 a+ J  {, D- p2 }/ @" esystemctl status ceph-mon@ceph01
" q% Q) |3 k( N2 E1( C8 D7 e* _: w; o( [# A8 C2 \
2
: o" {2 h3 Z, q/ V3* z; [- z! y) A  w8 c
同步配置文件、密钥、monmap到其他节点中(Ceph 01执行)
/ q. c3 T) j, l. A7 [复制ceph.client.admin.keyring、client.bootstrap-osd key、ceph.mon.keyring、monitor map、ceph.conf到另外2个节点
: }9 T; {( I3 o5 _scp /etc/ceph/ceph.client.admin.keyring root@ceph02:/etc/ceph/
, U8 Y8 B) E) k* v( G% ?5 K* u; Z  T& ascp /etc/ceph/ceph.client.admin.keyring root@ceph03:/etc/ceph/
6 K8 P* K% f7 d# L+ J6 S1
, Q1 U6 C& ^8 o4 v0 M28 A$ ?# m- m4 m5 T+ S. D! q* u
scp /var/lib/ceph/bootstrap-osd/ceph.keyring root@ceph02:/var/lib/ceph/bootstrap-osd/0 m& L2 S1 Q* O' r4 J8 O
scp /var/lib/ceph/bootstrap-osd/ceph.keyring root@ceph03:/var/lib/ceph/bootstrap-osd/
" c% e8 i4 y* _1: I- W1 Y" Y' L
2
( q5 {. j* Y+ t( t0 d( D7 jscp /tmp/ceph.mon.keyring root@ceph02:/tmp/! A) d7 a9 I" ^2 E( C
scp /tmp/ceph.mon.keyring root@ceph03:/tmp/
( F' o& d* I, l& I1 o( r1
9 h* F( z9 G/ G, ]0 [* ?" X, B3 e& ^27 I  q3 ^9 b; f4 J2 E4 P% f
scp /tmp/monmap root@ceph02:/tmp/# G4 g+ a4 _+ U/ ?" X, B' Z
scp /tmp/monmap root@ceph03:/tmp/0 U% d# q8 p6 \, b2 f9 x. E
11 I3 H: K: \9 X! {5 h3 ~
2
. D- e: K; @, H( Nscp /etc/ceph/ceph.conf root@ceph02:/etc/ceph/6 @2 s1 d" y( Y  f: v& W. C  u) @
scp /etc/ceph/ceph.conf root@ceph03:/etc/ceph/
; C) ~  V" x) _, R1
4 ^1 \( ?9 |* ~23 M, e, X$ y% u  J
启动其他节点的monitor服务(Ceph 02执行)4 F, O; p* g& o7 o! H: G
sudo -u ceph mkdir /var/lib/ceph/mon/ceph-ceph02
! F# Z+ ~- M$ j& k6 e2 ?: echown ceph.ceph -R /var/lib/ceph /etc/ceph /tmp/ceph.mon.keyring /tmp/monmap
3 r  w7 }% [; Dsudo -u ceph ceph-mon --mkfs -i ceph02 --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring
8 d6 N  E7 O4 lls /var/lib/ceph/mon/ceph-ceph02/
( n$ W6 u& a: A1 I/ k. q1
6 Z' G- [% }. {; H4 R2& T- Q' {7 O6 n0 `8 `& V
3* Q" V6 u0 L/ A2 z9 N; B
40 h' `1 y% T; u3 k2 a
systemctl start ceph-mon@ceph02: f! R* `  ^2 a
systemctl enable ceph-mon@ceph02- [, @; R1 x3 s' p, @' ^
systemctl status ceph-mon@ceph02" q1 n2 ?! L4 D- t) W
17 ~( m9 j6 ?1 w, M
2+ G" M' Y! ]. M+ m8 I$ V7 F
3
6 M' \$ X+ O, u启动其他节点的monitor服务(Ceph 03执行)
& y+ v* {/ ]! |" S$ m! \0 b) Z0 Rsudo -u ceph mkdir /var/lib/ceph/mon/ceph-ceph03$ g) Z) D* o4 R5 V8 [6 g
chown ceph.ceph -R /var/lib/ceph /etc/ceph /tmp/ceph.mon.keyring /tmp/monmap
) c; d! I7 ?3 ysudo -u ceph ceph-mon --mkfs -i ceph03 --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring
3 q% ~  m( y0 _/ P+ y+ {ls /var/lib/ceph/mon/ceph-ceph03/0 f. o3 k- C7 ?) q6 ?; q
1
" p& P8 Y# B6 D! R" S; x  m2
3 T# V5 ~+ z- U% u0 M- Y0 ?5 _" \30 f3 j' z3 W+ e* {& E) W- T( I
4
! [8 H1 @+ ~8 @" [  B* l: qsystemctl start ceph-mon@ceph03
: c; _: l; j) |, |8 Vsystemctl enable ceph-mon@ceph03
4 f& h$ }+ A; z2 i$ ysystemctl status ceph-mon@ceph03% c; I% v! m. j  U- K! X
1
3 o& Y6 U( A0 C( U/ N2
. o$ x8 M( p- q2 |- C  w3
/ }' z4 I9 K0 G6 u( s: ?$ e! r查看当前集群状态(任意节点执行)% I' G& O2 E& f7 u1 Q
通过ceph -s命令查询集群状态,可见集群services中3个mon服务已启动。. w, h% p5 [7 I3 `2 o% g3 W
ceph -s4 {; N/ [' f; M# q8 p$ X1 n
> cluster: ) M" v& D0 m* B! u, t- U
>   id:     cf3862c5-f8f6-423e-a03e-beb40fecb74a  
- w* T8 N% A4 {' q9 _3 A! w" @>   health: HEALTH_OK, K% O" F; I* y( W9 Y
>
6 l) ^6 ^3 H. P1 Q& c> services:  8 b0 N% ], C# {. g$ n) ~% N
>   mon:  3 daemons, quorum ceph03,ceph02,ceph01 (age 12d)  + C9 n3 ?3 A- F
>   mgr:    6 R$ u6 |- C! E4 q/ Y$ ]- p
>   osd: . r( j2 ?9 o' T' o* m: W
>
5 `- j" M6 i" P) O  D, ?& B8 M. G> data:
. J* z/ ]- j, v; i' @: L>    pools:   
$ h/ O. M$ z: {6 `4 f- R>   objects:
6 b5 ~- Y9 ~$ w( l9 }- f>   usage:   
6 W% F( W" z/ I- W, H: Z>   pgs:# j% ~3 q7 y" i: B  a
1$ h% R  d' N, m2 `3 W8 ]
2
* f! l, b/ j$ e3$ z8 X+ U- A  C
46 z9 f0 [. i: L  d3 ^
5
6 h# H, U% z& D5 H6
4 B; |1 ~- P. K2 }" u7# P  \1 h$ A4 t9 Q/ I) e: s( T
8
% Q( ^: w0 x; j9 P' F9! y( G1 }2 s* O9 A1 [
10
( g6 `3 g! o9 ]11
3 g, U( A, y. i- a! a* v12
) P9 u, T9 \: R9 z- m9 O' t13
  L9 y# R; H. H7 e: o" w14
% r& M3 c  P" Z4 [15( z4 i( ^4 g$ Q8 K- x2 B3 R( U
部署Ceph mon服务(ceph-volume 自动化创建)
( S) ~" Q, d/ m) L/ j安装Ceph-osd服务程序(所有设备执行)
3 w6 w$ K  v+ C" Y- d" q- Nyum install -y ceph-osd
9 t3 A0 o0 E, U# F; G& F2 }! |17 L- P* n; d$ t. E7 V
初始化osd服务(所有设备执行)& a! @6 j8 q0 T* ^3 x0 X5 }' }1 h
通过fdisk等工具查看磁盘盘符,然后利用ceph-volume工具自动化创建osd服务。2 U$ g. Q1 o, z( o' ~3 }
ceph-volume lvm create --data /dev/sda  \% q% r. Q! `4 M
ceph-volume lvm create --data /dev/sdb
" u; V; _! R% n/ h# T2 Rceph-volume lvm create --data /dev/sdc
4 A6 W5 j5 @! `1/ g: Q5 [7 S& l0 z9 {, y7 M( v
2& S4 F- ~( a1 W4 W- q9 _; X
3
( d1 F5 W9 I3 k& w' f" E6 C查看当前集群状态(任意节点执行)# L4 L0 Q2 b% c9 V' g5 _
通过ceph osd tree命令查询集群状态,可见集群services中所有osd服务已启动。
! b1 f/ O" C, |$ w. _, x2 Jceph osd tree9 f# B7 r  v  _7 Y$ ^7 Z
> ID CLASS WEIGHT   TYPE NAME       STATUS REWEIGHT PRI-AFF( }" j& J: V7 |) |7 m, N7 \
> -1       16.36908 root default
9 F- J; K% L. W# f3 V> -3        5.45636     host ceph011 k# N& |7 I8 X# V4 ~
>  0   hdd  1.81879         osd.0       up  1.00000 1.00000
5 z% G0 u2 M  n& j7 f; C>  1   hdd  1.81879         osd.1       up  1.00000 1.00000: L& ?( D, Y8 z4 e3 O  ?
>  2   hdd  1.81879         osd.2       up  1.00000 1.000007 d6 s  O6 g& E8 ?8 i- K. `4 ?' |
> -5        5.45636     host ceph02
7 F1 X/ W" f8 \2 w# _1 b>  3   hdd  1.81879         osd.3       up  1.00000 1.000009 Y' w2 N2 X) f2 g( a/ `( _: ?
>  4   hdd  1.81879         osd.4       up  1.00000 1.000001 @6 [5 p5 X, G* W3 [
>  5   hdd  1.81879         osd.5       up  1.00000 1.00000* J  d# {% P( k! Y* d
> -7        5.45636     host ceph03# A( _; _5 R. ~/ _# X) q( S8 X( Z
>  6   hdd  1.81879         osd.6       up  1.00000 1.00000
; f( ~1 O2 f; `>  7   hdd  1.81879         osd.7       up  1.00000 1.00000
- x3 s6 l. g! H5 [7 ?! t* ]>  8   hdd  1.81879         osd.8       up  1.00000 1.00000% H, L5 S- N; M' M6 _: f6 s
1
9 i6 h7 [; ]5 {- G' P2" S3 O3 d/ l5 e  d: B
3
% a: C* s  W' S. Z$ T" P: C43 t/ q  g& `  W, |
5
! f; M2 X4 ?5 y9 `6 j6% l1 ]1 j% f; |6 W/ D9 h# L
7
3 c6 ~/ a% k% {" w: S) v88 e; j0 w' J1 r# \, f7 ~
9
, @- }0 ~8 c- B10! P4 G% o$ }! Y$ `  D) B
11
& [! B, [! }% X; g# j12
7 V7 l% X: v$ l! s13- O) x& M9 y0 w3 s
14
9 ?, b2 Z+ T3 ~$ T$ N; w4 Y156 X9 q+ e% t) |' x& k, G
部署Ceph mgr服务并开启Dashboard
  d3 M) {% l* A) R, w安装Ceph-mgr服务程序(所有设备执行)
  @9 m7 |7 `9 h( A# ]. `yum install -y ceph-mgr( V6 `: {  w# M* m0 w
1
: `0 _  z2 ^9 F7 o1 E, r; `' o初始化并启动主MGR服务(Ceph01执行)
, \: L8 {/ }' P4 Wmkdir -p /var/lib/ceph/mgr/ceph-ceph01
6 K/ E! H: j* ?# N6 _2 Xchown ceph.ceph -R /var/lib/ceph
4 I+ s- e9 L9 ^6 Jceph-authtool --create-keyring /etc/ceph/ceph.mgr.ceph01.keyring --gen-key -n mgr.ceph01 --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *'
/ c& F* l$ U, M& P" Kceph auth import -i /etc/ceph/ceph.mgr.ceph01.keyring8 Q( o4 g3 C: a
ceph auth get-or-create mgr.ceph01 -o /var/lib/ceph/mgr/ceph-ceph01/keyring  X" z' R4 Z6 U( {& j
1& g0 {. |) T0 U$ j6 v( A
24 o4 Y. d8 t  }: @. O5 v/ w. L# _
3/ ?9 a+ k1 I% _  H
4# j; G+ S/ e( g8 U) A
52 i( W; t  n5 P8 E7 g
systemctl start ceph-mgr@ceph01
' d' V5 Z; \: }8 z/ Nsystemctl enable ceph-mgr@ceph01
) Y  H7 T$ x1 s5 M, N& Q& K# ]systemctl status ceph-mgr@ceph010 g- q' p# Z1 m0 _
1
, W4 f& V) J* O/ V- a/ r4 G1 B2
. M9 g0 @; J! i2 n3- ~& x5 v5 Z8 d8 g# l
初始化并启动从MGR服务(Ceph02执行)
- Z6 E, [) @4 d$ e: qmkdir -p /var/lib/ceph/mgr/ceph-ceph020 a! O3 D  d' z& n& T& d2 {/ N
chown ceph.ceph -R /var/lib/ceph
! r4 D, G. D" p$ g# Iceph-authtool --create-keyring /etc/ceph/ceph.mgr.ceph02.keyring --gen-key -n mgr.ceph02 --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *'
. B3 X! R+ ]" G& a# {* z/ wceph auth import -i /etc/ceph/ceph.mgr.ceph02.keyring6 g: U. W7 H* d3 u
ceph auth get-or-create mgr.ceph02 -o /var/lib/ceph/mgr/ceph-ceph02/keyring+ s% ^3 n+ D7 t: X, x. e$ d
1
# m( Y2 _0 w2 m) \" A0 X21 o* l& F6 h8 y) y9 P; |
3% r* z+ N2 W/ x! l( U
4
+ r% Y% n. B8 b3 E5/ v8 Y) m, R1 c+ b
systemctl start ceph-mgr@ceph02
+ \1 V) w( m* Y, s3 Msystemctl enable ceph-mgr@ceph02
' R: h! V8 |( j4 b; I9 Psystemctl status ceph-mgr@ceph02
( p. H/ }7 {$ E% e& E. X" P1
4 X$ Q+ L+ U7 n; {2
6 r! \+ [) u, q3' z1 g  o/ L/ J( T7 D' }6 |
初始化并启动从MGR服务(Ceph03执行)1 z+ a$ Y# |. r( @- o. ]: d6 U
mkdir -p /var/lib/ceph/mgr/ceph-ceph03& B6 M; ^) q% ^* Z
chown ceph.ceph -R /var/lib/ceph  S1 h8 A7 H  Y+ L  i: h+ d1 a
ceph-authtool --create-keyring /etc/ceph/ceph.mgr.ceph03.keyring --gen-key -n mgr.ceph03 --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *'
+ o' R) F- g  W/ w+ i# h+ t* Xceph auth import -i /etc/ceph/ceph.mgr.ceph03.keyring
5 Z$ k. z4 [; L; L9 Zceph auth get-or-create mgr.ceph03 -o /var/lib/ceph/mgr/ceph-ceph03/keyring
) O$ U5 u1 N- C* M. W18 }* }4 l6 F) W3 ]3 ]( |
2
& m: D$ G3 t/ ]- s! ?3) d* X( ~7 e% O- E4 x8 q
4
+ }3 Q1 K# T) f. b# k& j/ L7 ?5* ]% d' L# M' i. C4 x
systemctl start ceph-mgr@ceph03) v. [( H& i  h& c
systemctl enable ceph-mgr@ceph03
6 `+ m7 \1 `$ P) }" R' H; N& n' D: Csystemctl status ceph-mgr@ceph03' x6 Z# }" M/ w0 w0 ~+ A; [
13 b, c8 ^4 P8 S9 F
2
2 O4 u5 _5 s: z: p9 S3" y5 I$ b' M! o# Z& ~
查看当前集群状态(任意节点执行)
$ j* {+ T9 Z& o8 u0 _' G2 p通过ceph -s命令查询集群状态,可见集群services中3个mgr服务已启动。- d% s8 b8 n4 w: \& H
ceph -s' W+ X# T6 `0 [# R) b6 F# v
> cluster: 5 X( o$ h* t9 Q; S$ ]$ c; m" Z7 b
>   id:     cf3862c5-f8f6-423e-a03e-beb40fecb74a  
+ B6 f& u7 C! Z$ J4 z. g>   health: HEALTH_OK
* ]1 U% U1 ]  ~9 h0 L>
' P9 ]- K1 t: @) l) d- s> services:  
! I/ }) F, m# y0 V0 j% F1 n3 Q>   mon:  3 daemons, quorum ceph03,ceph02,ceph01 (age 12d)  
1 S% g3 Y9 j2 H) Z>   mgr: ceph01(active, since 3w), standbys: ceph01、ceph02   
6 k  J9 u. T+ T$ w9 a% ~>   osd:   _. ~( F0 B7 W  u+ S* M& }( A
>
. B' p9 f) O1 g8 k> data:
$ {$ t9 b2 v7 _' J8 N4 N>    pools:   & [5 C( O" Y( F
>   objects:
3 o# e% V! W/ L7 `# w>   usage:   7 T6 t  i5 F( x2 w
>   pgs:8 h. l' b. ]8 X1 E& M
1+ b! o9 i* ~8 i$ R; G$ k
2
% r% n8 B" y, `( h" Q# {( [3
% U% ?% n" C, `. k( B! d; Y3 C7 |9 _+ X4
$ K7 M2 l' Y+ p: J6 a( j* B! n5
6 r( C, Y" r8 n, I6
( O, L# {/ ]2 S# e73 y/ H8 i# v2 x
8* R5 d6 Y- J: {$ f
9- E" \& J' `9 D
10( ?- T- f; x3 x8 l+ A1 A: C  V5 C
11
2 t2 Z- `, ?; o3 d" z+ M' A12
& L6 Z( \! T  p/ Q+ k8 ?13+ w( I) K+ h; z7 ]
14
) a- W& \4 S) e9 z15
# g" f1 k* w% B& d2 I; Q& ~1 T" v使能Dashboard访问功能(任意节点执行)
* n  `, l' b5 ]7 G开启mgr dashboard功能
: [+ X2 I, A" `: F! ?ceph mgr module enable dashboard5 T0 ^$ `2 G. f* O; c5 Y5 o+ c
13 s2 Q& D( x6 A  W
生成并安装自签名的证书, r  ]* P5 ~0 q8 h+ [
ceph dashboard create-self-signed-cert
7 ^0 p2 K, u# Q: L1
! z# f% h5 O! h2 P/ `* S配置dashboard0 j: V7 [- G3 ~- a6 o
ceph config set mgr mgr/dashboard/server_addr 10.40.65.148
% @) [5 r+ ^. ]7 z1 [ceph config set mgr mgr/dashboard/server_port 8080( ^( l. x& c6 O/ p" u+ r# S
ceph config set mgr mgr/dashboard/ssl_server_port 8443; @; E' i# e* m- {
1( m  x7 Z1 b8 L5 K! Z* `
2
% d2 |  T8 s3 q. S3 ?( x5 n3
8 M! I. l; Q/ R' L* u创建一个dashboard登录用户名密码& P1 m! x8 `5 J1 R6 j
echo '123456' > password.txt
6 t  U3 N0 f% Y; r- T- i- wceph dashboard ac-user-create admin  administrator -i password.txt
; W) G0 f8 o. W1
3 Q2 E/ ]+ Q; _& K5 V& p2
9 X+ S/ j* L; O+ [, M查看服务访问方式5 |+ O. V! [" @! X) J( L  _/ f3 H
ceph mgr services
1 W2 \3 b8 g9 c( @* m1
7 T; y5 i$ u( L# O3 V, }* E通过web访问Ceph Dashboard,用户名密码为admin/123456& `3 V  [& }/ p2 I
https://10.40.65.148:8443! d1 a9 c1 l  f9 ~( v9 G7 h8 n

3 w; f) O, Q1 q

1

主题

0

回帖

12

积分

管理员

积分
12
QQ
 楼主| 发表于 2022-7-20 13:47:56 | 显示全部楼层
部署Ceph mon服务- e; T" ?9 H5 ^0 |* ~! A& Y
安装Ceph-mon服务程序(所有设备执行)
& N$ E+ o: o0 q. a6 y8 [! y
* L2 |3 Y4 h' ^yum install -y ceph-mon
+ }$ O! Y$ t4 J+ u, T8 g) X. I1( z: X4 F+ W: ]; a9 a# E
初始化Mon服务(Ceph 01执行)
6 Z! W) \2 V2 l/ x" p" w# e) x; O  [. Y
生成uuid
  ?: v5 b& a3 a4 j) ~
; k3 i# X+ R0 X) Euuidgen
% q. K+ U' P6 o* N) l9 _+ x8 U0 y% @> 9bf24809-220b-4910-b384-c1f06ea80728% ^9 w4 ~1 m8 o
1  t/ z  n7 H; p, H
28 e, m/ K: Q3 O( `9 G, E7 W
创建Ceph配置文件& z2 x' L/ S1 x5 V# N4 [) w
4 x5 @% o- @. a6 x5 J
cat >> /etc/ceph/ceph.conf <<EOF
7 z# z4 W6 {' Q[global]4 a; m% H  a1 ?3 B% s# i" F9 x
fsid = 9bf24809-220b-4910-b384-c1f06ea80728; _5 ?& S, |5 `0 J0 N2 d$ h
mon_initial_members = ceph01,ceph02,ceph03$ j9 j' t4 j& w# g5 |4 Q
mon_host = 10.40.65.156,10.40.65.175,10.40.65.129
8 u* a/ q( N" X9 Qpublic_network = 10.40.65.0/24
- N) [/ X5 u1 `  u5 D% ]% i7 }8 kauth_cluster_required = cephx/ @+ H, r8 `& S. c, z! ^1 F/ E
auth_service_required = cephx
5 @0 l; u6 z/ k1 K* C& y: q& z7 Yauth_client_required = cephx: m+ w5 V# G& U) I
osd_journal_size = 1024
* E- a$ Q" h$ G8 e) g# \: D! Aosd_pool_default_size = 3
% s; N+ g8 ?  V7 g5 Aosd_pool_default_min_size = 2
, f# w1 T+ i( {# Josd_pool_default_pg_num = 64) {3 N1 T* R- F% F6 r6 M
osd_pool_default_pgp_num = 642 r  f6 Q+ w5 W8 S
osd_crush_chooseleaf_type = 1
* R) {7 F2 R4 }- C4 z% V; fEOF/ \. N: V* {& B5 R2 c* g
1' C% A7 B% }4 s- K. _
2
* {" H- y7 Y5 O3
% h$ n2 H# o4 V" C% W" \4
- n! j) q- a* `) `8 W6 S0 M- v2 r4 a59 U; x; e" d/ }" {' r7 v) }9 O
6
# Y' X1 T# n7 U: {* F7
  s1 W( w  ~4 W  w1 R% u" h8
6 b+ u3 H0 w3 [  g9; f, s2 u  v1 }: }
10
. b4 u% T8 e6 Z, z119 r4 W6 P. b  k: c& s& D
127 `; ~0 t: I4 a! ^1 [' g. ]
13
+ d# Y% i, v0 y% q" q14, U, b$ e* @0 I7 X/ \8 D; s
15, K! W" D% h& }& \/ A
16
) ^: }( q- x  Z3 @- o创建集群Monitor密钥。
. z$ i/ Q- \6 l% D, D" p3 ^) J9 z# u+ b9 y3 O1 o
ceph-authtool --create-keyring /tmp/ceph.mon.keyring --gen-key -n mon. --cap mon 'allow *'
0 G9 O3 D. j1 o* X" ]/ g% I1. D# Z. R% j; p, u
创建client.admin用户、client.bootstrap-osd用户密钥,添加到集群密钥中。
' [6 ~7 V& v5 T: l+ Q$ \; I1 \8 Q
1 N+ `+ A# L( w  t$ g2 D! aceph-authtool --create-keyring /etc/ceph/ceph.client.admin.keyring --gen-key -n client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *'
' c" J, B0 J* T1 @$ rceph-authtool /tmp/ceph.mon.keyring --import-keyring /etc/ceph/ceph.client.admin.keyring
* ?- K. L* Q: R( ?4 W1
; e7 b4 t7 k& m- c5 P" Y( c2/ R# k3 q$ A' C) W& j
ceph-authtool --create-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring --gen-key -n client.bootstrap-osd --cap mon 'profile bootstrap-osd' --cap mgr 'allow r'
# n+ D  v0 `1 [. bceph-authtool /tmp/ceph.mon.keyring --import-keyring /var/lib/ceph/bootstrap-osd/ceph.keyring
2 \0 o) K2 }% ]. Z5 w- m1
* W! Q2 @2 G! \2
, f( `1 L' q, k) q+ N. _2 D使用主机名、主机IP地址、FSID生成monitor map。
5 B! _' E7 Q  E" Q/ M9 {/ o# G2 ?0 ~- H  S0 S  w  l
monmaptool --create --add ceph01 10.40.65.156 --add ceph02 10.40.65.175 --add ceph03 10.40.65.129 --fsid 9bf24809-220b-4910-b384-c1f06ea80728 /tmp/monmap. |% v4 w* N! v% Z
1
6 R0 t. E. \' b! p  t0 K! Y3 Z& T' l初始化并启动monitor服务, R) T% Q8 T9 H& `) O6 F' u
5 D" @6 d& F8 P) r
sudo -u ceph mkdir /var/lib/ceph/mon/ceph-ceph01+ _" l, V- X/ f3 g8 s
chown ceph.ceph -R /var/lib/ceph /etc/ceph /tmp/ceph.mon.keyring /tmp/monmap
! a- @: u8 u8 C) z% T; J# Fsudo -u ceph ceph-mon --mkfs -i ceph01 --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring
& S3 H5 n* U+ ]* p' v2 {; y+ j  wls /var/lib/ceph/mon/ceph-ceph01/
- \5 p- e, F. k( A& ~1 F10 P* [) _& E1 Z' P) U
2
6 x# C* R- x) c' @  x3" ?# e5 b# x  d0 S; d, y
4% R& ]% k' G! ?, x. v
systemctl start ceph-mon@ceph01
# o0 i1 J5 ]; ~; \/ ?systemctl enable ceph-mon@ceph01% M' l8 ]1 U$ V4 Y5 z6 T
systemctl status ceph-mon@ceph01  N: U7 p2 x7 G" j& Z  Z
1
3 ]# K4 `$ T2 {) e9 W2
9 X- g! u# s0 T$ f: R1 g: T8 V$ }3
' k6 }" I5 p7 k同步配置文件、密钥、monmap到其他节点中(Ceph 01执行)
& _$ I) Y) o& |5 m; l, h9 ?
8 Z! J. @1 D8 Q* Z复制ceph.client.admin.keyring、client.bootstrap-osd key、ceph.mon.keyring、monitor map、ceph.conf到另外2个节点6 S+ h' G! d7 `0 ?3 B
* Q' w: Y9 y8 \* I" d0 p5 P' j
scp /etc/ceph/ceph.client.admin.keyring root@ceph02:/etc/ceph/
% V4 B7 b& \, d& k$ V' b6 pscp /etc/ceph/ceph.client.admin.keyring root@ceph03:/etc/ceph/
% s/ K/ V/ N1 f/ I' q0 Y( R1
7 o+ l. S7 ~2 ~; e! D" f2
  H/ j# O2 u; ^8 P  j4 T% m+ Hscp /var/lib/ceph/bootstrap-osd/ceph.keyring root@ceph02:/var/lib/ceph/bootstrap-osd/1 q, U% p( X. }, Z: d: k
scp /var/lib/ceph/bootstrap-osd/ceph.keyring root@ceph03:/var/lib/ceph/bootstrap-osd/
+ _# a" x! s/ c, b+ H& n1
1 r0 s( L- t* `3 V2 Z6 e3 ]9 N2
2 ~3 L7 @4 I. s" r' v# vscp /tmp/ceph.mon.keyring root@ceph02:/tmp/
) m3 W+ `9 Q, u# Q. D/ ^scp /tmp/ceph.mon.keyring root@ceph03:/tmp/
. ?: R. i; O, m. t1# s+ \& K! ~: f4 ~+ r4 f6 M/ E/ G
21 ?6 B/ ]& Y1 `/ c6 S1 |% t0 m
scp /tmp/monmap root@ceph02:/tmp/6 o  ~0 L) M! K( ]. H
scp /tmp/monmap root@ceph03:/tmp/
# a, O# I4 @1 Z" o1
- U# g- K! a( T$ Z5 y  z2
' Q, i0 @" H4 R2 u- xscp /etc/ceph/ceph.conf root@ceph02:/etc/ceph/1 [4 }; U1 Z# s
scp /etc/ceph/ceph.conf root@ceph03:/etc/ceph/
6 w$ g9 J+ p4 e* }2 R* I; m1
* D; @/ h+ f9 G( Q+ o; K  O$ u+ a2
- c. }! [) @1 L- N% I$ X8 F" X启动其他节点的monitor服务(Ceph 02执行)& r6 K1 S0 o9 T; O8 v

" |: K( Y' S$ N! v, `sudo -u ceph mkdir /var/lib/ceph/mon/ceph-ceph02
; K1 b& V- w2 bchown ceph.ceph -R /var/lib/ceph /etc/ceph /tmp/ceph.mon.keyring /tmp/monmap
$ N! p6 n- t3 [5 ]* ]6 f$ ?6 J4 Hsudo -u ceph ceph-mon --mkfs -i ceph02 --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring
8 [3 H' u' W3 d$ H- Q3 u& T5 m% Els /var/lib/ceph/mon/ceph-ceph02/& z( `. ^0 ?" U! ~
1# J6 J4 N/ [' l( U
2
5 c$ J2 ^7 n( o, K5 `* ]3
' r) I6 h5 q, @! T; k% M4
% m( \: @+ ^( S/ S! H- a3 Isystemctl start ceph-mon@ceph02
9 v. @/ p. e3 s/ M. {systemctl enable ceph-mon@ceph02. e8 I; A7 ?/ X& t+ k5 S
systemctl status ceph-mon@ceph02
  v# F+ [; f* S. _1& A) f& n3 H) r- G, S
2( M- _/ N! u! w2 v% R2 Q( t" O
3, U4 U  o( I9 W6 G
启动其他节点的monitor服务(Ceph 03执行)
! y6 _' ~( _1 S1 F7 {* i* l% G: ]4 I; D: ]2 h5 V
sudo -u ceph mkdir /var/lib/ceph/mon/ceph-ceph03
1 e" j9 T7 z& T5 d% D2 _chown ceph.ceph -R /var/lib/ceph /etc/ceph /tmp/ceph.mon.keyring /tmp/monmap
# V2 v- }$ f5 a  i  J4 msudo -u ceph ceph-mon --mkfs -i ceph03 --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring" |# m, L8 [) o% O+ v9 T  h
ls /var/lib/ceph/mon/ceph-ceph03/
5 J% \* g! J4 v  L1! ~' i9 a' P& ?, s
22 _: W3 j! ~6 k( c1 V. j# P7 o( ]
33 ^, h2 t. a& n  r' m
40 y0 f& I) y: F8 R: D9 W
systemctl start ceph-mon@ceph03
8 p9 z& {1 g: i. T. y, Usystemctl enable ceph-mon@ceph03
; K; d+ [- T' ^9 |( lsystemctl status ceph-mon@ceph039 x$ n1 ]/ O( _8 s4 o( C: F/ X
17 f5 Q9 g6 o, E
2! F" d2 f# O5 Q7 V- Z7 l' d8 i
3
. D. g' x, H( n4 e2 J查看当前集群状态(任意节点执行)
9 w8 _, ~7 P4 q4 G/ @9 m6 E2 [( V通过ceph -s命令查询集群状态,可见集群services中3个mon服务已启动。& e/ p2 l% n9 I/ g) M$ n
# ~- p# v8 b! Y$ D+ _+ d
ceph -s3 ]$ G! c' p$ |
> cluster:
- T3 @9 x, B' U$ }  S>                 id:     cf3862c5-f8f6-423e-a03e-beb40fecb74a        
7 `$ {; r: R. G" l>                 health: HEALTH_OK
1 Z8 A! U7 H5 s) T  W& e7 }> * [$ d) }& H5 ]: q0 v+ u- S1 M
> services:        
1 P* d& ?4 N* Q# F8 N, t5 P>                 mon:         3 daemons, quorum ceph03,ceph02,ceph01 (age 12d)         / r# Y$ h( h; R+ P+ K+ n9 B
>                 mgr:                                ' R" @5 @$ g- g
>                 osd: ; A* b- P1 h1 }0 i' S
> # {! M  e8 D& T. s" a4 w) a
> data:6 W* |$ F1 Z$ d4 K. ^
>                  pools:   
7 k8 M, l- `( {6 B3 G6 F8 J  C>                 objects: 8 Z. g2 d. G: d+ p6 U
>                 usage:   
% y0 b3 ]% l" f9 D>                 pgs:2 _: @% L1 i0 O$ f& m2 z: m
1
  R& q4 C, V0 U$ j7 v4 f* v21 u3 ~/ l8 v" \# U* ?' X
3
# f; c. O; m2 ~* z$ P9 `" x/ J/ [2 M5 g49 X+ L# X. t. G: ~' a5 ]9 i5 O$ q
5
8 c' k# ?3 p5 g3 q6! X: |: K0 t1 z* k6 i$ z9 X
7" x; [+ p7 q2 _! U6 D7 |
8
2 U+ Y( _+ I2 K3 y! C& H9, _0 B4 U1 v, t7 N& M* {; m8 T
10  o! |9 K* T4 c* e
11. o+ Y0 g6 w$ X' c1 [  I; U
12% Z  x& i% F& _) |2 F
137 u. _5 O, z. L& H
14/ \3 R' O, {* Y5 O( i
15
" F' y! S$ z3 E- i2 {+ }部署Ceph mon服务(ceph-volume 自动化创建)8 A% {0 U5 `- g# x* L4 h, Z2 D' d
安装Ceph-osd服务程序(所有设备执行)
' L8 T, j. B: ]' c5 tyum install -y ceph-osd, H+ g6 d1 j+ \$ E
1
2 ]- N$ E) z# t: C7 c! e! f$ M初始化osd服务(所有设备执行)
5 I  N5 Z7 X/ g( z! r1 o* }通过fdisk等工具查看磁盘盘符,然后利用ceph-volume工具自动化创建osd服务。9 @; {* x1 w; W1 f
ceph-volume lvm create --data /dev/sda
2 ^& y7 G" n( T' j5 s- Y4 d( d  |ceph-volume lvm create --data /dev/sdb7 |* y' G: Q. j0 N4 \
ceph-volume lvm create --data /dev/sdc
2 L. A4 n; e) F1# b8 h7 s  U1 {* W2 o  V
2! X$ G, k3 E" {$ A# i5 a
3
$ e8 u0 @5 ?8 D: F3 }查看当前集群状态(任意节点执行)5 c4 T8 i5 e4 S' p) l7 |
通过ceph osd tree命令查询集群状态,可见集群services中所有osd服务已启动。
1 G) d/ V0 _$ u; L7 Nceph osd tree: |- z0 T* D* }( w0 ^
> ID CLASS WEIGHT   TYPE NAME       STATUS REWEIGHT PRI-AFF7 I1 M0 q( J# _
> -1       16.36908 root default
8 f9 ~; {3 P8 Q0 F> -3        5.45636     host ceph01# O; ]2 T, L& @6 M
>  0   hdd  1.81879         osd.0       up  1.00000 1.00000
" a. C, x  z) ^# F. \  c, Q>  1   hdd  1.81879         osd.1       up  1.00000 1.00000/ t8 n1 I9 A$ X. t
>  2   hdd  1.81879         osd.2       up  1.00000 1.000006 E% [2 o4 U: B, D! o
> -5        5.45636     host ceph02
% p' q+ y7 F- _+ t/ q  r2 E>  3   hdd  1.81879         osd.3       up  1.00000 1.00000: z2 k. a& G9 S7 z% [) _: r" B/ C
>  4   hdd  1.81879         osd.4       up  1.00000 1.00000
5 n# [! O( L: }1 e3 l>  5   hdd  1.81879         osd.5       up  1.00000 1.00000& H" B+ ~/ K; x4 e  y* c
> -7        5.45636     host ceph03
. T5 V$ u% `) Y$ r' E>  6   hdd  1.81879         osd.6       up  1.00000 1.00000
' Z/ [9 n+ [& y1 B3 K; T>  7   hdd  1.81879         osd.7       up  1.00000 1.00000
2 H7 J/ A  n+ [7 z>  8   hdd  1.81879         osd.8       up  1.00000 1.00000
9 y: \" R  N6 |: c1- m5 C& J: b- S' l* @/ _  C
2
: q( k+ M2 O; ?; t6 S+ S7 ~" C; g# J9 {2 f* ]3
+ ^8 m6 `2 d5 |9 G- }: v0 i: `: f' d4
, Y# x6 N5 T: S) Q% S5
6 N# g- E4 x+ ]6
7 f0 `( B5 Z! z4 k# S7
7 g. s: I  T0 [. f! F8, o6 f" ~: F4 ~. t
9
) j: b" _7 W' D3 M! [10+ H; q. g& V0 z) {4 o# {
11+ @/ q4 @: b* w0 k& @$ c9 x5 G
12, ]0 `, g' @9 a( Z2 g! O8 T
13
* _) U: o" a& X142 n. I" q9 R9 {2 w) Y
15
* J9 j+ s/ B  A! B部署Ceph mgr服务并开启Dashboard' F2 s9 W1 c6 @( \
安装Ceph-mgr服务程序(所有设备执行)* }# ]8 R1 b5 b8 Z' R1 z8 J: t
yum install -y ceph-mgr! t: _- e; J1 q, U) q' R% z
1. d/ y: T0 \' R5 S' ~3 p6 Z. A, i: Y
初始化并启动主MGR服务(Ceph01执行)
) C5 ^1 W, g1 j5 J9 n! E/ Umkdir -p /var/lib/ceph/mgr/ceph-ceph017 E$ L7 R8 a2 v: W6 H: g* ~
chown ceph.ceph -R /var/lib/ceph
( S' z/ F$ F, ^4 @1 M. Tceph-authtool --create-keyring /etc/ceph/ceph.mgr.ceph01.keyring --gen-key -n mgr.ceph01 --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *'
: o$ \: X/ `$ |8 P/ G* j5 Eceph auth import -i /etc/ceph/ceph.mgr.ceph01.keyring
/ L% @3 m- Y8 f+ R( |* bceph auth get-or-create mgr.ceph01 -o /var/lib/ceph/mgr/ceph-ceph01/keyring% X! s' Z% y% M6 N; }
1; P; n0 H+ E  d
24 J0 ]. }9 z1 }
3
0 R7 u7 L  K1 u" V: z4
! I2 I* G; t! S% B' d# D5+ P$ i- @- z# v0 @
systemctl start ceph-mgr@ceph01
9 K, D! \/ a( h' W  vsystemctl enable ceph-mgr@ceph01
1 }2 x/ H3 R3 J- P$ r8 L+ H5 Isystemctl status ceph-mgr@ceph01
' L# E- r; M$ P) X# n# G1
; p( M* H) m8 _0 D1 P2# p1 b4 O5 n" U: h
3
  C. O% U) C2 Z# R: T6 |初始化并启动从MGR服务(Ceph02执行)7 Z/ X: A+ h" ?5 }$ M. z
mkdir -p /var/lib/ceph/mgr/ceph-ceph02
% S0 H/ N5 p$ ]" mchown ceph.ceph -R /var/lib/ceph
7 g6 C8 b' j) S4 Fceph-authtool --create-keyring /etc/ceph/ceph.mgr.ceph02.keyring --gen-key -n mgr.ceph02 --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *'
; e) H9 G4 [7 Z) }' O, g' Tceph auth import -i /etc/ceph/ceph.mgr.ceph02.keyring
) j- ~8 c* ?; A# `# ^5 Vceph auth get-or-create mgr.ceph02 -o /var/lib/ceph/mgr/ceph-ceph02/keyring
( a, B! w+ M) H" n1 m4 e" L( N4 [6 M1
* i0 d- [$ ]! d- q4 _. N8 [2
. {& u9 g; Y- R$ {  W. S: N3, ^$ P0 x/ H. `: I0 X" v
4
6 \2 r! @" D' M# R( H( l5
. b. h0 Z9 O% b3 v* `6 [systemctl start ceph-mgr@ceph02; U. u9 J2 ~8 z* \
systemctl enable ceph-mgr@ceph02) ^3 g% S0 e# N& e1 H
systemctl status ceph-mgr@ceph02
( R' ~& W" V8 W+ D/ C1
) W: f8 g5 ^$ W1 J/ q8 M7 g# K2: G! M$ x* E3 N0 y& Y+ L
3
" Q& S+ @( D) i  \. A* d" R4 {初始化并启动从MGR服务(Ceph03执行)2 s# d6 f; o0 e1 K% `
mkdir -p /var/lib/ceph/mgr/ceph-ceph03; o$ u- v# c$ X
chown ceph.ceph -R /var/lib/ceph6 F5 t, B+ [+ W% T7 H% L
ceph-authtool --create-keyring /etc/ceph/ceph.mgr.ceph03.keyring --gen-key -n mgr.ceph03 --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *'
5 ]# y1 @+ z5 P% f* hceph auth import -i /etc/ceph/ceph.mgr.ceph03.keyring
5 o8 O# C* [; ?ceph auth get-or-create mgr.ceph03 -o /var/lib/ceph/mgr/ceph-ceph03/keyring0 S6 ?; D, h( m8 b1 D  Q. ]( j
1
0 v, ^6 j- M8 @9 w, k$ O2* o/ T4 o6 b- [5 _* {1 P) N
3
1 W2 b8 m6 Z8 m2 G1 a8 P4
, M1 f& z1 M" S' F. g, S, @* c7 C5/ t* x% e" Z% c, s, \! K4 m* @
systemctl start ceph-mgr@ceph03" `( Q5 Y1 A0 A1 |' z
systemctl enable ceph-mgr@ceph03& w$ `$ ~+ V7 A" S. U6 B
systemctl status ceph-mgr@ceph03
; u, d9 S1 N0 `3 q8 W0 t1
+ e+ E9 g9 b& _6 R2
. b8 c# c! v: h8 A3
6 h0 [/ w- n  @$ ^  ^. V) W( M查看当前集群状态(任意节点执行)
5 [# B5 l, g) W$ ?2 w0 F& Y: b1 N通过ceph -s命令查询集群状态,可见集群services中3个mgr服务已启动。
& I* Z  z: O0 ^) l! b8 [3 cceph -s% L/ x1 I: n2 ^, N0 _3 v4 j6 d
> cluster: & j' w9 a5 ^( Q4 h" a
>                 id:     cf3862c5-f8f6-423e-a03e-beb40fecb74a         & A. e7 Z. o0 }- P8 m* A2 t
>                 health: HEALTH_OK; D5 v/ ]7 D+ }0 M1 s+ w7 b1 R% n1 C
> 7 }4 v: z9 c( P2 {- [  Y" j1 d! K
> services:        
$ V$ Z2 l/ [4 D2 n  c+ y3 t>                 mon:         3 daemons, quorum ceph03,ceph02,ceph01 (age 12d)         , Y  ^9 b& }: ~: R
>                 mgr:        ceph01(active, since 3w), standbys: ceph01、ceph02                       
/ Z' i5 e7 G' H4 ?+ i, \. G>                 osd:
; f  Q/ t  @; N# F3 w5 F$ a>
/ _: S4 ?2 \8 N" R> data:
8 V$ ^# c: c# C>                  pools:   + o$ r- R% u/ }0 F+ m5 B
>                 objects:
: M4 j/ S* S& t3 i5 S>                 usage:   + z9 o/ M6 Y5 y4 n$ w7 T
>                 pgs:* D* a) i4 [' Z1 k
18 w2 j( w0 u- |
2" T& C  |7 K5 R# f  M& [& `
3
$ W7 t* \( Q7 _% P0 \* x6 b4: X4 [/ C! x6 ]4 j, ]
5
  G, @4 E9 s7 x; v6
: I* J$ p- i; o; S! Q, E7
% I; e% E7 O9 I' ?# n. u, F' S8
/ ]6 a8 R7 b+ J9
4 h9 }4 x! Z9 X4 N# j, z) O5 \10
% k: k: Z$ d8 s) j6 Z6 r! r11
% @1 b8 d* n! c. S12
# e9 Z  c1 N  L+ S2 w+ U13* Y# C! Z) ~, o3 N& b  N# V# D+ F
14& a, g- p0 o$ `+ M; B" D8 G# U2 h2 I
15
' ?: y6 `! U& ~. D, i) Y4 V2 m使能Dashboard访问功能(任意节点执行), W2 [5 \7 O9 ~  ]/ T' O7 ~) X
开启mgr dashboard功能3 x5 r* {) c# L* B7 P: f) [5 u
ceph mgr module enable dashboard8 b8 ]/ b$ F6 T
1
6 I& s, |% n1 U) m! W$ u  k0 F生成并安装自签名的证书
7 W+ M% U0 G5 s& X6 Fceph dashboard create-self-signed-cert
  D, ]9 l* c; ^; q  s) q* `" E1
0 L9 S' {$ Q; W! h5 c: m4 E配置dashboard; C' F5 _  ^: q
ceph config set mgr mgr/dashboard/server_addr 10.40.65.148
, W' N  @4 P; ?! f/ x8 [ceph config set mgr mgr/dashboard/server_port 8080
% j/ t0 a) z& \# v+ uceph config set mgr mgr/dashboard/ssl_server_port 8443& H0 U* A/ F5 t0 j2 k3 W* U% y0 |
19 d5 D7 }6 F$ e5 O* H5 j+ z. k
2% C0 ^5 n' Z& K
30 K4 K5 y5 w+ E: A( v
创建一个dashboard登录用户名密码6 N$ O" c# a: N: G) l6 S$ z
echo '123456' > password.txt
, ]1 I! }7 a9 z7 _ceph dashboard ac-user-create admin  administrator -i password.txt
$ A' Q/ Z7 C4 X4 H# p17 |9 S* P  v$ E7 [
2( x2 O# s  l" ^1 G3 s
查看服务访问方式; n! x% z8 W) g: I( U( F" X/ ?0 v" K
ceph mgr services. ?4 o; p9 y/ g2 f9 J- P! L
1; I$ p6 v# b4 Q
通过web访问Ceph Dashboard,用户名密码为admin/123456- v8 r  {/ u& f$ U: f! Z
https://10.40.65.148:8443
您需要登录后才可以回帖 登录 | 注册

本版积分规则

返回首页|Archiver|手机版|小黑屋|易陆发现技术论坛 ( 蜀ICP备2026014127号-1 )

GMT+8, 2026-6-11 23:59 , Processed in 0.018178 second(s), 23 queries .

Powered by Discuz! X5.0

© 2001-2026 Discuz! Team.

快速回复 返回顶部 返回列表