ceph health detail 显示pgs not active

admin

单节点安装完ceph后，只创建了两个OSD，导致状态是34.000% pgs not active，用python rados往pool存数据一直卡着，
6 R$ J' o! G2 t; ~" A6 ?
! |$ V( ]0 k7 q/ {% n  D8 X5 }
  Y" }! Y% |" Q; S将crush map导出
ceph osd getcrushmap -o /usr/src/crushmap
输出，这一步的输出和osd数量有关系
0 V( z" d. h, \( a: E将crush map反编译为可读模式
3 j7 z+ x! O0 [: Q  L: Z. Ccrushtool -d /usr/src/crushmap -o /usr/src/crushmap.txt
修改curshmap，主要是修改OSD级别


% O  ?5 E' I5 l  a
7 |9 u" b' i4 I! R7 y! I, G: s& Hvim   /usr/src/crushmap.txt
找到step chooseleaf firstn 0 type host
改成step chooseleaf firstn 0 type osd

保存退出：
4 }) k6 h5 i0 H+ S" _8 _+ @或者通过sed 修改：
  n: W1 I: A# Y0 Fsed -i ‘s/step chooseleaf firstn 0 type host/step chooseleaf firstn 0 type osd/’ /usr/src/crushmap.txt

+ r  K# k' J& h
, A' G5 Y' Q* `7 J9 [# \9 m
grep ‘step chooseleaf’ /usr/src/crushmap.txt
输出step chooseleaf firstn 0 type osd
/ o$ m4 E! E; r& tcrushtool -c /usr/src/crushmap.txt -o /usr/src/crushmap-new
将修改过的crush map导入集群
. J! c* U; w/ r- z6 q0 E! hceph osd setcrushmap -i /usr/src/crushmap-new
输出数值比导出数值大就对了。

  p, t3 _1 T  |! Y. T1 B/ E

) w5 o1 \/ ?: Z! J& C9 h现在再看ceph -s，pg数状态
- ?& |0 K& j: s* B) D1 h

admin

我现在再运行代码，没问题，数据存进去了，对象名是/root/ceph-test/abc.txt
, x' I5 d7 t% V
[root@controller ceph-test]# python pooltest.py
Object contents = /root/ceph-test/abc.txt
[root@controller ceph-test]# rados -p data2 ls
9 I+ v4 f; U5 l/root/ceph-test/abc.txt

admin

1、没有创建pool之前

复制代码
[root@cluster9 ceph-cluster]# ceph -s
. _. l# c8 b+ s! D- f  cluster:
    id:     d81b3ce4-bcbc-4b43-870e-430950652315
8 F. w" _: B9 `! m+ b5 z# o    health: HEALTH_OK

5 a6 Y9 h6 W! h% Q  services:
8 ~( n! r' C3 j( e    mon: 1 daemons, quorum cluster9
    mgr: cluster9(active)
    osd: 3 osds: 3 up, 3 in

  w6 x  P3 O+ G5 V  data:
$ Z4 X0 {! z3 G7 a    pools:   0 pools, 0 pgs
: B$ A( x3 v+ l. w  L. r6 \% ?    objects: 0 objects, 0B
    usage:   3.06GiB used, 10.9TiB / 10.9TiB avail
    pgs:     
复制代码
0 E% h6 B% j9 c6 v7 u3 L2、创建pool后

2 X7 ?1 r7 d- \  u/ P5 p2 o' K复制代码
[root@cluster9 ceph-cluster]# ceph -s
. S. [8 V# }0 t& h; z; {  cluster:
    id:     d81b3ce4-bcbc-4b43-870e-430950652315
. C5 V# D( S; f$ Q/ t    health: HEALTH_OK

  services:
  K7 L% x+ l) @- t' G* V( `0 G    mon: 1 daemons, quorum cluster9
    mgr: cluster9(active)
$ X0 E$ W9 M5 ?4 i    osd: 3 osds: 3 up, 3 in

5 t0 C0 p2 ^# C! @$ b& y  data:
    pools:   1 pools, 128 pgs
9 U9 l# |7 l8 h4 n8 q    objects: 0 objects, 0B
6 r4 _# S  A3 L+ f0 Y* k0 [    usage:   3.06GiB used, 10.9TiB / 10.9TiB avail
    pgs:     100.000% pgs not active
             128 undersized+peered
+ ~0 A  X. w' N2 E, @复制代码
3、修改osd级别
9 }( r/ S8 q* g8 S/ o5 e/ u& X% X5 E
' V4 ]" ?$ f1 y$ i) W- g6 ?复制代码
- G0 ^4 l. X/ ~+ N[root@cluster9 ceph-cluster]# cd /etc/ceph/
* {) u+ n; {0 e3 w; j4 a) v[root@cluster9 ceph]# ceph osd getcrushmap -o /etc/ceph/crushmap
18
[root@cluster9 ceph]# crushtool -d /etc/ceph/crushmap -o /etc/ceph/crushmap.txt
2 F$ Z& \5 U& z% n4 Y& I[root@cluster9 ceph]# sed -i 's/step chooseleaf firstn 0 type host/step chooseleaf firstn 0 type osd/' /etc/ceph/crushmap.txt
[root@cluster9 ceph]# grep 'step chooseleaf' /etc/ceph/crushmap.txt
% C! V4 x% ~0 g9 ~# Y    step chooseleaf firstn 0 type osd
5 f+ A6 j2 F' e8 D[root@cluster9 ceph]# crushtool -c /etc/ceph/crushmap.txt -o /etc/ceph/crushmap-new
) t' |/ R2 Q# L6 T) x[root@cluster9 ceph]# ceph osd setcrushmap -i /etc/ceph/crushmap-new
19
0 I# _& T+ A/ [  i# U) V: H7 y8 L复制代码
+ G/ i, B2 f4 k, d; j- K5 k4、再次查看ceph状态
( m0 X/ x: I) C0 ?' ^
复制代码
[root@cluster9 ceph]# ceph -s
  cluster:
4 z0 E% Q4 ]2 ^4 f2 e    id:     d81b3ce4-bcbc-4b43-870e-430950652315
( `8 X. y+ h: w    health: HEALTH_OK

7 A) s" A2 ~0 Z) F  services:
    mon: 1 daemons, quorum cluster9
6 ?' n' P) a; U) n5 e    mgr: cluster9(active)
    osd: 3 osds: 3 up, 3 in
2 k) q! k" Y9 v. }
% n" o- V8 h* Y  data:
; E/ o4 P: `' T, q0 s1 C" M    pools:   1 pools, 128 pgs
0 o) c+ T% ~' l) G% l    objects: 0 objects, 0B
; O3 v3 A* C/ C/ K( W0 R7 ~    usage:   3.06GiB used, 10.9TiB / 10.9TiB avail
    pgs:     128 active+clean
复制代码