ceph health detail 显示pgs not active

admin

单节点安装完ceph后，只创建了两个OSD，导致状态是34.000% pgs not active，用python rados往pool存数据一直卡着，
: V9 m9 K& s  A9 c, d

$ Q) j. b1 v0 O4 v- k, I- O+ q7 }将crush map导出
( Z8 R8 @8 ^3 y, n. u- [/ ~7 Gceph osd getcrushmap -o /usr/src/crushmap
/ D" D* k" u4 Y; @: g输出，这一步的输出和osd数量有关系
' ~" A- X. H# s9 ?0 r" |% y将crush map反编译为可读模式
% X- H0 t4 D% x7 |0 f. Fcrushtool -d /usr/src/crushmap -o /usr/src/crushmap.txt
; x& d7 D$ g7 u2 j$ l7 [修改curshmap，主要是修改OSD级别

. i- B# U& [6 I5 b/ p% b1 h( g
+ w* K+ |7 a% d% f' Y" t  d- v- e
vim   /usr/src/crushmap.txt
找到step chooseleaf firstn 0 type host
改成step chooseleaf firstn 0 type osd
; w# A: B' x' q
" C0 [8 }. t# s' J& C1 b. Z保存退出：
或者通过sed 修改：
' ?% B7 T9 Q! P+ g9 J% H, m" }sed -i ‘s/step chooseleaf firstn 0 type host/step chooseleaf firstn 0 type osd/’ /usr/src/crushmap.txt


3 F/ N" H  N3 l( a# u8 f0 ?
5 Z; X2 f7 d( K; M& e. q5 O& J( e9 }  hgrep ‘step chooseleaf’ /usr/src/crushmap.txt
输出step chooseleaf firstn 0 type osd
crushtool -c /usr/src/crushmap.txt -o /usr/src/crushmap-new
/ k4 w& |% R/ q8 z! {3 ~9 F将修改过的crush map导入集群
! X" S+ e/ _. rceph osd setcrushmap -i /usr/src/crushmap-new
6 ~5 I: W! o. ]  i6 [) H7 Q  Z# o输出数值比导出数值大就对了。
- _! I% a) Q$ F5 p* Y6 ]
0 m: h, x* s0 z( f) k% O

现在再看ceph -s，pg数状态
1 h, T5 ?% Z6 u# \' {$ {1 B

: o* z/ n. x) ?% a

admin

我现在再运行代码，没问题，数据存进去了，对象名是/root/ceph-test/abc.txt

[root@controller ceph-test]# python pooltest.py
Object contents = /root/ceph-test/abc.txt
4 H. f; w- \& @( Q2 {0 i8 {[root@controller ceph-test]# rados -p data2 ls
/root/ceph-test/abc.txt

admin

1、没有创建pool之前

( B- v( ?0 ^/ B) I复制代码
[root@cluster9 ceph-cluster]# ceph -s
. U% y6 n! G' I! r$ e  cluster:
    id:     d81b3ce4-bcbc-4b43-870e-430950652315
1 |0 Y" ^+ W# Y' F6 _    health: HEALTH_OK
/ b. D) x9 e" J5 ]3 U
  services:
    mon: 1 daemons, quorum cluster9
    mgr: cluster9(active)
    osd: 3 osds: 3 up, 3 in
1 I# o4 X/ Q0 z$ U! X
  data:
    pools:   0 pools, 0 pgs
    objects: 0 objects, 0B
    usage:   3.06GiB used, 10.9TiB / 10.9TiB avail
    pgs:     
2 y! G* Y4 t6 @) G8 q8 c' J复制代码
  f  G. X* I8 E: l2、创建pool后
9 x: t5 p* C- U( k  p, E
0 \- U: x. L, u# G复制代码
  M+ T% i, r# g* c[root@cluster9 ceph-cluster]# ceph -s
8 M5 t4 g% w' o5 i$ i  cluster:
    id:     d81b3ce4-bcbc-4b43-870e-430950652315
* a! E9 \: S4 i& F  I, P1 i/ N! r    health: HEALTH_OK

& \) w& @. I8 F$ q  services:
    mon: 1 daemons, quorum cluster9
$ L8 a; N3 v" ]. x    mgr: cluster9(active)
    osd: 3 osds: 3 up, 3 in

+ r; C: [, Z' o- b% ?$ d: K( i6 b  data:
    pools:   1 pools, 128 pgs
    objects: 0 objects, 0B
    usage:   3.06GiB used, 10.9TiB / 10.9TiB avail
9 T: D" Y7 v* ]$ d' r6 C( @4 z6 D    pgs:     100.000% pgs not active
             128 undersized+peered
复制代码
3、修改osd级别

复制代码
9 i" z" }+ N% |. Z2 t% g) Z[root@cluster9 ceph-cluster]# cd /etc/ceph/
[root@cluster9 ceph]# ceph osd getcrushmap -o /etc/ceph/crushmap
18
5 ?* e; `5 {) P* v[root@cluster9 ceph]# crushtool -d /etc/ceph/crushmap -o /etc/ceph/crushmap.txt
[root@cluster9 ceph]# sed -i 's/step chooseleaf firstn 0 type host/step chooseleaf firstn 0 type osd/' /etc/ceph/crushmap.txt
[root@cluster9 ceph]# grep 'step chooseleaf' /etc/ceph/crushmap.txt
    step chooseleaf firstn 0 type osd
[root@cluster9 ceph]# crushtool -c /etc/ceph/crushmap.txt -o /etc/ceph/crushmap-new
7 o! F  O3 w3 a[root@cluster9 ceph]# ceph osd setcrushmap -i /etc/ceph/crushmap-new
19
% f1 d* r7 m- E( f2 V; a  J2 {  o% P* z复制代码
. U' H% f# G6 J. }! c4、再次查看ceph状态

复制代码
* i# k6 D9 h# m2 D) p! d[root@cluster9 ceph]# ceph -s
, Y# z/ \" }$ f% f( h  cluster:
( g, Y$ D7 P* d' A( a    id:     d81b3ce4-bcbc-4b43-870e-430950652315
    health: HEALTH_OK
% x7 |* h7 |- V7 G7 }# N
  services:
    mon: 1 daemons, quorum cluster9
    mgr: cluster9(active)
    osd: 3 osds: 3 up, 3 in
; y6 }% L* [7 B2 x* j: r* y- g
! w$ S1 `2 M: v+ ^) U  data:
3 ]3 e3 x: r3 S/ W2 @; E( C    pools:   1 pools, 128 pgs
    objects: 0 objects, 0B
    usage:   3.06GiB used, 10.9TiB / 10.9TiB avail
    pgs:     128 active+clean
复制代码