[WRN] PG_DEGRADED: Degraded data redundancy: 64 pgs undersized

admin

   [WRN] PG_DEGRADED: Degraded data redundancy: 64 pgs undersized
: T& n  S3 n) T& O- I3 E( T
health: HEALTH_WARN  too many PGs per OSD (342 > max 250)


8 z; z0 r5 p' s5 ^ceph --admin-daemon /var/run/ceph/4c1f752a-ed1a-11eb-8ce5-0025908471d6/ceph-mon.controller.asok config get mon_max_pg_per_osd
{
    "mon_max_pg_per_osd": "1000"
}

ceph --admin-daemon /var/run/ceph/4c1f752a-ed1a-11eb-8ce5-0025908471d6/ceph-osd.0.asok config get osd_max_pg_per_osd_hard_ratio
1 x- ^+ ]9 T2 U{
# p. G* i6 J8 w: S" j# k    "osd_max_pg_per_osd_hard_ratio": "3.000000"
3 q0 F/ o3 g& p  v$ k! c3 P}

调整pg数即可。

1 V2 v4 c0 @$ Z9 ^7 {$ @

admin

[global]

0 N2 ]. G2 P6 f& F: x* Smon_max_pg_per_osd = 800                              # < depends on you amount of PGs
osd_max_pg_per_osd_hard_ratio = 10                # < default is 2, try to set at least 5. It will be
6 ~! j" L! l( pmon_allow_pool_delete = true                           # without it you can't remove a pool

admin

osd_max_pg_per_osd_hard_ratio = 10

admin

[root@controller ~]# ceph health detail
5 B! |% m* k4 ^; s2 A) b. h, hHEALTH_WARN Degraded data redundancy: 64 pgs undersized
* n: E" C2 A6 X1 o+ P. N[WRN] PG_DEGRADED: Degraded data redundancy: 64 pgs undersized
, I5 S1 u  [' S" d9 i6 p% u6 Y5 m    pg 9.d is stuck undersized for 115s, current state active+undersized, last acting [2,1]
    pg 9.e is stuck undersized for 115s, current state active+undersized, last acting [2,1]
( Z7 [4 e/ G8 O! c    pg 9.f is stuck undersized for 115s, current state active+undersized, last acting [2,1]
5 D  y8 z4 s! c0 K    pg 9.10 is stuck undersized for 115s, current state active+undersized, last acting [2,0]
    pg 9.11 is stuck undersized for 115s, current state active+undersized, last acting [2,0]
    pg 9.12 is stuck undersized for 115s, current state active+undersized, last acting [2,1]
    pg 9.13 is stuck undersized for 115s, current state active+undersized, last acting [2,0]
, L- n. n  l$ Y3 ~  h: }    pg 9.14 is stuck undersized for 115s, current state active+undersized, last acting [1,2]
/ J  u. f  a4 u0 G    pg 9.15 is stuck undersized for 115s, current state active+undersized, last acting [2,1]
/ F: S3 R9 t  b* R9 w$ [; y% t    pg 9.16 is stuck undersized for 115s, current state active+undersized, last acting [2,1]
5 y9 h- f' @( C4 |    pg 9.17 is stuck undersized for 115s, current state active+undersized, last acting [2,1]
1 h( c" V4 r$ a! f- [( ~9 `* Z: G    pg 9.18 is stuck undersized for 115s, current state active+undersized, last acting [2,0]
    pg 9.19 is stuck undersized for 115s, current state active+undersized, last acting [0,2]
    pg 9.1a is stuck undersized for 115s, current state active+undersized, last acting [1,2]
    pg 9.1b is stuck undersized for 115s, current state active+undersized, last acting [0,2]
    pg 9.1c is stuck undersized for 115s, current state active+undersized, last acting [0,2]
. a9 G8 v6 M+ A! B    pg 9.1d is stuck undersized for 115s, current state active+undersized, last acting [0,2]
9 l0 S" ~) V- t: Y5 Z    pg 9.1e is stuck undersized for 115s, current state active+undersized, last acting [0,2]
    pg 9.1f is stuck undersized for 115s, current state active+undersized, last acting [1,2]
3 H6 k& _& p: q0 O: z, W: K    pg 9.20 is stuck undersized for 115s, current state active+undersized, last acting [0,2]
    pg 9.21 is stuck undersized for 115s, current state active+undersized, last acting [0,2]
    pg 9.22 is stuck undersized for 115s, current state active+undersized, last acting [2,1]
6 j) U/ T2 C1 ~- Y6 d2 f' l) y5 U    pg 9.23 is stuck undersized for 115s, current state active+undersized, last acting [0,2]
8 w0 w8 E3 s& X9 L( V    pg 9.24 is stuck undersized for 115s, current state active+undersized, last acting [1,2]
1 W" b7 l9 l- G5 ]* J5 |  j    pg 9.25 is stuck undersized for 115s, current state active+undersized, last acting [0,2]
    pg 9.26 is stuck undersized for 115s, current state active+undersized, last acting [2,0]
    pg 9.27 is stuck undersized for 115s, current state active+undersized, last acting [0,2]
9 G) P9 A% Q* Z$ V0 I    pg 9.28 is stuck undersized for 115s, current state active+undersized, last acting [2,0]
    pg 9.29 is stuck undersized for 115s, current state active+undersized, last acting [2,0]
    pg 9.2a is stuck undersized for 115s, current state active+undersized, last acting [1,2]
: h0 f( a2 k" C    pg 9.2b is stuck undersized for 115s, current state active+undersized, last acting [2,0]
5 W4 O, y, ]' W! Z) k- x    pg 9.2c is stuck undersized for 115s, current state active+undersized, last acting [1,2]
/ S/ ?4 Q& Q: F6 _# f    pg 9.2d is stuck undersized for 115s, current state active+undersized, last acting [0,2]
    pg 9.2e is stuck undersized for 115s, current state active+undersized, last acting [2,0]
6 E- C6 w4 R6 [# Y; T    pg 9.2f is stuck undersized for 115s, current state active+undersized, last acting [1,2]
8 P! N7 @6 |: [) S    pg 9.30 is stuck undersized for 115s, current state active+undersized, last acting [0,2]
/ E5 h- ?. g& |7 s    pg 9.31 is stuck undersized for 115s, current state active+undersized, last acting [2,1]
    pg 9.32 is stuck undersized for 115s, current state active+undersized, last acting [2,0]
    pg 9.33 is stuck undersized for 115s, current state active+undersized, last acting [2,0]
    pg 9.34 is stuck undersized for 115s, current state active+undersized, last acting [2,1]
    pg 9.35 is stuck undersized for 115s, current state active+undersized, last acting [1,2]
4 J5 M( E$ Y& c. Y! `    pg 9.36 is stuck undersized for 115s, current state active+undersized, last acting [1,2]
* r$ i6 {* _/ l2 D. s4 L/ l    pg 9.37 is stuck undersized for 115s, current state active+undersized, last acting [2,0]
    pg 9.38 is stuck undersized for 115s, current state active+undersized, last acting [0,2]
* ^2 P  M  {9 [; H4 V1 g& q    pg 9.39 is stuck undersized for 115s, current state active+undersized, last acting [2,0]
    pg 9.3a is stuck undersized for 115s, current state active+undersized, last acting [0,2]
    pg 9.3b is stuck undersized for 115s, current state active+undersized, last acting [1,2]
    pg 9.3c is stuck undersized for 115s, current state active+undersized, last acting [2,0]
/ [# N1 c4 V9 f2 N3 B: r7 R    pg 9.3d is stuck undersized for 115s, current state active+undersized, last acting [2,1]
( T8 C! I, o4 Q% E& {! T. J    pg 9.3e is stuck undersized for 115s, current state active+undersized, last acting [1,2]
    pg 9.3f is stuck undersized for 115s, current state active+undersized, last acting [1,2]
[root@controller ~]# ceph pg dump |grep remapped
; f: l  s8 e8 t& H4 M' P; A' jdumped all
: J$ }! g7 N4 F: {' w; T[root@controller ~]# ceph osd pool ls
images
6 q  N7 E# S1 K' M1 l7 l( o[root@controller ~]# ceph osd pool ls detail
/ |2 q/ e2 N  Z  j4 s- @pool 9 'images' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode on last_change 7458 flags hashpspool stripe_width 0 application rbd

总结：

$ B& B1 d% u' I( P: z' r6 Q原因大致可以定位为是副本数设置有问题，因为本环境就两台设备，副本数size设置为3，导致同步失败。根据设备数设置相应的副本数，否者就会报此类错误。