找回密码
 注册
查看: 2121|回复: 1

HEALTH_WARN Degraded data redundancy ... has slow ops

[复制链接]

1

主题

0

回帖

12

积分

管理员

积分
12
QQ
发表于 2021-10-25 17:00:42 | 显示全部楼层 |阅读模式
HEALTH_WARN Degraded data redundancy: 822/429494 objects degraded (0.191%), 115 pgs degraded; 154 slow ops, oldest one blocked for 389 sec, mon.ceph1 has slow ops
: Y4 T/ C" M" a' w! B1 b2 hPG_DEGRADED Degraded data redundancy: 822/429494 objects degraded (0.191%), 115 pgs degraded
  s0 x/ k3 \) L* r    pg 8.473 is active+undersized+degraded, acting [18,9]3 w& n/ f% n3 `0 ]# z& e
    pg 8.483 is active+undersized+degraded, acting [9,20]7 |" f7 E5 v+ ^. o# X+ y: L
    pg 8.48a is active+undersized+degraded, acting [11,21]! F: K" [3 Q, I! ]0 f
    pg 8.49d is active+undersized+degraded, acting [23,6]  Z  ^6 f' D" Z3 b$ W
    pg 8.4a7 is active+undersized+degraded, acting [11,20]) D; [4 s' y( y; N
    pg 8.4a8 is active+undersized+degraded, acting [8,20]
9 ?1 d/ N1 z: ^    pg 8.4a9 is active+undersized+degraded, acting [18,8]
2 U- I$ }' b: u+ N/ T    pg 8.4b2 is active+undersized+degraded, acting [11,21]8 t: n/ t9 {# I; U4 ]6 }# P
    pg 8.4c8 is active+undersized+degraded, acting [7,22]
5 \! U0 ^" U- a    pg 8.4e4 is active+undersized+degraded, acting [9,20]7 p+ ^, @, s9 Z/ a" V
    pg 8.4e7 is active+undersized+degraded, acting [18,11]" |8 B$ ~: D) t. u
    pg 8.4f2 is active+undersized+degraded, acting [23,8]
2 l8 k* z+ M) D# J3 T    pg 8.4fb is active+undersized+degraded, acting [18,7]  I! {' ]8 A0 X: C; z/ o+ Y9 g
    pg 8.505 is active+undersized+degraded, acting [8,21]- X2 U* b0 I2 Q; N4 i: S
    pg 8.54e is active+undersized+degraded, acting [8,22]. I" t3 X" _) _4 y
    pg 8.566 is active+undersized+degraded, acting [23,10]
5 c  d  [( p5 C. C6 K    pg 8.567 is active+undersized+degraded, acting [8,19]
. j: q/ A" f# k( I$ T- ^    pg 8.57c is active+undersized+degraded, acting [23,9]
* G- n2 A6 F1 i2 }, h    pg 8.57d is active+undersized+degraded, acting [8,22]
+ s# {0 K  i, h# N    pg 8.596 is active+undersized+degraded, acting [23,8]
% f, Z: i# q5 n0 R4 a5 W3 m7 e4 B! q: u    pg 8.5b3 is active+undersized+degraded, acting [9,18]
9 D0 v7 W. X6 x/ E$ I    pg 8.5fc is active+undersized+degraded, acting [8,19]
/ c0 p/ x2 n% v8 p5 I3 ?    pg 8.606 is active+undersized+degraded, acting [8,19]
5 Z4 z/ J7 _, ?, E5 a    pg 8.627 is active+undersized+degraded, acting [11,22]! m/ y( z* B! x, l
    pg 8.636 is active+recovery_wait+degraded, acting [15,21,10]) o1 s/ M3 X' y
    pg 8.651 is active+undersized+degraded, acting [23,6]" O7 F. }" o% l+ d& d
    pg 8.65b is active+undersized+degraded, acting [11,21]
+ ^# p, Q* v2 o' m    pg 8.666 is active+undersized+degraded, acting [23,6]( x7 ~' {+ o6 |% s0 ?& t
    pg 8.679 is active+undersized+degraded, acting [11,21]6 _: h0 c1 i+ `9 _' ?2 f
    pg 8.6a3 is active+undersized+degraded, acting [7,22]
4 ~; G& P' `4 A: }6 p8 N, c    pg 8.6c3 is active+undersized+degraded, acting [23,6]
8 y( h# n2 g; t7 a! D7 T    pg 8.6e1 is active+undersized+degraded, acting [8,18]. g# v4 ^1 ^5 M; l- |+ k4 C/ D
    pg 8.6ed is active+undersized+degraded, acting [8,22]! R# \2 [! a6 x- E2 V; A5 `
    pg 8.701 is active+undersized+degraded, acting [7,19], \6 g4 a0 _6 y' e  W* ^" X
    pg 8.724 is active+undersized+degraded, acting [23,10]5 m. [$ u/ T' y3 T$ p8 a$ X$ Q
    pg 8.728 is active+undersized+degraded, acting [18,10]
  g+ [/ k" q$ v( V    pg 8.72f is active+undersized+degraded, acting [7,23]# c8 I! e# t) L' [
    pg 8.732 is active+undersized+degraded, acting [9,18]# ~* O, F$ Z0 E6 f' Z) P
    pg 8.739 is active+undersized+degraded, acting [9,22]" x# s; V9 P! O4 K1 ~; F  s8 H7 L
    pg 8.76a is active+undersized+degraded, acting [11,21]
$ ^" t4 s5 W6 Z( S" W    pg 8.770 is active+undersized+degraded, acting [23,9]
: b* X+ f7 k; n4 Y    pg 8.781 is active+undersized+degraded, acting [7,20]/ c6 j* L# p( g$ b1 b: b3 |4 j& d
    pg 8.791 is active+undersized+degraded, acting [11,22]
, i. {1 o4 K& C( |. }    pg 8.7b5 is active+undersized+degraded, acting [23,10]
! V: o/ k* r9 j" Y" Q    pg 8.7bb is active+undersized+degraded, acting [8,18]. n+ E9 d( M/ I2 }/ i
    pg 8.7be is active+recovery_wait+degraded, acting [15,19,6]1 R% A0 x$ L0 A8 @9 N: U
    pg 8.7c3 is active+undersized+degraded, acting [23,9]# i4 x  k9 n3 }0 T. e9 z
    pg 8.7d4 is active+undersized+degraded, acting [9,18]
" R! S6 V" f0 p& v    pg 8.7da is active+undersized+degraded, acting [18,10]9 _7 c; `4 s; I2 f
    pg 8.7e5 is active+undersized+degraded, acting [18,8]# `+ r% Y; f; b3 Q, f
    pg 8.7fa is active+undersized+degraded, acting [8,21]
$ U3 F$ X$ W0 W: vSLOW_OPS 154 slow ops, oldest one blocked for 389 sec, mon.ceph1 has slow ops
: C" H! N- H& l: \( v% c5 g, Q0 A( O0 Z( ~  `2 y

1

主题

0

回帖

12

积分

管理员

积分
12
QQ
 楼主| 发表于 2021-10-25 17:04:32 | 显示全部楼层
解决办法:
1 T6 ~, A( S* r0 Q: Usystemctl restart ceph-mon@`hostname -s`.service) b( J' G. e/ V% V
0 X: \6 F( T* V( j6 X$ |3 j- F4 B8 x, s
7 P+ [& K' U& O' V: d
报mon的错误被清除了。
  i$ S$ I$ l# R1 p; \6 I4 S
$ [5 n3 C, |* h9 b[root@ceph1 ~]# ceph health detail / S' g. `7 W; B1 U' A& H' o1 M. y
HEALTH_WARN 6 osds down; Reduced data availability: 170 pgs inactive; Degraded data redundancy: 9370/429503 objects degraded (2.182%), 1102 pgs degraded
9 x" j/ Y( b1 f7 C6 Z) MOSD_DOWN 6 osds down
0 U0 I4 x+ F# k% @! P; ?9 L    osd.20 (root=hdd,host=hdd-ceph3) is down' p* ]7 w, s2 |% e' y: G7 a5 s, I
    osd.22 (root=hdd,host=hdd-ceph3) is down1 O3 Z) h$ }$ f) |  K' K
    osd.23 (root=hdd,host=hdd-ceph3) is down
$ g5 Y7 ?6 w  Y- p  H$ k0 S( y    osd.25 (root=ssd-openstack,host=ssd-compute1) is down# y* @. x, s! t: n: P. K- x
    osd.28 (root=ssd-openstack,host=ssd-compute2) is down( j2 D- I' m& j0 h2 p2 c! _
    osd.29 (root=ssd-openstack,host=ssd-compute2) is down* |/ V( A9 O9 O9 M8 {" S
PG_AVAILABILITY Reduced data availability: 170 pgs inactive
' k3 r, D- d+ m% `+ h    pg 7.13a is stuck inactive for 268.657552, current state undersized+peered, last acting [7]7 F4 @% m2 v$ d7 m0 u0 _
    pg 7.140 is stuck inactive for 173.462639, current state undersized+peered, last acting [6]
& @- U6 z/ m" P    pg 7.149 is stuck inactive for 188.507770, current state undersized+peered, last acting [15]
. C6 ^, E8 m- n5 O. d    pg 7.14a is stuck inactive for 296.614537, current state undersized+peered, last acting [11]
: n0 \2 f8 R) ^) H7 W" d    pg 7.15a is stuck inactive for 230.993717, current state undersized+peered, last acting [7]- z. H' T, |  d  N6 e  y
    pg 7.160 is stuck inactive for 157.298697, current state undersized+peered, last acting [16]
" d6 U  ~: c5 O' x    pg 7.162 is stuck inactive for 168.378304, current state undersized+peered, last acting [8]) ^: ^2 m' z$ p9 \0 o
    pg 7.163 is stuck inactive for 230.964708, current state undersized+peered, last acting [11]
5 g: _+ @1 i$ M% v1 k    pg 7.164 is stuck inactive for 193.535188, current state undersized+peered, last acting [12]9 r  @2 g* I! d
    pg 7.166 is stuck inactive for 157.314779, current state undersized+peered, last acting [9]
0 |3 W/ y8 |. e    pg 7.167 is stuck inactive for 231.001296, current state undersized+peered, last acting [14]
0 f3 F# t  Z! K( Q7 v    pg 7.16f is stuck inactive for 157.301826, current state undersized+peered, last acting [6]9 ~" ^4 j& Q0 ^$ k' h9 Q
    pg 7.18e is stuck inactive for 230.971182, current state undersized+peered, last acting [17]: ~! M7 K  W2 o1 ]  s
    pg 7.196 is stuck inactive for 157.295252, current state undersized+peered, last acting [10]  u4 R- C; B. [& r$ P
    pg 7.332 is stuck inactive for 188.514026, current state undersized+peered, last acting [15]& R" w5 \' T1 M9 K4 U
    pg 7.333 is stuck inactive for 230.989099, current state undersized+peered, last acting [17]
# g+ }0 F% M, p0 l# n5 z! y    pg 7.347 is stuck inactive for 243.608764, current state undersized+peered, last acting [14]2 Y. W+ F4 Y5 t' ~9 j3 Y9 J' o0 `7 @
    pg 7.352 is stuck inactive for 230.961993, current state undersized+peered, last acting [17]$ n# H8 b- A# m; A7 W
    pg 7.358 is stuck inactive for 296.668417, current state undersized+peered, last acting [13]
5 b* W3 b4 }% i. P. B    pg 7.35b is stuck inactive for 296.611193, current state undersized+peered, last acting [9]( O/ r& @9 ^) I1 w
    pg 7.364 is stuck inactive for 231.014057, current state undersized+peered, last acting [7]
! R; j8 j, X0 X, ^' V4 G) |6 p) a! Q    pg 7.372 is stuck inactive for 230.976751, current state undersized+peered, last acting [11]
8 d* y4 R: {! u    pg 7.374 is stuck inactive for 157.314500, current state undersized+peered, last acting [13]
" m, I' O5 c7 B! T8 O9 @) S    pg 7.375 is stuck inactive for 268.647540, current state undersized+peered, last acting [7]3 q; y: I" x  j6 N3 v0 |
    pg 7.376 is stuck inactive for 157.302772, current state undersized+peered, last acting [7]; R, ^( A* `% F! Q
    pg 7.37c is stuck inactive for 157.292778, current state undersized+peered, last acting [9]
! I+ o* \6 e) c3 |    pg 7.381 is stuck inactive for 268.670664, current state undersized+peered, last acting [7]# Y* R( D* m5 y- W2 q( V# g- P3 w6 P
    pg 7.38a is stuck inactive for 157.311084, current state undersized+peered, last acting [6]  A3 b' `, A; I
    pg 7.38b is stuck inactive for 230.974616, current state undersized+peered, last acting [10]. r+ w# D( O3 X5 O6 G1 P' E6 P
    pg 7.391 is stuck inactive for 253.571528, current state undersized+peered, last acting [10]; y: |& ^" ?8 T( A/ r. S* Z1 }/ _
    pg 7.399 is stuck inactive for 231.014969, current state undersized+peered, last acting [7]
) j6 S. d* Z1 c5 Y, E: X, @$ C8 R    pg 7.3a0 is stuck inactive for 157.314500, current state undersized+peered, last acting [17]
5 R! B; c0 K7 ?    pg 7.3a3 is stuck inactive for 157.310316, current state undersized+peered, last acting [16]
+ D9 d9 M( Q! D( f+ B* z    pg 7.3ad is stuck inactive for 243.592394, current state undersized+peered, last acting [14]
6 A# E! k; c9 I7 b2 z1 o    pg 7.3af is stuck inactive for 188.502747, current state undersized+peered, last acting [15]& i( Q, v5 d8 {) v) t9 P; J8 n/ g
    pg 7.3b2 is stuck inactive for 168.373475, current state undersized+peered, last acting [8]
8 @1 w4 b' |% `* `7 x5 Y' A    pg 7.3b4 is stuck inactive for 173.465040, current state undersized+peered, last acting [6]
6 R; i9 d+ l1 Y    pg 7.3b5 is stuck inactive for 157.322812, current state undersized+peered, last acting [9]' U/ ]4 {7 v' M* o" t- s/ Z$ G
    pg 7.3b6 is stuck inactive for 230.969301, current state undersized+peered, last acting [11]
. x8 e  R- ]2 E& [5 R4 l7 y    pg 7.3b9 is stuck inactive for 157.296625, current state undersized+peered, last acting [14]- G5 P: C" N5 ?7 ]# v
    pg 7.3ba is stuck inactive for 231.015066, current state undersized+peered, last acting [7]/ H0 G% u; p) z: s
    pg 7.3be is stuck inactive for 157.311582, current state undersized+peered, last acting [8]
7 F8 l2 \9 h5 P% _$ s+ D# e    pg 7.3c5 is stuck inactive for 168.378059, current state undersized+peered, last acting [8]) w$ Z5 u5 P6 X% H5 s+ k. f# [
    pg 7.3cd is stuck inactive for 230.974691, current state undersized+peered, last acting [10]
* t' m2 U- M$ E    pg 7.3cf is stuck inactive for 168.383761, current state undersized+peered, last acting [8]2 v" C" i5 A5 O  b
    pg 7.3df is stuck inactive for 157.314279, current state undersized+peered, last acting [10]
$ f: Z( C2 U9 @  I3 J7 P    pg 7.3e3 is stuck inactive for 296.651325, current state undersized+peered, last acting [16]
' @* n3 J7 s% |    pg 7.3eb is stuck inactive for 173.465968, current state undersized+peered, last acting [6]
4 K# C7 E& t! w% }2 V: M9 v( F    pg 7.3f8 is stuck inactive for 230.983836, current state undersized+peered, last acting [11]8 }# q+ _! i2 W. k: [0 |: n+ c/ X
    pg 7.3fb is stuck inactive for 157.305951, current state undersized+peered, last acting [16]
: T7 I& i( U* D( q$ S    pg 7.3ff is stuck inactive for 268.659557, current state undersized+peered, last acting [7]
8 ^- q( ^4 P/ D% f0 q0 {; z+ KPG_DEGRADED Degraded data redundancy: 9370/429503 objects degraded (2.182%), 1102 pgs degraded
! ?/ q+ v2 Q, y+ [    pg 8.795 is active+undersized+degraded, acting [11,17]
( B  V9 P! X- [/ a3 C! F, a9 v    pg 8.79b is active+undersized+degraded, acting [10,12], ?$ K0 M8 B6 a, h( ^
    pg 8.79c is active+undersized+degraded, acting [13,10]
3 y) G+ p8 d6 f0 e3 a    pg 8.79f is active+undersized+degraded, acting [9,13]
" q- E/ l1 d# _8 O# `    pg 8.7a0 is active+undersized+degraded, acting [8,17]
0 L- Z! E: T& y: t    pg 8.7a1 is active+undersized+degraded, acting [16,9]5 `3 p8 A% A( n0 s
    pg 8.7a2 is active+undersized+degraded, acting [10,17], `% V, N$ z, t
    pg 8.7a3 is active+undersized+degraded, acting [17,11]7 T+ d: Z2 g& ?5 s
    pg 8.7a5 is active+undersized+degraded, acting [11,17]
4 t) D! k4 R* J    pg 8.7a8 is active+undersized+degraded, acting [13,7]4 W! G* x1 J, `6 j. q7 K) \
    pg 8.7a9 is active+undersized+degraded, acting [15,7]
/ t9 a, `$ a$ u: I    pg 8.7ad is active+undersized+degraded, acting [15,9]
' S& [) H! P5 x/ l    pg 8.7ae is active+undersized+degraded, acting [14,10]
7 G& Y4 A# {' ?. g5 L$ {, T    pg 8.7b0 is active+undersized+degraded, acting [12,6]
2 _' F" K; ?1 F/ H% R/ H7 V    pg 8.7b5 is active+undersized+degraded, acting [10,15]* ]! F+ {: C, D" \, ]
    pg 8.7b6 is active+undersized+degraded, acting [15,9]
( n; H2 |* f4 _2 g6 Q    pg 8.7b8 is active+undersized+degraded, acting [8,16]4 |' T% `" \1 }, i$ e0 O3 V! H2 f
    pg 8.7ba is active+undersized+degraded, acting [13,6], ?: x; X9 {2 Z% q$ l* d4 k
    pg 8.7bd is active+undersized+degraded, acting [7,13]
+ Q& |5 v4 J7 ]# Z    pg 8.7bf is active+undersized+degraded, acting [17,11]
9 a: l5 V3 _! G( \; o. R2 |    pg 8.7c1 is active+undersized+degraded, acting [12,10]* ?9 U& m+ X, l
    pg 8.7c2 is active+undersized+degraded, acting [6,12]( {/ z( H7 T; a( n5 c
    pg 8.7c3 is active+undersized+degraded, acting [9,15]) k! j) V9 ^' Q# V$ j: z) c
    pg 8.7c4 is active+undersized+degraded, acting [6,16]
9 V/ r2 M( G+ A7 {- ~5 m/ L    pg 8.7c5 is active+undersized+degraded, acting [8,14]3 N8 V; ^, Q9 C1 N
    pg 8.7c6 is active+undersized+degraded, acting [17,7]
- B& {/ f* C  E* A& e    pg 8.7c7 is active+undersized+degraded, acting [16,11]5 m; M; Z1 ~( D+ f; h: e7 _. @
    pg 8.7ca is active+undersized+degraded, acting [16,11]
6 S' D8 `7 y. ~  g    pg 8.7cd is active+undersized+degraded, acting [13,6]
) \; i* E1 i- ^1 U$ v9 F6 I    pg 8.7ce is active+undersized+degraded, acting [13,8]
' a+ q# O8 m' M3 b    pg 8.7d0 is active+undersized+degraded, acting [17,10]
5 C& S+ L0 C( M, L: ~    pg 8.7d5 is active+undersized+degraded, acting [14,11]
( X2 Q0 I- r) H/ N+ s( b# S# R    pg 8.7d8 is active+undersized+degraded, acting [13,10]8 t0 \4 I" J2 h* @- v7 c
    pg 8.7d9 is active+undersized+degraded, acting [9,13]3 T. ?$ Y7 f. C  s1 b
    pg 8.7dd is active+undersized+degraded, acting [12,11]
" N! W  {3 g# j  m    pg 8.7df is active+undersized+degraded, acting [11,16]8 L8 T0 g3 S2 V2 z9 X% p; P! E
    pg 8.7e0 is active+undersized+degraded, acting [17,6]7 t0 N# r) z+ k; r/ W- p, C6 K
    pg 8.7e1 is active+undersized+degraded, acting [14,10]+ ]- y; }" y* Z$ P8 O) U
    pg 8.7e2 is active+undersized+degraded, acting [9,13]. f& X+ U7 N1 H+ J
    pg 8.7e3 is active+undersized+degraded, acting [14,9]8 E- ^3 z' C4 N: g: t
    pg 8.7e7 is active+undersized+degraded, acting [6,15]
$ I& ?5 S, Q$ n- c" M  |    pg 8.7ea is active+undersized+degraded, acting [9,16]
# E+ ^; @) z& l    pg 8.7eb is active+undersized+degraded, acting [14,9], l* Y( t5 e2 P4 h0 E
    pg 8.7ec is active+undersized+degraded, acting [16,11]
) {  n" \: H5 Q6 ]$ N7 s  O    pg 8.7ed is active+undersized+degraded, acting [14,10]" u. c' u6 }8 }, A& w! u
    pg 8.7f0 is active+undersized+degraded, acting [17,6]
, \( g: p# A& E2 P$ O    pg 8.7f1 is active+undersized+degraded, acting [15,7]
# u6 \9 x7 L$ ]+ r$ I    pg 8.7f3 is active+undersized+degraded, acting [9,17]
5 j/ R) N# L& p    pg 8.7f4 is active+undersized+degraded, acting [9,14]: J* X) Q/ h" t+ `/ l6 s
    pg 8.7fb is active+undersized+degraded, acting [9,17]7 P9 w4 Q& I( h  D3 _
    pg 8.7fc is active+undersized+degraded, acting [15,8]9 a/ n8 i" L( A9 y
您需要登录后才可以回帖 登录 | 注册

本版积分规则

返回首页|Archiver|手机版|小黑屋|易陆发现技术论坛 ( 蜀ICP备2026014127号-1 )

GMT+8, 2026-6-12 00:58 , Processed in 0.020348 second(s), 23 queries .

Powered by Discuz! X5.0

© 2001-2026 Discuz! Team.

快速回复 返回顶部 返回列表