找回密码
 注册
查看: 2119|回复: 1

HEALTH_WARN Degraded data redundancy ... has slow ops

[复制链接]

1

主题

0

回帖

12

积分

管理员

积分
12
QQ
发表于 2021-10-25 17:00:42 | 显示全部楼层 |阅读模式
HEALTH_WARN Degraded data redundancy: 822/429494 objects degraded (0.191%), 115 pgs degraded; 154 slow ops, oldest one blocked for 389 sec, mon.ceph1 has slow ops3 v: i- b! U% U7 H& x7 P& T
PG_DEGRADED Degraded data redundancy: 822/429494 objects degraded (0.191%), 115 pgs degraded( h& D) @7 _4 S" N% H
    pg 8.473 is active+undersized+degraded, acting [18,9]1 ~  j) C( I+ T/ q. L6 q& y/ U2 F
    pg 8.483 is active+undersized+degraded, acting [9,20]8 G. i: |/ D- M" X' I
    pg 8.48a is active+undersized+degraded, acting [11,21]
9 B! ^; q8 y4 N- c  K9 V    pg 8.49d is active+undersized+degraded, acting [23,6]
. n, v/ D! o7 t( D0 E- ^/ i9 n    pg 8.4a7 is active+undersized+degraded, acting [11,20]
7 u3 w$ j5 c3 g% S# P    pg 8.4a8 is active+undersized+degraded, acting [8,20]
4 {4 C( X$ S5 u; B    pg 8.4a9 is active+undersized+degraded, acting [18,8]' I) n; n% w4 V) ?
    pg 8.4b2 is active+undersized+degraded, acting [11,21]1 R0 `: e8 ?; U. g4 V# t8 G1 c6 Y
    pg 8.4c8 is active+undersized+degraded, acting [7,22]0 {6 w* n8 P8 [
    pg 8.4e4 is active+undersized+degraded, acting [9,20]5 s4 i. e5 p& Z6 N0 |8 D
    pg 8.4e7 is active+undersized+degraded, acting [18,11]
1 H8 Y1 ^; r+ U/ y    pg 8.4f2 is active+undersized+degraded, acting [23,8]3 b( |, N' j! K# ~  K
    pg 8.4fb is active+undersized+degraded, acting [18,7]$ X6 b8 r2 z5 Z+ _0 ?; C% J
    pg 8.505 is active+undersized+degraded, acting [8,21]' p7 \. R, O9 S, O! L, `
    pg 8.54e is active+undersized+degraded, acting [8,22]
+ y( q3 y6 P8 d. \    pg 8.566 is active+undersized+degraded, acting [23,10]
9 J4 Y0 [. p5 N4 n) E# V& U    pg 8.567 is active+undersized+degraded, acting [8,19]. f% i! A- b" l# E$ K' \1 J% f
    pg 8.57c is active+undersized+degraded, acting [23,9]
0 z% y4 T/ [2 T/ I    pg 8.57d is active+undersized+degraded, acting [8,22]' u) R# @; J! n" m
    pg 8.596 is active+undersized+degraded, acting [23,8]) h- o: a; p; ?+ i$ T) G! F& g# o
    pg 8.5b3 is active+undersized+degraded, acting [9,18]# a9 D2 M! Q2 x2 r
    pg 8.5fc is active+undersized+degraded, acting [8,19]! l0 Q* H' P9 p
    pg 8.606 is active+undersized+degraded, acting [8,19]* x2 g: y0 W- P  z- X: |" Q. N
    pg 8.627 is active+undersized+degraded, acting [11,22]; c8 Y9 N6 s! }1 r; k
    pg 8.636 is active+recovery_wait+degraded, acting [15,21,10]
& o1 g: |/ \  J; Y4 }0 K0 f    pg 8.651 is active+undersized+degraded, acting [23,6]2 p9 }- q/ f7 P' i
    pg 8.65b is active+undersized+degraded, acting [11,21]
6 i6 b1 R: s. }* I, `* I    pg 8.666 is active+undersized+degraded, acting [23,6]- Y: Q' C6 t3 N7 z! ]& T8 U
    pg 8.679 is active+undersized+degraded, acting [11,21]
4 a% W6 a% N/ l& s! X    pg 8.6a3 is active+undersized+degraded, acting [7,22]
0 V" t% \2 v7 w4 R& [8 p+ X    pg 8.6c3 is active+undersized+degraded, acting [23,6]" |2 \* ^" M7 W+ }
    pg 8.6e1 is active+undersized+degraded, acting [8,18]
% l5 ?% d1 \& w- F: m% G' O    pg 8.6ed is active+undersized+degraded, acting [8,22]0 I% H. w& f9 z$ J* J$ u9 Q4 k  d
    pg 8.701 is active+undersized+degraded, acting [7,19]
# f. Q$ q/ M: Y) Y    pg 8.724 is active+undersized+degraded, acting [23,10]
$ v: r; f& w/ J    pg 8.728 is active+undersized+degraded, acting [18,10]* ^1 o. X; S6 I( o1 Y4 \* A4 i
    pg 8.72f is active+undersized+degraded, acting [7,23]# H( s% r( }: X7 }7 F  N7 q
    pg 8.732 is active+undersized+degraded, acting [9,18]
0 c2 v9 x" e( P    pg 8.739 is active+undersized+degraded, acting [9,22]. x5 j0 M* M: p' b7 S& r
    pg 8.76a is active+undersized+degraded, acting [11,21]
6 b8 H, Q# H2 n# c1 m# k, O3 ^; o    pg 8.770 is active+undersized+degraded, acting [23,9]6 s% U6 v+ J, {; H7 a+ n7 `* K
    pg 8.781 is active+undersized+degraded, acting [7,20]/ M4 u# d! Q, B. z4 J6 P& L) p" m- p
    pg 8.791 is active+undersized+degraded, acting [11,22]
$ S4 V+ o, I" W    pg 8.7b5 is active+undersized+degraded, acting [23,10]' O4 Z& ?2 |" P, z7 c' X
    pg 8.7bb is active+undersized+degraded, acting [8,18]% z1 }9 Q/ t% L4 O4 h3 x
    pg 8.7be is active+recovery_wait+degraded, acting [15,19,6]: b1 Q: u3 J4 Q1 G, I
    pg 8.7c3 is active+undersized+degraded, acting [23,9]
6 l% {; T' Q. G    pg 8.7d4 is active+undersized+degraded, acting [9,18]
2 F2 X, P1 S, y! u    pg 8.7da is active+undersized+degraded, acting [18,10]
: I# t# }1 [9 N5 |; F2 R    pg 8.7e5 is active+undersized+degraded, acting [18,8]6 ~: U& l. ]: W$ K' `# ^
    pg 8.7fa is active+undersized+degraded, acting [8,21]
( d6 t2 m. R5 w2 |: B/ tSLOW_OPS 154 slow ops, oldest one blocked for 389 sec, mon.ceph1 has slow ops- u$ L# ]! z. c$ a1 ]
3 X! G$ G$ {8 m1 _8 L+ y

1

主题

0

回帖

12

积分

管理员

积分
12
QQ
 楼主| 发表于 2021-10-25 17:04:32 | 显示全部楼层
解决办法:
# k9 f/ c3 H# @& |6 K8 }systemctl restart ceph-mon@`hostname -s`.service
' ]4 G! T$ B  X# E1 ]  R* b4 H! e, o; }

% A" e% c4 ]' _( i5 l6 M7 L报mon的错误被清除了。0 R4 Y  }6 C4 D8 T0 d  ~  N

2 L0 ?- u9 p4 u' ?4 m2 S# d[root@ceph1 ~]# ceph health detail 9 o" y% `3 c  o; F
HEALTH_WARN 6 osds down; Reduced data availability: 170 pgs inactive; Degraded data redundancy: 9370/429503 objects degraded (2.182%), 1102 pgs degraded4 J* |; `! X% t1 Z. B( @  S0 B9 W
OSD_DOWN 6 osds down3 g; m. _5 o, I  a
    osd.20 (root=hdd,host=hdd-ceph3) is down9 q6 f; B& D8 W
    osd.22 (root=hdd,host=hdd-ceph3) is down- s: e! P9 W) q+ @0 i
    osd.23 (root=hdd,host=hdd-ceph3) is down, `* i9 r4 e3 b  |: q
    osd.25 (root=ssd-openstack,host=ssd-compute1) is down
- g: N& L7 d& P    osd.28 (root=ssd-openstack,host=ssd-compute2) is down
( M& |" r( K# w5 O# G0 d2 I8 p    osd.29 (root=ssd-openstack,host=ssd-compute2) is down1 z3 H9 J4 d% O* C
PG_AVAILABILITY Reduced data availability: 170 pgs inactive% \+ k" r/ S0 O, P$ E- w6 S7 |  R
    pg 7.13a is stuck inactive for 268.657552, current state undersized+peered, last acting [7]
6 ~) n/ N+ T1 y6 [/ K    pg 7.140 is stuck inactive for 173.462639, current state undersized+peered, last acting [6]: G' p! Y6 {  W& I) s- N' K* x3 ~
    pg 7.149 is stuck inactive for 188.507770, current state undersized+peered, last acting [15]" f) C0 G. P: [1 r1 O
    pg 7.14a is stuck inactive for 296.614537, current state undersized+peered, last acting [11]% ?+ I! }+ R* y3 R# A8 E( t; z
    pg 7.15a is stuck inactive for 230.993717, current state undersized+peered, last acting [7]' m) _1 [. A+ c( R6 y
    pg 7.160 is stuck inactive for 157.298697, current state undersized+peered, last acting [16]
: i  \6 F: E; ~3 e9 M( C    pg 7.162 is stuck inactive for 168.378304, current state undersized+peered, last acting [8]
* ^5 {' _7 t; m    pg 7.163 is stuck inactive for 230.964708, current state undersized+peered, last acting [11]
7 G/ D1 T( _% R& U# X% E    pg 7.164 is stuck inactive for 193.535188, current state undersized+peered, last acting [12]
+ F7 `4 g' c7 W7 Z  n* e7 S* k    pg 7.166 is stuck inactive for 157.314779, current state undersized+peered, last acting [9]8 m3 X5 [* }& ^) Y$ I* \4 t
    pg 7.167 is stuck inactive for 231.001296, current state undersized+peered, last acting [14]
3 t% N, S  x- \" S: l2 z) h2 c    pg 7.16f is stuck inactive for 157.301826, current state undersized+peered, last acting [6]' x5 I7 O* l, ?: [
    pg 7.18e is stuck inactive for 230.971182, current state undersized+peered, last acting [17]
# h# H* h$ y9 @( [2 J    pg 7.196 is stuck inactive for 157.295252, current state undersized+peered, last acting [10]
/ o: P6 ?0 f3 g    pg 7.332 is stuck inactive for 188.514026, current state undersized+peered, last acting [15]  F8 z9 W! _  o) I" N& O/ p
    pg 7.333 is stuck inactive for 230.989099, current state undersized+peered, last acting [17]8 B- x" O1 b" ]) Y, ^
    pg 7.347 is stuck inactive for 243.608764, current state undersized+peered, last acting [14]& K" Z- p# `+ o1 \
    pg 7.352 is stuck inactive for 230.961993, current state undersized+peered, last acting [17]
" L+ g' C9 Y, @0 ], @. k* l    pg 7.358 is stuck inactive for 296.668417, current state undersized+peered, last acting [13]
3 x2 n% d; L) U! [/ u  _    pg 7.35b is stuck inactive for 296.611193, current state undersized+peered, last acting [9]
- [7 x* m+ \. Y  d5 n5 N. A, Q    pg 7.364 is stuck inactive for 231.014057, current state undersized+peered, last acting [7]
8 U! `1 I, t. B. S. W7 x# q  ]/ v    pg 7.372 is stuck inactive for 230.976751, current state undersized+peered, last acting [11]- }! S; b6 @- S  O# c
    pg 7.374 is stuck inactive for 157.314500, current state undersized+peered, last acting [13]
2 ]! [( f8 p3 N    pg 7.375 is stuck inactive for 268.647540, current state undersized+peered, last acting [7]8 B8 z$ a* b0 }4 Z. [
    pg 7.376 is stuck inactive for 157.302772, current state undersized+peered, last acting [7]
0 J4 a- u! p  u/ S# u+ r/ D    pg 7.37c is stuck inactive for 157.292778, current state undersized+peered, last acting [9]
. O3 u0 Y3 Y8 w; p5 _' h4 V$ o! D    pg 7.381 is stuck inactive for 268.670664, current state undersized+peered, last acting [7]
$ ^! r, c" v2 ^! N    pg 7.38a is stuck inactive for 157.311084, current state undersized+peered, last acting [6]9 ?5 `9 I- ~6 a6 S/ G6 |. q2 d
    pg 7.38b is stuck inactive for 230.974616, current state undersized+peered, last acting [10]
5 ^! l% A& P" D' E- C; [. R    pg 7.391 is stuck inactive for 253.571528, current state undersized+peered, last acting [10]$ Z+ A: B" z4 a8 ]3 _0 x% y
    pg 7.399 is stuck inactive for 231.014969, current state undersized+peered, last acting [7]
7 H& s- o9 ^) X  l- y# [& D# S/ [    pg 7.3a0 is stuck inactive for 157.314500, current state undersized+peered, last acting [17]$ d4 a, u9 O4 e* u) }: l, H  P
    pg 7.3a3 is stuck inactive for 157.310316, current state undersized+peered, last acting [16]
  J+ ]( `3 `5 d) L5 O, H& U) I- V! }3 x    pg 7.3ad is stuck inactive for 243.592394, current state undersized+peered, last acting [14]
4 l3 [1 z" {4 B3 P# C. e/ ?    pg 7.3af is stuck inactive for 188.502747, current state undersized+peered, last acting [15], b- u7 N. U2 `+ L  I
    pg 7.3b2 is stuck inactive for 168.373475, current state undersized+peered, last acting [8], ^* O1 F! {8 M8 P
    pg 7.3b4 is stuck inactive for 173.465040, current state undersized+peered, last acting [6]! l" j6 f2 u6 M) y; D0 j
    pg 7.3b5 is stuck inactive for 157.322812, current state undersized+peered, last acting [9], A3 D( l7 h2 s9 Q
    pg 7.3b6 is stuck inactive for 230.969301, current state undersized+peered, last acting [11]
+ R$ D' q0 W: v' S+ d    pg 7.3b9 is stuck inactive for 157.296625, current state undersized+peered, last acting [14]# F3 \; P0 o  N3 G$ W9 @
    pg 7.3ba is stuck inactive for 231.015066, current state undersized+peered, last acting [7]
+ l4 q6 r9 \. A  @) k+ ?; l    pg 7.3be is stuck inactive for 157.311582, current state undersized+peered, last acting [8]
2 C$ c  H3 W/ r, t) X( d2 Z    pg 7.3c5 is stuck inactive for 168.378059, current state undersized+peered, last acting [8]
6 W1 w0 t8 V+ ^5 D7 ^$ M    pg 7.3cd is stuck inactive for 230.974691, current state undersized+peered, last acting [10]) t0 [  Q/ \, l& P4 f) y0 `4 Y
    pg 7.3cf is stuck inactive for 168.383761, current state undersized+peered, last acting [8]) w3 x/ j' N* Y5 l* Q) g+ b
    pg 7.3df is stuck inactive for 157.314279, current state undersized+peered, last acting [10]- r) ?: _4 E- i" ~7 D' x) \" [
    pg 7.3e3 is stuck inactive for 296.651325, current state undersized+peered, last acting [16]1 l. V3 P3 [- F5 c6 A
    pg 7.3eb is stuck inactive for 173.465968, current state undersized+peered, last acting [6]
# u5 l5 R% l. \& D; _$ ^    pg 7.3f8 is stuck inactive for 230.983836, current state undersized+peered, last acting [11]: h- o" e4 Z4 N1 H& ?
    pg 7.3fb is stuck inactive for 157.305951, current state undersized+peered, last acting [16]2 y+ ~" H. F1 z/ c7 Z6 z1 J' H/ p
    pg 7.3ff is stuck inactive for 268.659557, current state undersized+peered, last acting [7]: t7 p/ a+ l2 K( n7 O
PG_DEGRADED Degraded data redundancy: 9370/429503 objects degraded (2.182%), 1102 pgs degraded
0 A8 H+ a4 [3 ]7 M% b; f7 ]$ a    pg 8.795 is active+undersized+degraded, acting [11,17]
' U, h6 O% }$ d6 c" `+ ?2 e    pg 8.79b is active+undersized+degraded, acting [10,12]
1 T' m) l: m- Q# V    pg 8.79c is active+undersized+degraded, acting [13,10]
/ q8 p; S! g' Q0 X8 i! l. V    pg 8.79f is active+undersized+degraded, acting [9,13]) X! F5 t+ ]/ ~
    pg 8.7a0 is active+undersized+degraded, acting [8,17]
- ?/ c- X8 O' N0 e    pg 8.7a1 is active+undersized+degraded, acting [16,9]
: i  f$ E+ [7 l# _$ F8 C0 I3 M. e    pg 8.7a2 is active+undersized+degraded, acting [10,17]) b: h& \% F6 G
    pg 8.7a3 is active+undersized+degraded, acting [17,11]6 l% H6 ^; q' l
    pg 8.7a5 is active+undersized+degraded, acting [11,17]7 m3 p2 v: {3 h9 x6 B- m: [
    pg 8.7a8 is active+undersized+degraded, acting [13,7]
; Z, M! C" P0 B/ J: x    pg 8.7a9 is active+undersized+degraded, acting [15,7]
0 U9 ^5 \$ [9 h% p0 M6 Y    pg 8.7ad is active+undersized+degraded, acting [15,9]
; D) S  G" }. A- m: x    pg 8.7ae is active+undersized+degraded, acting [14,10]
/ ]. H# W0 {- R9 z" H/ I    pg 8.7b0 is active+undersized+degraded, acting [12,6]
) J. p. k( Z* }: }" h: t# K2 A    pg 8.7b5 is active+undersized+degraded, acting [10,15]
+ n  I: P$ j. f$ r    pg 8.7b6 is active+undersized+degraded, acting [15,9]* y$ o  j8 `. e* l0 ~- R
    pg 8.7b8 is active+undersized+degraded, acting [8,16]" T% Q9 w# h+ ^  D1 m) `
    pg 8.7ba is active+undersized+degraded, acting [13,6]4 C1 I7 O7 h+ v
    pg 8.7bd is active+undersized+degraded, acting [7,13]7 E' T- N1 ?, B. C* \( l% N
    pg 8.7bf is active+undersized+degraded, acting [17,11]
( Q$ ?2 m% ~( j) T- j8 E3 l" s; q    pg 8.7c1 is active+undersized+degraded, acting [12,10]$ K5 b6 V7 T$ Q8 z( b! V* C
    pg 8.7c2 is active+undersized+degraded, acting [6,12]( m7 [/ q3 w! e, g, h
    pg 8.7c3 is active+undersized+degraded, acting [9,15]  h: Y  G3 Y: O  Y- F+ \$ G
    pg 8.7c4 is active+undersized+degraded, acting [6,16]
9 z! }& ^& T0 o1 m( j    pg 8.7c5 is active+undersized+degraded, acting [8,14]; v5 @4 t/ p% x2 w1 M
    pg 8.7c6 is active+undersized+degraded, acting [17,7]
6 I- c- w; m2 f* k9 T6 p    pg 8.7c7 is active+undersized+degraded, acting [16,11]
8 d& ^8 l2 K. F! ]- t) a# C    pg 8.7ca is active+undersized+degraded, acting [16,11]
$ G( M4 U0 u; U6 I1 Y5 K    pg 8.7cd is active+undersized+degraded, acting [13,6]  n  x; ^, P5 L+ T, O: m6 o7 S5 f* p* c
    pg 8.7ce is active+undersized+degraded, acting [13,8]" c- V8 o8 v  s: S/ O: p; j8 h
    pg 8.7d0 is active+undersized+degraded, acting [17,10]
6 C, \2 s. {# A1 a. H1 j    pg 8.7d5 is active+undersized+degraded, acting [14,11]
0 ^" _( p( q0 m( w$ X5 k& w3 W: \( c    pg 8.7d8 is active+undersized+degraded, acting [13,10]4 c5 f: I/ D2 [
    pg 8.7d9 is active+undersized+degraded, acting [9,13]
  K7 J3 m1 V7 G/ x. @9 k; q    pg 8.7dd is active+undersized+degraded, acting [12,11]& b$ g. V1 q3 Y) r
    pg 8.7df is active+undersized+degraded, acting [11,16]
2 y; R) [$ |/ d1 Z  B; J+ D* w    pg 8.7e0 is active+undersized+degraded, acting [17,6]! V6 ]6 z3 M0 ?
    pg 8.7e1 is active+undersized+degraded, acting [14,10]; S6 O- r, ^' ~) t
    pg 8.7e2 is active+undersized+degraded, acting [9,13]
# @/ p- |0 O+ i( g* V    pg 8.7e3 is active+undersized+degraded, acting [14,9]$ x8 F3 b* K8 }3 `
    pg 8.7e7 is active+undersized+degraded, acting [6,15]! w4 A7 y% R! E  q
    pg 8.7ea is active+undersized+degraded, acting [9,16]* g7 B8 Q3 l: _5 y% n; C0 i6 ^; S
    pg 8.7eb is active+undersized+degraded, acting [14,9]. S) ~8 t8 y( Y3 C
    pg 8.7ec is active+undersized+degraded, acting [16,11]* h) x" ?) X! U/ I/ n& n& ?5 Z' v1 t4 [
    pg 8.7ed is active+undersized+degraded, acting [14,10]  K6 _& S: M; K" n# X/ f
    pg 8.7f0 is active+undersized+degraded, acting [17,6]7 J) U+ f5 S. T( X
    pg 8.7f1 is active+undersized+degraded, acting [15,7]
8 y2 {( X/ V0 d6 @    pg 8.7f3 is active+undersized+degraded, acting [9,17]8 l% ]/ ~5 f0 V" D9 C9 n
    pg 8.7f4 is active+undersized+degraded, acting [9,14]  L8 [" k) [! o9 \; x% t) X3 k! x) j
    pg 8.7fb is active+undersized+degraded, acting [9,17]
; O7 O5 @9 @" {( S5 B, }* f6 h. q+ z    pg 8.7fc is active+undersized+degraded, acting [15,8]
+ ^: S6 C0 Z& I; k  M- T+ {
您需要登录后才可以回帖 登录 | 注册

本版积分规则

返回首页|Archiver|手机版|小黑屋|易陆发现技术论坛 ( 蜀ICP备2026014127号-1 )

GMT+8, 2026-6-12 00:01 , Processed in 0.016256 second(s), 23 queries .

Powered by Discuz! X5.0

© 2001-2026 Discuz! Team.

快速回复 返回顶部 返回列表