找回密码
 注册
查看: 1351|回复: 1

记一次ceph存储同步问题noout,nobackfill,norecover flag(s) set Degraded data r

[复制链接]

1

主题

0

回帖

12

积分

管理员

积分
12
QQ
发表于 2021-12-20 11:00:13 | 显示全部楼层 |阅读模式
检查ceph 状态:5 J- ^( a: D6 @  h- s' O3 s5 h) M5 p
[root@compute01 ~]# ceph -s, E2 F" |6 t3 l# l& l# m& E0 K
  cluster:
4 q3 i$ z+ H6 a9 Q    id:     a6d0359e-8295-4e62-8620-54273462e79f
% P8 v. d( L5 i! r2 _3 E. W5 ^    health: HEALTH_WARN- L; r" j4 i6 }: K, y9 E5 U% C/ G  ^
            noout,nobackfill,norecover flag(s) set
/ r2 d+ }; |" L& u/ o  w            Degraded data redundancy: 3793/96483 objects degraded (3.931%), 876 pgs degraded, 512 pgs undersized* g2 |- H7 \3 V$ @. }; c
            6 pgs not deep-scrubbed in time9 z, {8 x/ [; T" j4 _* I
            clock skew detected on mon.compute02, mon.compute03
. L! A- X3 ~2 [  P7 E, l+ \1 h& Z8 z
  services:
  q- @2 y5 a/ h2 l. X# u    mon: 3 daemons, quorum compute01,compute02,compute03 (age 22m)
" b6 e1 v  s9 a9 p: `7 s    mgr: compute02(active, since 4d), standbys: compute03, compute015 h+ p0 c8 N7 r: v3 \2 }# y
    osd: 6 osds: 6 up (since 22m), 6 in (since 2w); 512 remapped pgs
4 d* I- x- t, |& h: }2 u$ h1 H1 A, W         flags noout,nobackfill,norecover. N/ X" A4 m. w. S

! Z$ Z% |0 P- ~5 m. z0 ?" J) z  data:
: L# Z7 G& S1 y9 _% p: N2 {    pools:   5 pools, 1408 pgs
8 M& d2 a) [( P7 x* y; V8 O    objects: 32.65k objects, 132 GiB  ?5 P3 l& }; f; g* F) u( d. b
    usage:   381 GiB used, 4.9 TiB / 5.2 TiB avail
9 H0 m4 d& E) L! S. j- w8 ]    pgs:     3793/96483 objects degraded (3.931%)
. Y+ r2 J8 b7 i" `& M' w             450 active+recovery_wait+undersized+degraded+remapped# K" i0 g: y; G8 B2 c! N$ N  H
             444 active+clean
5 b; e' h( E  a             362 active+recovery_wait+degraded
6 u3 @7 b6 d; v& t6 J             87  active+recovery_wait1 {+ [4 S" r, p* {8 v: J* c
             62  active+undersized+degraded+remapped+backfill_wait
- \6 W8 P9 c9 g" p) H) J0 t; Z: W             2   active+recovering+degraded
$ G5 L1 e; `0 r4 ]             1   active+recovering
' {) N& h( I: u% G1 b6 P
: _. J( x, a  k" I" f) m( q  io:2 y1 Y+ v- ]6 f
    client:   156 KiB/s rd, 78 KiB/s wr, 14 op/s rd, 8 op/s wr+ A6 e% [) x/ f# T1 F( o# m

8 a7 P* l7 B2 j. _% ~- c+ |/ j0 z( j) N- y5 ]' z3 Z
设置ceph osd 开启recover和覆盖同步配置:
; O/ M& ]! t- c5 l
1 {2 d7 {/ d" I! j1 \2 V& l; G( n6 w7 x
/ E8 m4 g2 k% {" F7 {[root@compute01 ~]# ceph osd unset norecover ! `$ r& v+ H! @" O
norecover is unset
6 d$ I  {# ~. m8 v' p[root@compute01 ~]# ceph osd unset nobackfill
& P) N, i/ b# p$ m5 H+ Onobackfill is unset) h$ t: c/ v+ i7 {5 g& {  P1 f5 @

( U8 |1 z) z9 Y5 L9 ^* R' t9 H- ]" S查看状态:
" x$ ?3 ~! _) V[root@compute01 ~]# ceph -s
" e9 K3 f3 G% f  c4 ^9 e; u" j  cluster:% F6 t' A4 _/ H4 W5 v, `2 b
    id:     a6d0359e-8295-4e62-8620-54273462e79f( T8 ?0 o- H3 k: _7 H
    health: HEALTH_WARN2 x  ?" u6 p" q  ?2 j4 _2 a
            noout flag(s) set
3 g7 K* N, V/ B. @( g: o            Degraded data redundancy: 3537/96486 objects degraded (3.666%), 705 pgs degraded, 512 pgs undersized
! r. V* g# F( ]8 J: o            6 pgs not deep-scrubbed in time! p9 x2 w+ A- y" r* G; P
            clock skew detected on mon.compute02, mon.compute03
+ `9 W* R/ {7 |9 x' B' ^& o* f' S. ]# W' ~& d, M6 y
  services:! A8 ?1 h, u5 Z1 {3 }' ^
    mon: 3 daemons, quorum compute01,compute02,compute03 (age 23m)
0 q2 {- Y$ t- w* \7 O    mgr: compute02(active, since 4d), standbys: compute03, compute01
/ k% `6 h- c) p7 V2 p0 _2 ^    osd: 6 osds: 6 up (since 23m), 6 in (since 2w); 512 remapped pgs' s; X; l  p- C+ g. Q8 G9 }3 e
         flags noout' ]  I2 s8 m" V
& X. k8 d8 f; |$ \+ Z9 X' B
  data:
5 a: S. }8 b" ?8 \' ~; o  Y    pools:   5 pools, 1408 pgs
9 N- m7 K9 n; u, s! f    objects: 32.65k objects, 132 GiB; x) k+ l/ q- n& y- c
    usage:   382 GiB used, 4.9 TiB / 5.2 TiB avail" D8 r5 Z0 r1 f1 O0 z5 S/ O
    pgs:     3537/96486 objects degraded (3.666%)
7 k: l7 L3 ?) o) B             649 active+clean
, m' l' N+ X* ^6 O* A3 Q9 g4 o             450 active+recovery_wait+undersized+degraded+remapped
* O* u8 F* E; F3 _1 q1 B             191 active+recovery_wait+degraded
( N% e* p/ u, [9 }             62  active+undersized+degraded+remapped+backfill_wait
; F* j  v3 K) b. \5 u3 v+ M1 M5 ?             54  active+recovery_wait7 _5 w* J& F* a) t" s/ K8 a; T
             2   active+recovering+degraded8 K, l; C! y' W/ S8 i8 R- [' _1 ~

6 k4 t3 P" w: {% O+ x  io:5 H2 g4 H( z7 F) Q7 x% @
    client:   204 KiB/s rd, 797 KiB/s wr, 147 op/s rd, 42 op/s wr
8 W+ w; N6 f" M8 f- N: @+ v    recovery: 89 MiB/s, 21 objects/s
1 w% b: [( ]0 Z! ]+ O 上面看到数据开始在同步了,覆盖了。" V$ l% F% L9 F. {  w
4 H7 F$ y2 n$ e8 f' a
查看osd是否都正常:  A- h) c% Z- |; B) y; v
[root@compute01 ~]# ceph osd tree
; h5 J0 ^( ~9 D5 n" m. z- a% cID  CLASS WEIGHT  TYPE NAME              STATUS REWEIGHT PRI-AFF
) f8 d: j; K4 i8 E" u$ A-11             0 host ssd-compute05                             
6 c5 J! v: j! v -3       6.00000 root ssd                                       % G1 B8 A( A" r# r( M, [
-4       2.00000     host ssd-compute01                        
! P: k" u% z+ O9 d5 I  0   ssd 1.00000         osd.0              up  1.00000 1.00000
3 x8 T1 S3 m' I  1   ssd 1.00000         osd.1              up  1.00000 1.00000 6 y  J+ [+ y% Y1 e+ E. l& `
-5       2.00000     host ssd-compute02                         5 N3 @- t4 z' v0 ?, A$ ]/ B
  2   ssd 1.00000         osd.2              up  1.00000 1.00000
# f1 n  l7 h9 z% h  3   ssd 1.00000         osd.3              up  1.00000 1.00000
' e7 w' N+ j! P' s4 g -6       2.00000     host ssd-compute03                        
' n" k- t* `1 H6 h  5   ssd 1.00000         osd.5              up  1.00000 1.00000 # s. w1 |! z- \, h1 Q( }
  6   ssd 1.00000         osd.6              up  1.00000 1.00000
- [) v! B5 c! W1 Q8 ^* R0 t -1             0 root default                                   
# n! R! `& x$ c" @. V4 ^2 ]5 _# Q1 O9 {
[root@compute01 ~]# ceph -s
) v8 B; R; [' K) Q3 ^2 u  cluster:
( \; [% w, J: d3 D! N    id:     a6d0359e-8295-4e62-8620-54273462e79f
( D8 \, d; x7 ^& \6 {) ?2 Z; ?2 s+ D    health: HEALTH_WARN
* o* D) a& x, U0 M2 }7 |7 u6 o            noout flag(s) set# q  V" i6 L* I" M
            Reduced data availability: 3 pgs peering5 q5 ?4 v) j1 e0 y
            Degraded data redundancy: 2173/96486 objects degraded (2.252%), 125 pgs degraded, 133 pgs undersized( c7 U' x  C+ _9 p, z: Z0 z( G* |
            6 pgs not deep-scrubbed in time$ `# D8 b' Q; O3 ~; u% p3 u
            clock skew detected on mon.compute02, mon.compute03
7 _: `* z6 s1 _6 {$ i) l2 n7 Y0 L& X
  services:
+ w' G; e' N' w; e6 ~" I6 n; D    mon: 3 daemons, quorum compute01,compute02,compute03 (age 24m)0 t% m2 u. h- Y0 U, D6 Z
    mgr: compute02(active, since 4d), standbys: compute03, compute01! Y# s; ^! j! m4 i& |
    osd: 6 osds: 6 up (since 24m), 6 in (since 2w); 104 remapped pgs
' Z& c+ H, Q% V         flags noout. j* a, @$ s/ Q. s5 v) n$ W
& E: j7 q9 |- Z' A" X- w4 h
  data:
! A) l7 Q, `" J+ k( s0 J% ]    pools:   5 pools, 1408 pgs
: j7 H0 u) G. E  g    objects: 32.65k objects, 132 GiB+ z" D- ]8 s! N/ P" W# {+ t
    usage:   383 GiB used, 4.9 TiB / 5.2 TiB avail0 e+ f$ y+ E1 K& D" {/ I' U
    pgs:     0.568% pgs not active
# z3 m7 }$ `- N             2173/96486 objects degraded (2.252%)
( {1 I4 v' S3 ]: V1 b             1267 active+clean8 x3 w+ A2 @2 F$ X
             62   active+undersized+degraded+remapped+backfill_wait
8 h% ^8 \/ O" y( P; ]             62   active+recovery_wait+undersized+degraded+remapped2 F1 }. Z; h$ C* Y1 W
             8    active+undersized+remapped  {% m7 f9 d, o  z0 r
             8    peering
( v, \& X4 A! e# O; C             1    active+recovering+undersized+degraded+remapped
3 Z0 h( C' Y. ?  f7 M: S0 h" A- I# `7 b8 Q. [
  io:
6 U6 s, V8 M  E; V- J    client:   407 KiB/s rd, 792 KiB/s wr, 34 op/s rd, 52 op/s wr
! @+ U) V& m6 a* L( t    recovery: 72 MiB/s, 2 keys/s, 22 objects/s5 [. Z( h; z$ h4 e
: S  H0 B0 G2 @. Y1 p; a) f9 Z

7 c" {  R1 z: f% f1 B( p查看时间:
# N: u9 |6 p( c- ~[root@compute01 ~]# date
7 E4 H5 O/ C# Q$ k7 Z2 J3 }Mon Dec 20 10:32:23 CST 2021
: f, w# Y3 e- W, G[root@compute01 ~]# date
6 t+ I5 a5 I* ]Mon Dec 20 10:32:38 CST 20219 w9 j5 g: C" m. D  P
[root@compute01 ~]# date
; ^2 [, k& K! W6 K" IMon Dec 20 10:33:00 CST 2021
) A( X$ h* w6 H( [
) \' J2 |& b; q; V[root@compute01 ~]# chronyc sources $ j2 \$ O. ?; c/ G7 g5 _1 K: G: w
210 Number of sources = 2
$ _- E. A: u' f) g8 hMS Name/IP address         Stratum Poll Reach LastRx Last sample               
5 z$ U& `1 h4 W8 V===============================================================================4 }) e+ Y( m! O# V8 J' c
^? compute01                     0   9     0     -     +0ns[   +0ns] +/-    0ns
& P8 I; ]( k1 L' |; i8 |^? compute02                     0   9     0     -     +0ns[   +0ns] +/-    0ns- q4 [4 X" B0 p. d
[root@compute01 ~]# vim /etc/chrony.conf 0 U4 m, J% a) {% J; l, s. C
3 Q! J: b7 J' v2 d" e% b; F& z
[root@compute01 ~]# systemctl restart chronyd.service 4 B! l. M; M- O; m% A" l
[root@compute01 ~]# chronyc sources % i' J6 a2 R5 T6 ^
210 Number of sources = 2
2 d6 v; @" ^1 R" WMS Name/IP address         Stratum Poll Reach LastRx Last sample                 T: N. L$ ^" A6 ~+ J9 a4 k& F
===============================================================================
( ^$ K4 Y  d: R8 G^* compute01                    10   6    17     3  +2667ns[-7752ns] +/-   28us
, Z" f/ g5 m" I4 u& q^? compute02                     0   6     0     -     +0ns[   +0ns] +/-    0ns! O9 I$ v8 {, v. t
[root@compute01 ~]# $ T- T* k. O: c5 O4 Q$ j0 _* T
[root@compute01 ~]#
; U: z3 @& h2 t2 H! l3 Q[root@compute01 ~]# vim /etc/chrony.conf ; r7 C% x, D$ Z- u! O/ ^
[root@compute01 ~]# date
3 O5 T% s/ w" d' z  z7 y- Y! oMon Dec 20 10:36:29 CST 2021
2 I# R8 M* C4 ~0 u4 d6 \9 l$ v/ o8 h, Z2 R+ u- _$ ~4 Z) q" {8 V4 f
/ U  K2 I* {9 j5 y
时间同步即可:
. X! ^. x3 y7 z; T5 I% F/ c
5 q5 y% R) Y  T6 f$ L) k& t3 C[root@compute01 ~]# ceph -s& _  ~; Z* J, Z- G
  cluster:
% t7 Y: L7 Q6 c" P+ ?' b    id:     a6d0359e-8295-4e62-8620-54273462e79f0 }) k# T" n8 i  s( g
    health: HEALTH_WARN- K0 q- B$ \2 G  C7 ^
            noout flag(s) set
0 Y! T# M; R5 i' g0 J/ V            1 pgs not deep-scrubbed in time
/ f# g! [$ X0 d* A+ O* N( B! ?
  services:
+ @1 N$ R. m: E. l6 \3 G    mon: 3 daemons, quorum compute01,compute02,compute03 (age 48s)
! g/ [; r+ G8 A; g8 k; ^    mgr: compute02(active, since 4d), standbys: compute03, compute01
, t  ^+ f$ L# M    osd: 6 osds: 6 up (since 28m), 6 in (since 2w); N# l: F. i2 y2 l6 O) }2 H; Q
         flags noout
$ X) `% w( F4 d, z8 L/ j7 P
) K. g5 N1 v) A" P4 b- f1 z1 Z  data:
- _% a7 ~' e& j$ b4 a1 v( y    pools:   5 pools, 1408 pgs
+ {0 W8 d% V4 r, h    objects: 32.65k objects, 132 GiB
" t/ M3 O. q* N    usage:   384 GiB used, 4.9 TiB / 5.2 TiB avail1 l1 o, z0 ]/ {9 f$ k
    pgs:     1407 active+clean
6 f7 n, ~3 m* \4 d1 m% W2 n             1    active+clean+scrubbing+deep
  m* X& a8 P" o7 @5 A/ |) w+ a2 q& H8 G2 l) h% s& C% X- g5 w( R
  io:! z+ R8 M0 V& F! u, e4 T; n+ `/ E
    client:   20 KiB/s rd, 499 KiB/s wr, 4 op/s rd, 18 op/s wr
& |* ~: k/ k% n" G9 A5 g8 k& O* f4 ]7 L/ C  }! g
查看ceph存储是否正常:
" K# Y6 J( ?3 e7 y# ?
, {6 e, e# d1 N1 `' W2 Y; S. ?[root@compute01 ~]# ceph -s& x& D5 F5 G: I/ b+ ?  d+ d
  cluster:
( C' B2 C4 i) F1 a& E    id:     a6d0359e-8295-4e62-8620-54273462e79f
# d/ M( C4 [0 A9 H    health: HEALTH_WARN4 d/ n+ g5 s' k2 M. t
            noout flag(s) set
& K1 A. w; c0 B            1 pgs not deep-scrubbed in time& j, ?% a3 ]+ E: B2 r
& a: e3 |* w) @- ]' k
  services:' l( m& J2 d& _9 v- R: {
    mon: 3 daemons, quorum compute01,compute02,compute03 (age 60s)" k- Q/ d% P+ L7 F
    mgr: compute02(active, since 4d), standbys: compute03, compute018 B- N; y2 e7 g. M
    osd: 6 osds: 6 up (since 28m), 6 in (since 2w)' W6 p6 D  S; Q
         flags noout
  k7 Z1 w( n7 W' _9 l: i, R
/ [- `" X3 H' ~7 F: N  data:
1 D' a: ~8 p& T, `6 T: h3 ]    pools:   5 pools, 1408 pgs* ^6 n% n+ G- ?8 c# `1 i9 }
    objects: 32.65k objects, 132 GiB8 V6 T6 Z& B1 y) c* t3 l
    usage:   384 GiB used, 4.9 TiB / 5.2 TiB avail
4 U3 H- X8 i1 k( B    pgs:     1407 active+clean
" r1 J& [) s2 K! Q; ]2 o: F3 z             1    active+clean+scrubbing+deep/ ]$ G( M" `$ V$ k7 ?

, @* r: W2 J" T; V7 z, N4 S$ X. R3 f  io:
5 h  x! G- q* @8 B* r    client:   120 KiB/s rd, 595 KiB/s wr, 13 op/s rd, 33 op/s wr
# t2 T# `  Y- s1 `8 i
' ]3 b$ g, P3 z. e+ C) i& M$ b2 `/ V1 E8 j
[root@compute01 ~]# ceph osd unset noout
1 q* v/ A! T. U9 I9 Q' `noout is unset
( [7 {* J- O3 D% p% O* ?6 Y# Y1 M) }5 x7 ]9 R' H9 f2 [4 [
检查状态:2 K0 i0 ?+ `+ O- G7 E- m* F1 E
[root@compute01 ~]# ceph -s
: E8 ]6 I" {3 x( U  cluster:* \+ e, c3 ~  t
    id:     a6d0359e-8295-4e62-8620-54273462e79f! J* `0 o+ p1 K6 N8 u/ k2 h, x
    health: HEALTH_OK
1 p3 h' m. s7 F, V" k6 m" Z- ]5 @' L# w1 b/ Z( H
  services:
4 B; d( L: Q5 C1 Y0 P8 ~# \! g    mon: 3 daemons, quorum compute01,compute02,compute03 (age 100s)
! F; t$ d/ c! Q5 H1 Y# Y9 l    mgr: compute02(active, since 4d), standbys: compute03, compute01
+ X$ L$ e7 g  K4 @$ [) @9 s, u    osd: 6 osds: 6 up (since 29m), 6 in (since 2w)
. F. `  T/ {& U/ l, i( q; }9 a# I" T( g9 [4 u; b8 i! q
  data:% L# M3 o: v7 W
    pools:   5 pools, 1408 pgs# t6 o* ]5 v% F# |
    objects: 32.65k objects, 132 GiB8 ?- W& Q$ i4 V* g- j6 E
    usage:   384 GiB used, 4.9 TiB / 5.2 TiB avail
$ {) z  P- y! U! Z    pgs:     1408 active+clean
6 a1 i. a2 w/ \2 j4 g
1 e& j2 z' n0 H5 r4 G$ T- I  io:
6 F5 g+ B7 L, N; N0 T! z    client:   539 KiB/s rd, 285 KiB/s wr, 191 op/s rd, 26 op/s wr, V- l8 Z2 Z" M# `+ d
* k# O; _& ?4 N: X# Q
[root@compute01 ~]# ceph -s$ H* h  @7 n1 U) P5 U- E% a
  cluster:. S8 z3 x/ a! m9 a! G" V
    id:     a6d0359e-8295-4e62-8620-54273462e79f
+ t* m( k# }  q$ o+ J8 D  [    health: HEALTH_OK
( ^  R# q. k# G0 k+ Y7 A: d, p! V. r" ^. f& p) V
  services:6 \) R/ D- K7 B7 ^. O* ^8 o
    mon: 3 daemons, quorum compute01,compute02,compute03 (age 102s)
3 B7 W8 q3 O% r6 z* F* b  o7 `    mgr: compute02(active, since 4d), standbys: compute03, compute01
( h' J, [0 U+ U8 p7 u  `6 C    osd: 6 osds: 6 up (since 29m), 6 in (since 2w)
; `% w1 K$ P; ]7 _" M( S
" m+ V% B0 q! R- k  data:; u& x: h1 z" Q1 d: b
    pools:   5 pools, 1408 pgs5 I5 |5 D0 E' T2 j' F" @
    objects: 32.65k objects, 132 GiB
  S# [- Y7 f. U7 I    usage:   384 GiB used, 4.9 TiB / 5.2 TiB avail; r! D$ z1 |( k6 T; x- K: ^% @# ~; G  U
    pgs:     1407 active+clean
2 H$ s6 ^3 [0 X             1    active+clean+scrubbing+deep4 `' o1 p; }3 ^
, t3 l. I# d* f5 R2 C6 j( g
  io:5 \& W5 \  g% ?( _: J
    client:   110 KiB/s rd, 196 KiB/s wr, 12 op/s rd, 18 op/s wr
. F% H8 D; G7 ]) q4 L; o5 u* r- a' Z7 S+ [
[root@compute01 ~]# ceph -s
" y7 @3 y8 ~2 `8 @# ~" p  R& G  cluster:
( J% a  C1 ^% h+ H, X) G. v$ h    id:     a6d0359e-8295-4e62-8620-54273462e79f* ~. G( e+ O; k( p0 r$ _
    health: HEALTH_OK
( @3 B" h, ?4 M9 y; r5 j2 D" Z+ h* v) T$ |; ^
  services:
- T4 m' z% y# T3 V+ }; {; ~. s  K( A    mon: 3 daemons, quorum compute01,compute02,compute03 (age 105s)
$ t+ [6 E, [% n. @6 R6 K( F    mgr: compute02(active, since 4d), standbys: compute03, compute01+ f" t" f+ C. p9 g9 K) d% f  f
    osd: 6 osds: 6 up (since 29m), 6 in (since 2w)0 k0 I$ V% R7 C! U

" r2 \! {; s' e/ v. b' {6 |  data:. k. t9 V7 z5 f
    pools:   5 pools, 1408 pgs
7 D# ?6 y" B' K. M, ?    objects: 32.65k objects, 132 GiB
5 ^* u( U: a: o2 i2 F. u7 L* i. p8 R    usage:   384 GiB used, 4.9 TiB / 5.2 TiB avail
6 D# o9 C8 @. T# l. l( r    pgs:     1407 active+clean
7 s; U: W4 f1 R+ ~! E" ?             1    active+clean+scrubbing+deep& T5 q7 [3 Y- {. t

* \/ \3 e( E7 U- U) g% N' [  io:
9 _) x" X) B( a, D) R; g( Z( p    client:   74 KiB/s rd, 193 KiB/s wr, 10 op/s rd, 18 op/s wr) V9 [: O- ^& I: Y4 ~9 D

. j% V5 h+ E' W1 M) I[root@compute01 ~]# ceph -s/ k4 G6 c. h2 T9 l7 z8 y8 w
  cluster:9 @/ }' W* t! Z' ~4 ?; H; _
    id:     a6d0359e-8295-4e62-8620-54273462e79f
* x1 P& B9 k# P; m, ]0 M6 v) G' I, I    health: HEALTH_OK6 q9 I% ?- N# \  Q
; O9 q& J8 l9 `; y
  services:
7 [- v- j' z, c    mon: 3 daemons, quorum compute01,compute02,compute03 (age 3m)
. k, C! [* W/ z9 j3 t$ ~* W1 w5 L    mgr: compute02(active, since 4d), standbys: compute03, compute01
) _9 x% z7 k8 F+ M: \( A! |    osd: 6 osds: 6 up (since 30m), 6 in (since 2w). i, I* r0 ~3 n' @3 m% O! X6 f

" B5 ?# w9 z. B! F+ {  data:  f% k( x/ I" j  ^
    pools:   5 pools, 1408 pgs! T4 F1 Y4 p5 P6 N+ x
    objects: 32.65k objects, 132 GiB
: S: _( }9 Y4 o    usage:   384 GiB used, 4.9 TiB / 5.2 TiB avail
! ~. z! p. j, D. m    pgs:     1408 active+clean: x# ]' J" H6 l& L

) n" n( B9 }$ H# `5 k# X; |  io:
5 ^" ^8 T! w8 V! N" S7 p/ t* M    client:   840 KiB/s rd, 1.4 MiB/s wr, 63 op/s rd, 43 op/s wr, {7 s$ n, [' I3 I) }
: q, ?9 g3 G9 u$ T
. g6 h7 g% a; a
' M) K2 S6 u  ?' d6 T, ]) [
" V% b, }  W; D
总结,此过程中出现的问题,一是时间同步服务器有问题,二是ceph 设置了不剔除osd,不同步,不分配问题。
8 C) W3 W9 N5 i" V1 k& t
9 Q3 g4 L1 }0 ?! l. H
& Z* l: E: Y$ a8 u5 U5 ^2 H# Y* q- c+ [; z

1

主题

0

回帖

12

积分

管理员

积分
12
QQ
 楼主| 发表于 2021-12-20 11:43:30 | 显示全部楼层
ceph存储同步数据需要时间同步,如果时间不同步,需要调整时间同步服务器,检查为什么不能同步时间。   以前chrony服务配置的时候,不用设置allow ,现在可能需要设置。或者以前也需要设置,只是没有注意这个细节性的地方。' \3 ~) V. `7 q- H
同步时间服务器,第一修改配置,第二检查配置allow 允许的范围,第三其他节点同步时间。第四启动服务。
3 r) {' R4 q" h! R
您需要登录后才可以回帖 登录 | 注册

本版积分规则

返回首页|Archiver|手机版|小黑屋|易陆发现技术论坛 ( 蜀ICP备2026014127号-1 )

GMT+8, 2026-6-12 02:04 , Processed in 0.015536 second(s), 22 queries .

Powered by Discuz! X5.0

© 2001-2026 Discuz! Team.

快速回复 返回顶部 返回列表