ceph health clock skew detected

admin

MON_CLOCK_SKEW: clock skew detected on mon.ch-mon-1, mon.ch-mon-2
ceph -s                                                                       Fri Sep 28 13:57:25 2018
" f6 ^9 k" @3 u  C  cluster:
( s1 \* H5 W# K7 ?4 B    id:     e6d7621d-4b97-4791-9d9b-92829a6f3044
+ @- x  B6 T( U- G' i! e8 J1 v2 L    health: HEALTH_WARN
            clock skew detected on mon.ch-mon-1, mon.ch-mon-2
5 O8 C$ x7 s4 \' V( @2 z  services:
! Z) f7 X8 S4 L  U+ |- W  x( P& i    mon: 7 daemons, quorum ch-osd-1,ch-osd-2,ch-osd-3,ch-osd-4,ch-mon-1,ch-mon-2,ch-mon-3
$ q) P( E2 ^3 T0 I) ~% P3 a" x: a    mgr: ch-mon-1(active)
# Y; Z: ^. `' T3 J: j7 S1 L0 g5 X; W8 }. W    osd: 12 osds: 12 up, 12 in
! X$ R+ q8 o8 S/ T) V- S+ X( C  data:
    pools:   0 pools, 0 pgs
% O3 [' ?; p# x' a) ?$ }  D4 Y    objects: 0 objects, 0B
3 J+ W1 c: ]* ~4 W  n    usage:   12.1GiB used, 1.16TiB / 1.17TiB avail
    pgs:

admin

ceph health detail
4 F+ U9 c' K2 l& G5 pHEALTH_WARN clock skew detected on mon.ceph2, mon.ceph3, mon.ceph4
+ J) x7 l# t- R' a2 S  T* `MON_CLOCK_SKEW clock skew detected on mon.ceph2, mon.ceph3, mon.ceph4
    mon.ceph2 addr 192.168.90.86:6789/0 clock skew 0.0630122s > max 0.05s (latency 0.00304483s)
2 P1 a5 E) P1 B8 `5 E    mon.ceph3 addr 192.168.90.87:6789/0 clock skew 0.076804s > max 0.05s (latency 0.00364158s)
    mon.ceph4 addr 192.168.90.88:6789/0 clock skew 0.115598s > max 0.05s (latency 0.00468623s)

7 q8 j% B$ i) m* w: G( z1 I7 B0 s
解决办法是：
0 w0 A5 ?- L/ h停掉服务ntp
# systemctl stop ntpd.service
同步时间：
) d9 ?5 R1 @4 J ntpdate -u 192.168.90.85
29 Sep 15:32:31 ntpdate[4396]: adjust time server 192.168.90.85 offset 0.034117 sec
! H! I) A( `: a% \) ?' L& g启动ntpd服务
* x6 ^8 F1 z) L4 T systemctl start ntpd.service

6 Q5 G) ]& f  k% m. C0 R8 `( |5 i4 eceph health detail
HEALTH_WARN clock skew detected on mon.ceph3, mon.ceph4
5 b/ V6 A0 d9 x3 l" u2 H8 qMON_CLOCK_SKEW clock skew detected on mon.ceph3, mon.ceph4
9 P# ^) h* T" y$ M$ Y( s    mon.ceph3 addr 192.168.90.87:6789/0 clock skew 0.0640733s > max 0.05s (latency 0.00336378s)
    mon.ceph4 addr 192.168.90.88:6789/0 clock skew 0.115242s > max 0.05s (latency 0.00423552s)

admin

systemctl restart ceph-mon.target
9 a, z9 a( z2 X7 D6 F1 A8 q& e' q% x. U如果还是出现这样的问题，重启下ceph的mon服务
ceph health detail
HEALTH_OK
[root@ceph1 ~]# ceph -s
  cluster:
    id:     9b8bcc5e-a0cf-4617-9e14-78acaf79e690
5 m. C- T9 e, N7 s8 H    health: HEALTH_OK
2 k4 d9 D2 [; r4 s4 W# ~
' M% Y) F* F  E+ ]1 ?  services:
    mon: 4 daemons, quorum ceph1,ceph2,ceph3,ceph4
    mgr: ceph1(active), standbys: ceph4, ceph2, ceph3
    osd: 8 osds: 8 up, 8 in
( T+ K' n( B7 ]/ [$ n* ]5 P$ E( R
# E6 E* G* v) J/ @% D3 |# D  data:
3 r' U6 N" I4 R  r. ^6 G% w    pools:   5 pools, 448 pgs
    objects: 0 objects, 0B
    usage:   876MiB used, 799GiB / 800GiB avail
    pgs:     448 active+clean
4 Y/ @/ }# `/ l
" {3 p/ O7 D$ E; w8 e! D* q
, \  J9 S8 A* B" b* ^3 u然后再启动ntpd
0 q7 K- F( C8 W* ~: T, C( Bsystemctl start ntpd.service