ceph health clock skew detected

admin

MON_CLOCK_SKEW: clock skew detected on mon.ch-mon-1, mon.ch-mon-2
ceph -s                                                                       Fri Sep 28 13:57:25 2018
  cluster:
6 z" D; W+ Z5 i6 G' B    id:     e6d7621d-4b97-4791-9d9b-92829a6f3044
6 }  H! J  _) [9 d( C    health: HEALTH_WARN
            clock skew detected on mon.ch-mon-1, mon.ch-mon-2
& T) C9 O1 q' b  services:
! E; @3 j" |3 A1 P, B    mon: 7 daemons, quorum ch-osd-1,ch-osd-2,ch-osd-3,ch-osd-4,ch-mon-1,ch-mon-2,ch-mon-3
. t; m; o+ q. G) x* D  r1 V    mgr: ch-mon-1(active)
    osd: 12 osds: 12 up, 12 in
  data:
% |  R, r! |/ T5 ^5 T. Q3 X    pools:   0 pools, 0 pgs
    objects: 0 objects, 0B
    usage:   12.1GiB used, 1.16TiB / 1.17TiB avail
    pgs:

admin

ceph health detail
HEALTH_WARN clock skew detected on mon.ceph2, mon.ceph3, mon.ceph4
5 X' b% C" h7 b+ RMON_CLOCK_SKEW clock skew detected on mon.ceph2, mon.ceph3, mon.ceph4
8 j9 y# L/ a( g, n  S    mon.ceph2 addr 192.168.90.86:6789/0 clock skew 0.0630122s > max 0.05s (latency 0.00304483s)
    mon.ceph3 addr 192.168.90.87:6789/0 clock skew 0.076804s > max 0.05s (latency 0.00364158s)
1 z$ E8 r* s- c! Q* a# p3 F    mon.ceph4 addr 192.168.90.88:6789/0 clock skew 0.115598s > max 0.05s (latency 0.00468623s)

5 E: R& @$ X  x/ E0 [7 @
解决办法是：
停掉服务ntp
# systemctl stop ntpd.service
同步时间：
ntpdate -u 192.168.90.85
29 Sep 15:32:31 ntpdate[4396]: adjust time server 192.168.90.85 offset 0.034117 sec
启动ntpd服务
systemctl start ntpd.service
( n! K# {: \0 K2 ^: f6 k
) ?2 F9 R6 C/ \: Lceph health detail
HEALTH_WARN clock skew detected on mon.ceph3, mon.ceph4
9 K5 w% P5 v: h4 t% D$ AMON_CLOCK_SKEW clock skew detected on mon.ceph3, mon.ceph4
% `" F; l# ~9 b: z+ E, l, H- F    mon.ceph3 addr 192.168.90.87:6789/0 clock skew 0.0640733s > max 0.05s (latency 0.00336378s)
4 i, g2 C: l2 o0 G* T    mon.ceph4 addr 192.168.90.88:6789/0 clock skew 0.115242s > max 0.05s (latency 0.00423552s)
1 Y& g) g) S  }" u4 R
+ y9 a3 i8 h. b9 ?- O0 ?: }

admin

systemctl restart ceph-mon.target
如果还是出现这样的问题，重启下ceph的mon服务
0 ?8 E* [% u* P! f' [1 d4 C+ e) C ceph health detail
HEALTH_OK
[root@ceph1 ~]# ceph -s
  cluster:
1 r5 |& E" y" ]: w    id:     9b8bcc5e-a0cf-4617-9e14-78acaf79e690
    health: HEALTH_OK

8 r1 P/ k* T' Y1 t' }, H  services:
    mon: 4 daemons, quorum ceph1,ceph2,ceph3,ceph4
    mgr: ceph1(active), standbys: ceph4, ceph2, ceph3
    osd: 8 osds: 8 up, 8 in
3 ~2 B! X1 o4 P) H+ q" k/ f4 Z
  data:
    pools:   5 pools, 448 pgs
    objects: 0 objects, 0B
    usage:   876MiB used, 799GiB / 800GiB avail
    pgs:     448 active+clean

8 Q3 x4 E" E8 t* F- {& d+ g
然后再启动ntpd
systemctl start ntpd.service