ceph 集群处理stale的pg
/ f+ U( |" P6 E2 w6 n+ L5 Z- ]9 |( D, M处理过程- 首先用ceph pg dump|grep stale 找出所有的stale的pg
- 然后用 ceph force_create_pg pg_id% g7 A6 d- p) K7 p' U
如果做到这里，可以看到之前的stale的状态的PG，现在已经是creating状态的了，这个时候一个关键的步骤需要做下： - 重启整个集群的OSD
/ y: F8 \1 M1 J3 i' P( y A
在重启完成了以后，集群的状态就会恢复正常了，也能够正常的写入新的数据了
5 @& l; A/ i; s% ]' N0 |) n7 v[root@mon1 ~]# ceph pg dump |grep stale
( c$ r! X0 Z0 {& Z* ?
dumped all7.385 19460 0 0 0 0 4363375984 1547 1547 stale+peering 2022-08-07 18:30:16.932885 9719'4511237 110154:5399674 [14] 14 [14] 14 9719'4511237 2022-08-06 07:29:51.095989 9719'4511237 2022-08-02 00:57:43.318114 0 7.2a6 19409 0 0 0 0 4324918151 1542 1542
1 F3 A0 K/ Q5 Y5 m
stale+peering 2022-08-07 16:09:26.464409 5938'4407602 99931:5800676 [15] 15 [15] 15 5938'4407602 2022-08-06 07:36:00.102984 5938'4407602 2022-08-01 23:45:58.573722 0 8.39 280 0 0 0 0 0 1597 1597 stale+peering 2022-08-07 16:09:26.461915 5938'2119986 99931:2386270 [15] 15 [15] 15 5938'2119986 2022-08-05 21:32:12.656384 5938'2119986 2022-08-01 22:58:58.614188 0 7.34 19337 0 0 0 0 4278284806 1580 1580
$ w0 F/ H" o6 Y3 T4 a) X( `
stale+peering 2022-08-07 16:09:26.461100 9719'4369235 99931:5261881 [15] 15 [15] 15 9719'4369235 2022-08-06 08:22:37.168815 9719'4369235 2022-08-04 21:34:38.449584 0 7.1d8 19383 0 0 0 0 4332924749 1593 1593
, Q5 z! U/ ]1 y8 |: P/ I
stale+peering 2022-08-07 18:30:16.914876 9719'4456286 110154:5409919 [14] 14 [14] 14 9719'4456286 2022-08-06 09:09:03.624425 9719'4456286 2022-08-02 01:25:18.343799 0 7.1e6 19375 0 0 0 0 4342149879 1564 1564 stale+peering 2022-08-07 18:30:16.930931 10754'4463130 110154:5047778 [14] 14 [14] 14 10754'4463130 2022-08-06 01:41:35.137028 10754'4463130 2022-08-04 21:39:21.624235
9 P+ H+ x: }. v) E- N# m( `; V; u# }2 r/ ]
[root@mon1 ~]# ceph pg 7.385 query
) t& l/ j6 G ^
Error ENOENT: i don't have pgid 7.385
4 N$ _- F1 K4 X ^$ O- }
[root@mon1 ~]# ceph pg 7.385 query Error ENOENT: i don't have pgid 7.385
$ \8 C2 D' R( ?
[root@mon1 ~]# ceph pg 7.2a6 query 8 @; p( b2 T+ E- W
Error ENOENT: i don't have pgid 7.2a6 [root@mon1 ~]# [root@mon1 ~]# [root@mon1 ~]# [root@mon1 ~]# cd /backup/ 1 z: F" `7 j6 c3 l
[root@mon1 backup]# ls 2 I/ |4 H. `5 W. C* W9 P
osd14pgs osd15pgs pgback-osd14 pgback-osd15 pgexport.sh 5 n# P. N3 P$ h# u9 I3 j- B3 _
[root@mon1 backup]# cd pgback-osd14/ 2 [+ P9 A) F5 {. @( Q4 q
[root@mon1 pgback-osd14]# ls |grep 7.385osd14pg-7.385.file 4 P9 G( J) L) k; T) N9 C6 F. i
一定要有这样的提示，才能回滚： Error ENOENT: i don't have pgid 7.385 先停止osd服务： ! e' W0 a! ~+ o# r! n; b& Z
导出ceph-remove [root@mon1 pgback-osd14]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-14/ --id=14 --op export-remove --pgid 7.385 --file /tmp/osd14-7.385pg * ^6 Y3 k& ~: ?, j$ G
导入pg ! b9 o1 U$ q. K# C2 H" M0 _5 q
[root@mon1 pgback-osd14]# sudo -u ceph ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-14/ --id=14 --op import --pgid 7.385 --file osd14pg-7.385.file 8 l: V$ D" |' o2 C- R
启动osd：
# x( _( X" |; a& h) y9 T7 s+ o9 A" @5 M/ B0 A
2 n8 X" {' }5 j! v0 Q
G9 g) W0 O, y8 G$ a2 K0 |2 m
1 l `2 d7 n6 m. f& p* y3 }; o! h没有的需要重建：* q/ ~& q5 D7 ^/ z$ {1 j" V" V
处理方法： 第一步，找到stale状态pg ceph pg dump |grep stale 第二步：重新创建pg ceph force_create_pg $pg_id
9 Q- P4 n! i# Y# h9 @; S% T* ~6 m" r
|
发表于 2022-8-10 10:39:18