易陆发现互联网技术论坛

 找回密码
 开始注册
查看: 3|回复: 0
收起左侧

notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: unknown erro

[复制链接]
发表于 2024-6-7 15:00:19 | 显示全部楼层 |阅读模式

马上注册,结交更多好友,享用更多功能,让你轻松玩转社区。

您需要 登录 才可以下载或查看,没有账号?开始注册

x
Jun  7 14:35:17 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2.log
' h6 i/ \' u3 X) }Jun  7 14:35:17 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2-sasl.log
2 q+ K0 Q( f  X* A6 bJun  7 14:35:18 controller2 systemd: Started RabbitMQ broker.( c  E) c0 d1 q
Jun  7 14:35:18 controller2 crmd[6475]:  notice: process_lrm_event: Operation ksc-os-rabbitmq_start_0: ok (node=controller2.ksc, call=548943, rc=0, cib-update=594832, confirmed=true)  p: ^9 N* c3 P8 [, g1 `: d* q
Jun  7 14:35:18 controller2 rabbitmq-server: {"init terminating in do_boot",{rabbit,failure_during_boot,{could_not_start,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot,{boot_step,recovery,{badmatch,{error,{{{badmatch,{error,{not_a_dets_file,"/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},[{rabbit_recovery_terms,open_table,0,[{file,"src/rabbit_recovery_terms.erl"},{line,130}]},{rabbit_recovery_terms,init,1,[{file,"src/rabbit_recovery_terms.erl"},{line,111}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]},{child,undefined,rabbit_recovery_terms,{rabbit_recovery_terms,start_link,[]},transient,4294967295,worker,[rabbit_recovery_terms]}}}}}}}}}}}}% Z# X/ r* x% |3 {( P0 r
Jun  7 14:35:18 controller2 crmd[6475]:  notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: ok (node=controller2.ksc, call=548944, rc=0, cib-update=594833, confirmed=false)) Y6 t" A$ n/ z0 {, m+ x0 V* h
Jun  7 14:35:19 controller2 filebeat: 2024-06-07T14:35:19.300+0800        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":890090,"time":{"ms":4}},"total":{"ticks":2986430,"time":{"ms":11},"value":2986430},"user":{"ticks":2096340,"time":{"ms":7}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":28},"info":{"ephemeral_id":"2ea13f88-dfe4-4d82-bb66-f2726d6d1965","uptime":{"ms":4384740183}},"memstats":{"gc_next":33883984,"memory_alloc":18558592,"memory_total":69310713440},"runtime":{"goroutines":133}},"filebeat":{"harvester":{"open_files":17,"running":10}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":13,"events":{"active":4126,"retry":100}}},"registrar":{"states":{"current":13}},"system":{"load":{"1":2.47,"15":2.54,"5":2.58,"norm":{"1":0.0618,"15":0.0635,"5":0.0645}}}}}}  `$ ~1 ?/ u6 t2 t- x3 |) j5 U/ p! P
Jun  7 14:35:19 controller2 rabbitmq-server: Crash dump was written to: erl_crash.dump
" t7 p+ }' c9 ]5 P- cJun  7 14:35:19 controller2 rabbitmq-server: init terminating in do_boot ()
- E& b& d: T4 f$ ZJun  7 14:35:19 controller2 systemd: rabbitmq-server.service: main process exited, code=exited, status=1/FAILURE2 k- v5 \' Q6 r6 S& f  B4 n8 E
Jun  7 14:35:20 controller2 rabbitmqctl: Stopping and halting node 'rabbit@controller2' ...
! i4 K$ Y- B2 M1 I  \4 H, u% F# ^6 @Jun  7 14:35:20 controller2 rabbitmqctl: Error: unable to connect to node 'rabbit@controller2': nodedown
( k; F  z6 x; W3 A! e" R: `Jun  7 14:35:20 controller2 rabbitmqctl: DIAGNOSTICS
9 A2 M7 P' t; _2 @$ _3 @% WJun  7 14:35:20 controller2 rabbitmqctl: ===========
. {0 K! G  K$ |8 Y# z$ i7 q" RJun  7 14:35:20 controller2 rabbitmqctl: attempted to contact: ['rabbit@controller2']) M9 ]* b! i* {0 z, {! m2 H( V
Jun  7 14:35:20 controller2 rabbitmqctl: rabbit@controller2:  j! e8 M5 j& G7 Y3 h& H
Jun  7 14:35:20 controller2 rabbitmqctl: * connected to epmd (port 4369) on controller29 |% N* `$ Y. k/ U; d
Jun  7 14:35:20 controller2 rabbitmqctl: * epmd reports: node 'rabbit' not running at all
% D* o' v) s* T9 v2 z2 U: wJun  7 14:35:20 controller2 rabbitmqctl: no other nodes on controller2
# O; g0 r( y4 m1 J" K% M* bJun  7 14:35:20 controller2 rabbitmqctl: * suggestion: start the node+ E) w  `- \* M
Jun  7 14:35:20 controller2 rabbitmqctl: current node details:' q8 l  u* E* ~  b
Jun  7 14:35:20 controller2 rabbitmqctl: - node name: 'rabbitmqctl-19643@controller2'
  X8 \1 b" O% [7 |% |( [* y/ BJun  7 14:35:20 controller2 rabbitmqctl: - home dir: /var/lib/rabbitmq
1 k/ m2 l. r( WJun  7 14:35:20 controller2 rabbitmqctl: - cookie hash: YqnFeQ57nhtexGsA970OWw==! s" p) U/ J: g' E( {
Jun  7 14:35:20 controller2 systemd: rabbitmq-server.service: control process exited, code=exited status=2
% B! B' E# D: |, r8 ]- fJun  7 14:35:20 controller2 systemd: Unit rabbitmq-server.service entered failed state.3 O! v  a2 R2 P  g' Q+ s" K
Jun  7 14:35:20 controller2 systemd: rabbitmq-server.service failed.
0 Q- o5 t; l' x; |2 ]" f
+ b$ o; \8 p6 t. G: U, ]8 m& J! F$ U) c8 u& C1 g

; m# N# r0 @7 U0 wJun  7 14:35:23 controller2 dhcpd: isc-dhcpd-4.2.5
7 Y2 B5 h! d# J+ Q5 eJun  7 14:35:25 controller2 systemd: Starting RabbitMQ broker...
; q& i' a+ `5 L1 {" Y/ K6 YJun  7 14:35:25 controller2 ocf-mysql-wss: INFO: GTID OK: 62a7fc52-52de-11e8-a423-ee3e8b5b2a73:2044451984/ h$ k" ]( f2 j
Jun  7 14:35:25 controller2 ocf-mysql-wss: INFO: Galera GTID: 62a7fc52-52de-11e8-a423-ee3e8b5b2a73:20444519841 N5 ~4 D8 t5 B
Jun  7 14:35:26 controller2 rabbitmqctl: Waiting for 'rabbit@controller2' ...
' B/ \6 X- C: l5 _- J- ]4 eJun  7 14:35:26 controller2 rabbitmqctl: pid is 20016 ...
' O. L# S4 ~- O$ vJun  7 14:35:27 controller2 rabbitmq-server: RabbitMQ 3.4.0. Copyright (C) 2007-2014 GoPivotal, Inc.2 f; {3 P% r( t( v' b
Jun  7 14:35:27 controller2 rabbitmq-server: ##  ##      Licensed under the MPL.  See http://www.rabbitmq.com/
  Y: m) \# j* b$ |9 L1 VJun  7 14:35:27 controller2 rabbitmq-server: ##  ##2 ?9 y# T  ?1 g2 @* ^
Jun  7 14:35:27 controller2 rabbitmq-server: ##########  Logs: /var/log/rabbitmq/rabbit@controller2.log: @8 n2 v  I; L" x% J( c8 P7 e3 S
Jun  7 14:35:27 controller2 rabbitmq-server: ######  ##        /var/log/rabbitmq/rabbit@controller2-sasl.log
: s" C. }2 C2 U. B- H8 @& q* d4 _Jun  7 14:35:27 controller2 rabbitmq-server: ##########
/ s1 _' z  E6 p" v1 U" zJun  7 14:35:27 controller2 rabbitmq-server: Starting broker...
  x- w# Y. @! p  HJun  7 14:35:27 controller2 rabbitmq-server: BOOT FAILED
: X( Z: r* {! I8 ?8 d- S$ _0 {Jun  7 14:35:27 controller2 rabbitmq-server: ===========
; {! K: D2 @3 ?: u$ n7 y/ vJun  7 14:35:27 controller2 rabbitmq-server: Error description:
- ~2 o- p+ `6 I$ L/ w4 J/ q' kJun  7 14:35:27 controller2 rabbitmq-server: {boot_step,recovery,
( `8 r9 P. v& U/ Y8 c* rJun  7 14:35:27 controller2 rabbitmq-server: {badmatch,; m/ }! z' Q6 L. B
Jun  7 14:35:27 controller2 rabbitmq-server: {error,4 }& X1 J7 X9 x7 {( ~
Jun  7 14:35:27 controller2 rabbitmq-server: {{{badmatch,/ O2 Z# i/ y, r$ t! ]1 B8 o, Y
Jun  7 14:35:27 controller2 rabbitmq-server: {error,$ H5 Q+ ^8 J4 ^+ u) s1 B
Jun  7 14:35:27 controller2 rabbitmq-server: {not_a_dets_file,
8 T" _0 @0 v# q, I/ c3 ]6 s' ]Jun  7 14:35:27 controller2 rabbitmq-server: "/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},; W) {/ G- r- i" ?+ `( w7 Q- u5 L) Y! [
Jun  7 14:35:27 controller2 rabbitmq-server: [{rabbit_recovery_terms,open_table,0,
( B/ v9 @+ S) \Jun  7 14:35:27 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,130}]},
+ I5 g  O8 O4 fJun  7 14:35:27 controller2 rabbitmq-server: {rabbit_recovery_terms,init,1,0 X6 o2 X* C# R0 F% X5 o- [
Jun  7 14:35:27 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,111}]}," {2 q0 Q7 [* N' I- j4 M
Jun  7 14:35:27 controller2 rabbitmq-server: {gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},0 o$ C: w& Y! e  Y6 h
Jun  7 14:35:27 controller2 rabbitmq-server: {proc_lib,init_p_do_apply,3,
( @! _( G/ j6 ~. B' `Jun  7 14:35:27 controller2 rabbitmq-server: [{file,"proc_lib.erl"},{line,237}]}]},
6 c3 j* t- ^5 H( PJun  7 14:35:27 controller2 rabbitmq-server: {child,undefined,rabbit_recovery_terms,
& V) h# V6 G7 r9 L+ NJun  7 14:35:27 controller2 rabbitmq-server: {rabbit_recovery_terms,start_link,[]},
) l* M& |: ^; A- m1 D/ q! yJun  7 14:35:27 controller2 rabbitmq-server: transient,4294967295,worker,
# x+ N3 M( C0 `+ k' g; K, G7 |Jun  7 14:35:27 controller2 rabbitmq-server: [rabbit_recovery_terms]}}}}}
: s" {1 q' d' w  V, q/ ZJun  7 14:35:27 controller2 rabbitmq-server: Log files (may contain more information):
7 w6 S: |' y+ z5 A/ zJun  7 14:35:27 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2.log
' r4 }# J* }4 }Jun  7 14:35:27 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2-sasl.log2 X8 G! u. G- d4 ~% P2 q
Jun  7 14:35:27 controller2 rabbitmq-server: Stack trace:
) p3 P! z' \- k+ u: ~Jun  7 14:35:27 controller2 rabbitmq-server: [{rabbit_queue_index,start,1," f1 h( R7 Y, w0 k( w! e6 ], R( D: \
Jun  7 14:35:27 controller2 rabbitmq-server: [{file,"src/rabbit_queue_index.erl"},{line,359}]},
7 {: n) A! u4 I% W/ ^3 {Jun  7 14:35:27 controller2 rabbitmq-server: {rabbit_variable_queue,start,1,$ ]& t& f- }$ F' S+ F6 b6 ~
Jun  7 14:35:27 controller2 rabbitmq-server: [{file,"src/rabbit_variable_queue.erl"},3 g( `0 e/ H' P
Jun  7 14:35:27 controller2 rabbitmq-server: {line,402}]},
& N) ]) n! \( c! [0 q  FJun  7 14:35:27 controller2 rabbitmq-server: {rabbit_amqqueue,recover,0,[{file,"src/rabbit_amqqueue.erl"},{line,216}]},
2 e% C' {' \" dJun  7 14:35:27 controller2 rabbitmq-server: {rabbit,recover,0,[{file,"src/rabbit.erl"},{line,656}]}," e# R. `1 Z. M/ }$ ^9 H9 d
Jun  7 14:35:27 controller2 rabbitmq-server: {rabbit,'-run_step/3-lc$^1/1-1-',2,[{file,"src/rabbit.erl"},{line,539}]},
7 Z. Q  y8 s' q2 m6 J$ P1 N! rJun  7 14:35:27 controller2 rabbitmq-server: {rabbit,run_step,3,[{file,"src/rabbit.erl"},{line,538}]},
" a. Y/ e/ l) r  CJun  7 14:35:27 controller2 rabbitmq-server: {rabbit,'-run_boot_steps/1-lc$^0/1-0-',1,+ W! }" I3 q5 _# ^& K
Jun  7 14:35:27 controller2 rabbitmq-server: [{file,"src/rabbit.erl"},{line,525}]},
9 I9 t! ?; W1 H$ g# C: gJun  7 14:35:27 controller2 rabbitmq-server: {rabbit,run_boot_steps,1,[{file,"src/rabbit.erl"},{line,525}]}]
; w9 a! t! I  a# x, @0 n- {6 ]2 J# W; P, X* F( p9 U

1 z) H- R) i1 t" \# l% Z* x: W) \& x  u$ K+ x/ [3 O
Jun  7 14:35:28 controller2 rabbitmq-server(ksc-os-rabbitmq)[20504]: ERROR: RabbitMQ server is failed.1 h& K$ @5 b' w% U# F: T) Q* L
Jun  7 14:35:28 controller2 crmd[6475]:  notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: unknown error (node=controller2.ksc, call=548944, rc=1, cib-update=594834, confirmed=false)4 j7 {( E* Q7 ]+ A( P& m6 D- h: ~
Jun  7 14:35:28 controller2 rabbitmq-server: BOOT FAILED8 w# q4 i, J" i; q7 x. W1 n; |
Jun  7 14:35:28 controller2 rabbitmq-server: ===========- O4 D8 D6 ]6 Y: d* q! V
Jun  7 14:35:28 controller2 rabbitmq-server: Error description:% r  {9 W( S& L+ y+ C$ P
Jun  7 14:35:28 controller2 rabbitmq-server: {could_not_start,rabbit,
8 Z( q- {. l, H" DJun  7 14:35:28 controller2 rabbitmq-server: {bad_return,' f$ V" f* ^+ S" B( q6 n( L# I
Jun  7 14:35:28 controller2 rabbitmq-server: {{rabbit,start,[normal,[]]},
  }3 b2 A2 T2 v, d: c5 A; i# w. ?) o$ HJun  7 14:35:28 controller2 rabbitmq-server: {'EXIT',
6 W# g" z8 K7 OJun  7 14:35:28 controller2 rabbitmq-server: {rabbit,failure_during_boot,; h) i2 g: ^- E2 d! L
Jun  7 14:35:28 controller2 rabbitmq-server: {boot_step,recovery,( v' a" q, g1 m- B5 q
Jun  7 14:35:28 controller2 rabbitmq-server: {badmatch,1 H* B% D9 K2 A9 o: Y
Jun  7 14:35:28 controller2 rabbitmq-server: {error,
# z' n: M" A3 I1 IJun  7 14:35:28 controller2 rabbitmq-server: {{{badmatch,9 `  U  y2 Q7 G  A
Jun  7 14:35:28 controller2 rabbitmq-server: {error,7 z) f5 H+ b5 G" I$ ]4 z! x
Jun  7 14:35:28 controller2 rabbitmq-server: {not_a_dets_file,: ]# N+ F/ S% c- p6 m# C
Jun  7 14:35:28 controller2 rabbitmq-server: "/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},2 {2 J3 z1 T+ i# W! a
Jun  7 14:35:28 controller2 rabbitmq-server: [{rabbit_recovery_terms,open_table,0,
, S! ?! F- X& O$ G) HJun  7 14:35:28 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,130}]},
7 t. B+ m4 f8 N3 }, P+ FJun  7 14:35:28 controller2 rabbitmq-server: {rabbit_recovery_terms,init,1,
! `2 A5 c; b5 q( U' Z! l( Z' B  F( v- J  b! n7 l
Jun  7 14:35:28 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,111}]},1 F- Y) x" c, W/ N! B0 b% Y( W
Jun  7 14:35:28 controller2 rabbitmq-server: {gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},8 Y7 `$ U% O# E3 e& t1 ?4 U# S
Jun  7 14:35:28 controller2 rabbitmq-server: {proc_lib,init_p_do_apply,3,2 o1 x: n+ A2 i. \, T
Jun  7 14:35:28 controller2 rabbitmq-server: [{file,"proc_lib.erl"},{line,237}]}]},% d* ~2 J7 O3 R# W5 G
Jun  7 14:35:28 controller2 rabbitmq-server: {child,undefined,rabbit_recovery_terms,
# ?# h$ g3 ~4 U. g5 r& pJun  7 14:35:28 controller2 rabbitmq-server: {rabbit_recovery_terms,start_link,[]},- Y& Z; P" E$ g1 W
Jun  7 14:35:28 controller2 rabbitmq-server: transient,4294967295,worker,
4 D) o0 e* }! g- X$ [Jun  7 14:35:28 controller2 rabbitmq-server: [rabbit_recovery_terms]}}}}}}}}}}
# u+ Y. V% `& _Jun  7 14:35:28 controller2 rabbitmq-server: Log files (may contain more information):
3 i) |$ N2 Q, \) ~Jun  7 14:35:28 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2.log3 C  V  o2 q% ~1 J4 O9 N
Jun  7 14:35:28 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2-sasl.log! P. f! p+ ^& [( t
Jun  7 14:35:28 controller2 rabbitmq-server(ksc-os-rabbitmq)[20526]: ERROR: RabbitMQ server is failed.
4 f% [9 n' _7 w5 ^( g6 HJun  7 14:35:28 controller2 rabbitmq-server(ksc-os-rabbitmq)[20526]: INFO: Resource not running.# m+ _1 w  g9 w
Jun  7 14:35:28 controller2 crmd[6475]:  notice: process_lrm_event: Operation ksc-os-rabbitmq_stop_0: ok (node=controller2.ksc, call=548946, rc=0, cib-update=594835, confirmed=true)
  [+ Y  x. F% DJun  7 14:35:28 controller2 rabbitmq-server(ksc-os-rabbitmq)[20544]: ERROR: RabbitMQ server is failed.) U  A! w, ?& \& [, p7 d( P5 O1 e. V

; ?) b* i2 ^- d$ NJun  7 14:35:29 controller2 systemd: Started RabbitMQ broker.
1 |. @) G% n0 D% Q/ I' }# eJun  7 14:35:29 controller2 rabbitmq-server: {"init terminating in do_boot",{rabbit,failure_during_boot,{could_not_start,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot,{boot_step,recovery,{badmatch,{error,{{{badmatch,{error,{not_a_dets_file,"/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},[{rabbit_recovery_terms,open_table,0,[{file,"src/rabbit_recovery_terms.erl"},{line,130}]},{rabbit_recovery_terms,init,1,[{file,"src/rabbit_recovery_terms.erl"},{line,111}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]},{child,undefined,rabbit_recovery_terms,{rabbit_recovery_terms,start_link,[]},transient,4294967295,worker,[rabbit_recovery_terms]}}}}}}}}}}}}
( u- C, W6 h8 k! I8 k9 ^$ ?( DJun  7 14:35:29 controller2 crmd[6475]:  notice: process_lrm_event: Operation ksc-os-rabbitmq_start_0: ok (node=controller2.ksc, call=548947, rc=0, cib-update=594836, confirmed=true)+ c+ ]* t4 M. I9 W
Jun  7 14:35:29 controller2 crmd[6475]:  notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: ok (node=controller2.ksc, call=548948, rc=0, cib-update=594837, confirmed=false)" A0 I# ?- `1 Q' U
Jun  7 14:35:31 controller2 rabbitmq-server: Crash dump was written to: erl_crash.dump
( W; [& L* E, i" M  d! l5 A( q2 J, aJun  7 14:35:31 controller2 rabbitmq-server: init terminating in do_boot ()2 v& I: s. @1 N) i* h! y
Jun  7 14:35:31 controller2 systemd: rabbitmq-server.service: main process exited, code=exited, status=1/FAILURE
0 Z3 d2 Q# J( o, j( f( b" p( |) mJun  7 14:35:31 controller2 rabbitmqctl: Stopping and halting node 'rabbit@controller2' ...4 q6 P; I" s0 G7 G9 R/ ]) [( l$ @) m- |
Jun  7 14:35:31 controller2 rabbitmqctl: Error: unable to connect to node 'rabbit@controller2': nodedown
' ?" j$ b& Z# s6 e) t. p- XJun  7 14:35:31 controller2 rabbitmqctl: DIAGNOSTICS$ ]" ?3 q: B& Q( @* [+ j
Jun  7 14:35:31 controller2 rabbitmqctl: ===========# ^4 s! Q4 U2 a3 k  t9 n6 Z: G
Jun  7 14:35:31 controller2 rabbitmqctl: attempted to contact: ['rabbit@controller2']
3 G: i$ U2 B! P% |1 g$ t) @Jun  7 14:35:31 controller2 rabbitmqctl: rabbit@controller2:
( E7 t( S6 L5 L9 F* b( XJun  7 14:35:31 controller2 rabbitmqctl: * connected to epmd (port 4369) on controller2  h* {/ G; h/ g& X
Jun  7 14:35:31 controller2 rabbitmqctl: * epmd reports: node 'rabbit' not running at all/ C2 ]# m4 k6 ^" u3 _4 I
Jun  7 14:35:31 controller2 rabbitmqctl: no other nodes on controller29 k# U7 Q; u( m5 `9 s1 I
Jun  7 14:35:31 controller2 rabbitmqctl: * suggestion: start the node' t( }0 f  ]- i& `' U9 z
Jun  7 14:35:31 controller2 rabbitmqctl: current node details:7 t% m) i! W( f% X1 Z6 l3 e
Jun  7 14:35:31 controller2 rabbitmqctl: - node name: 'rabbitmqctl-20636@controller2'
. k6 @& u( G% e/ Q( q! A% Z4 s8 i; }Jun  7 14:35:31 controller2 rabbitmqctl: - home dir: /var/lib/rabbitmq
  J5 H% Y+ m. m- q1 [8 o. t" FJun  7 14:35:31 controller2 rabbitmqctl: - cookie hash: YqnFeQ57nhtexGsA970OWw==
7 L) Z0 w1 X* K- L# F2 X' E+ VJun  7 14:35:31 controller2 systemd: rabbitmq-server.service: control process exited, code=exited status=2, v0 m4 \7 b- e0 J
Jun  7 14:35:31 controller2 systemd: Unit rabbitmq-server.service entered failed state.
! W$ b6 `% B% g# J: C( t9 WJun  7 14:35:31 controller2 systemd: rabbitmq-server.service failed.+ J1 }+ Y9 q* {; B! q6 `4 M
Jun  7 14:35:35 controller2 filebeat: 2024-06-07T14:35:35.162+0800        ERROR        [publisher_pipeline_output]        pipeline/output.go:154        Failed to connect to backoff(elasticsearch(http://172.24.110.127:9200)): Get "http://172.24.110.127:9200": dial tcp 172.24.110.127:9200: connect: connection timed out$ g* }  \2 p0 p
Jun  7 14:35:35 controller2 filebeat: 2024-06-07T14:35:35.162+0800        INFO        [publisher_pipeline_output]        pipeline/output.go:145        Attempting to reconnect to backoff(elasticsearch(http://172.24.110.127:9200)) with 84247 reconnect attempt(s)2 ~8 X6 y+ u! e' w1 s* x0 s0 ~2 h
Jun  7 14:35:35 controller2 filebeat: 2024-06-07T14:35:35.162+0800        INFO        [publisher]        pipeline/retry.go:213        retryer: send wait signal to consumer, H/ j% H$ C4 {5 t- L
Jun  7 14:35:35 controller2 filebeat: 2024-06-07T14:35:35.162+0800        INFO        [publisher]        pipeline/retry.go:217          done
1 }5 B& K9 c- h$ V5 z0 v5 T; QJun  7 14:35:37 controller2 filebeat: 2024-06-07T14:35:37.219+0800        ERROR        [publisher_pipeline_output]        pipeline/output.go:154        Failed to connect to backoff(elasticsearch(http://172.24.110.125:9200)): Get "http://172.24.110.125:9200": dial tcp 172.24.110.125:9200: connect: connection timed out/ i2 e# f  `" {" m/ j: I7 ]. ?
Jun  7 14:35:37 controller2 filebeat: 2024-06-07T14:35:37.219+0800        INFO        [publisher_pipeline_output]        pipeline/output.go:145        Attempting to reconnect to backoff(elasticsearch(http://172.24.110.125:9200)) with 84378 reconnect attempt(s)8 l: @) {, H0 \1 V
Jun  7 14:35:37 controller2 filebeat: 2024-06-07T14:35:37.219+0800        INFO        [publisher]        pipeline/retry.go:213        retryer: send wait signal to consumer( _2 M+ y& H$ j# x. V4 k) u
Jun  7 14:35:37 controller2 filebeat: 2024-06-07T14:35:37.219+0800        INFO        [publisher]        pipeline/retry.go:217          done9 C' e* {: B6 E+ u" w
Jun  7 14:35:39 controller2 startKingCloudOSAgent: 2024-06-07 14:35:39.186 5108 INFO kingcloudos.agent.tasks.agent_tasks [-] Start to report status of agent
+ C3 H8 P& D5 {" g$ wJun  7 14:35:39 controller2 startKingCloudOSAgent: 2024-06-07 14:35:39.211 5108 INFO kingcloudos.agent.tasks.agent_tasks [-] report agent status with {u'binary': u'startKingCloudOSAgent', u'host': u'controller2.ksc', u'uuid': u'9f541bc8-e6db-4816-ae39-0eb60cc58166', u'report_count': 3145039, u'state': u'up'}
) ^. i) t3 E- J. ~( JJun  7 14:35:39 controller2 rabbitmq-server(ksc-os-rabbitmq)[22417]: ERROR: RabbitMQ server is failed.
$ T+ W4 o# t( {4 \5 `4 G: Y' W3 A  WJun  7 14:35:39 controller2 crmd[6475]:  notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: unknown error (node=controller2.ksc, call=548948, rc=1, cib-update=594838, confirmed=false)
" W9 o2 S: k9 N8 {Jun  7 14:35:39 controller2 rabbitmq-server(ksc-os-rabbitmq)[22429]: ERROR: RabbitMQ server is failed.
7 O$ I5 B% O6 G* [5 KJun  7 14:35:39 controller2 rabbitmq-server(ksc-os-rabbitmq)[22429]: INFO: Resource not running.
; T' B4 b  h3 C- |! `- ]Jun  7 14:35:39 controller2 crmd[6475]:  notice: process_lrm_event: Operation ksc-os-rabbitmq_stop_0: ok (node=controller2.ksc, call=548950, rc=0, cib-update=594839, confirmed=true)
, y9 e/ Y0 Q9 G( N# D# JJun  7 14:35:39 controller2 rabbitmq-server(ksc-os-rabbitmq)[22448]: ERROR: RabbitMQ server is failed.
* |" j% f! o" K) S+ \Jun  7 14:35:39 controller2 systemd: Starting RabbitMQ broker...- Z. y9 `# \% Y# S6 t
Jun  7 14:35:40 controller2 rabbitmqctl: Waiting for 'rabbit@controller2' ...+ e8 P% {  \% [. n2 d+ v" b
Jun  7 14:35:40 controller2 rabbitmqctl: pid is 22597 ...
- d" K1 W6 B, J6 n. ^% GJun  7 14:35:41 controller2 rabbitmq-server: RabbitMQ 3.4.0. Copyright (C) 2007-2014 GoPivotal, Inc.
2 ]" i8 r4 Q( }  g9 _Jun  7 14:35:41 controller2 rabbitmq-server: ##  ##      Licensed under the MPL.  See http://www.rabbitmq.com/( f% ]' z; ^/ r5 s3 b% D( B- |
Jun  7 14:35:41 controller2 rabbitmq-server: ##  ##  F; }4 p2 R8 E" F
Jun  7 14:35:41 controller2 rabbitmq-server: ##########  Logs: /var/log/rabbitmq/rabbit@controller2.log
4 g1 m2 Z6 X. Z' }" Y) H2 rJun  7 14:35:41 controller2 rabbitmq-server: ######  ##        /var/log/rabbitmq/rabbit@controller2-sasl.log
, n3 f0 e5 d  |7 O3 k/ `Jun  7 14:35:41 controller2 rabbitmq-server: ##########& {: a* Q) o! B3 |* s, j7 D
Jun  7 14:35:41 controller2 rabbitmq-server: Starting broker...; K: y; G+ A. @4 U1 p! G
Jun  7 14:35:41 controller2 rabbitmq-server: BOOT FAILED
. ]+ s! q* @; e# KJun  7 14:35:41 controller2 rabbitmq-server: ===========
1 I  x7 E; S( N, z0 ^) JJun  7 14:35:41 controller2 rabbitmq-server: Error description:+ w. G$ d/ b0 p; E& ^, o: X
Jun  7 14:35:41 controller2 rabbitmq-server: {boot_step,recovery,7 n+ o. J6 @+ k3 y# x8 R
Jun  7 14:35:41 controller2 rabbitmq-server: {badmatch,1 A2 q0 A! X3 A5 J" _, M/ w
Jun  7 14:35:41 controller2 rabbitmq-server: {error,
: h" t: \% g' P, cJun  7 14:35:41 controller2 rabbitmq-server: {{{badmatch,
# l/ @7 F! g2 H& d) LJun  7 14:35:41 controller2 rabbitmq-server: {error,( q- M# H3 c. U7 m
Jun  7 14:35:41 controller2 rabbitmq-server: {not_a_dets_file,* D7 j5 ^. h: ~
Jun  7 14:35:41 controller2 rabbitmq-server: "/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},
3 B" i0 a7 S7 XJun  7 14:35:41 controller2 rabbitmq-server: [{rabbit_recovery_terms,open_table,0,% Z2 k/ [& R2 `) V% @0 j
Jun  7 14:35:41 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,130}]},1 a( o+ Y, \& d* D/ P
Jun  7 14:35:41 controller2 rabbitmq-server: {rabbit_recovery_terms,init,1,
( K% T1 G: z, y$ p/ l8 ~! ^# RJun  7 14:35:41 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,111}]},# l7 j" S* q* P9 Q9 A' b
Jun  7 14:35:41 controller2 rabbitmq-server: {gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},
; u7 N  L! K5 _8 d9 A; _; g0 PJun  7 14:35:41 controller2 rabbitmq-server: {proc_lib,init_p_do_apply,3,
) F! b3 {/ b% ^2 U, z( VJun  7 14:35:41 controller2 rabbitmq-server: [{file,"proc_lib.erl"},{line,237}]}]},
; t5 W7 n! F( hJun  7 14:35:41 controller2 rabbitmq-server: {child,undefined,rabbit_recovery_terms,( H3 E4 B' H/ S1 \
Jun  7 14:35:41 controller2 rabbitmq-server: {rabbit_recovery_terms,start_link,[]},5 k& J; F: q; R( Q: _$ E1 ?
Jun  7 14:35:41 controller2 rabbitmq-server: transient,4294967295,worker,4 |0 O, z& M+ X( G% ?, ~
Jun  7 14:35:41 controller2 rabbitmq-server: [rabbit_recovery_terms]}}}}}
5 R4 q- y& r% C/ |* Q3 p  uJun  7 14:35:41 controller2 rabbitmq-server: Log files (may contain more information):
. C: L$ J/ t6 jJun  7 14:35:41 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2.log: n4 M4 }' _1 Q" w, i! ^4 G
Jun  7 14:35:41 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2-sasl.log$ {1 o: h+ `* D+ _$ e6 A6 S! J' i
Jun  7 14:35:41 controller2 rabbitmq-server: Stack trace:
0 \* {% [4 W3 w9 O8 _  F. W" G/ DJun  7 14:35:41 controller2 rabbitmq-server: [{rabbit_queue_index,start,1,
4 p! _7 x/ r9 @4 z. wJun  7 14:35:41 controller2 rabbitmq-server: [{file,"src/rabbit_queue_index.erl"},{line,359}]},
, U" Y' s5 c5 i5 K$ PJun  7 14:35:41 controller2 rabbitmq-server: {rabbit_variable_queue,start,1," {. p7 P9 x2 g6 g( f, G* T6 i. e
Jun  7 14:35:41 controller2 rabbitmq-server: [{file,"src/rabbit_variable_queue.erl"}," ~/ _% s7 p8 F' {) I
Jun  7 14:35:41 controller2 rabbitmq-server: {line,402}]},% J& P' B3 v7 _8 h& b9 U  Y6 E
Jun  7 14:35:41 controller2 rabbitmq-server: {rabbit_amqqueue,recover,0,[{file,"src/rabbit_amqqueue.erl"},{line,216}]},
+ z4 J, z( C1 |Jun  7 14:35:41 controller2 rabbitmq-server: {rabbit,recover,0,[{file,"src/rabbit.erl"},{line,656}]},. m3 F) M6 v& z6 ^6 f
Jun  7 14:35:41 controller2 rabbitmq-server: {rabbit,'-run_step/3-lc$^1/1-1-',2,[{file,"src/rabbit.erl"},{line,539}]},+ N1 N- J( D1 X
Jun  7 14:35:41 controller2 rabbitmq-server: {rabbit,run_step,3,[{file,"src/rabbit.erl"},{line,538}]},, X1 A" Z! P. ?! f  k4 F
Jun  7 14:35:41 controller2 rabbitmq-server: {rabbit,'-run_boot_steps/1-lc$^0/1-0-',1,
+ H( J! Q8 h: E5 L' \Jun  7 14:35:41 controller2 rabbitmq-server: [{file,"src/rabbit.erl"},{line,525}]},! p' _9 ]4 K$ E$ F* l
Jun  7 14:35:41 controller2 rabbitmq-server: {rabbit,run_boot_steps,1,[{file,"src/rabbit.erl"},{line,525}]}]
" ^2 c. N& F5 S$ L4 K8 X) H7 PJun  7 14:35:42 controller2 rabbitmq-server: BOOT FAILED3 P6 G& C; M: Z; V; p+ `- o% J
Jun  7 14:35:42 controller2 rabbitmq-server: ===========
7 Y2 n7 l+ z0 C$ FJun  7 14:35:42 controller2 rabbitmq-server: Error description:
% P+ T& D! j  M. eJun  7 14:35:42 controller2 rabbitmq-server: {could_not_start,rabbit,$ G9 @& T# j( m, c
Jun  7 14:35:42 controller2 rabbitmq-server: {bad_return,
8 J5 `, E( b; uJun  7 14:35:42 controller2 rabbitmq-server: {{rabbit,start,[normal,[]]},
' ~, n1 C  u8 {; `+ \; ]% MJun  7 14:35:42 controller2 rabbitmq-server: {'EXIT',8 }+ x  y7 H4 J/ {" G% f2 V- @
Jun  7 14:35:42 controller2 rabbitmq-server: {rabbit,failure_during_boot,
8 G2 u9 U. t5 ^1 W' ]Jun  7 14:35:42 controller2 rabbitmq-server: {boot_step,recovery,
9 ?0 `5 p8 h* \% R" q$ p6 ?Jun  7 14:35:42 controller2 rabbitmq-server: {badmatch,+ l! l+ S$ O( R5 [6 M3 R0 p
Jun  7 14:35:42 controller2 rabbitmq-server: {error,5 s* Q' l0 o! }% K3 W8 E% w" [
Jun  7 14:35:42 controller2 rabbitmq-server: {{{badmatch,0 R5 g9 }  k. Y5 X# J$ Y' g
Jun  7 14:35:42 controller2 rabbitmq-server: {error,
$ ?% c3 ~9 B. ^  u0 {4 m1 GJun  7 14:35:42 controller2 rabbitmq-server: {not_a_dets_file,
: _$ v8 o$ N6 x/ Z6 C/ JJun  7 14:35:42 controller2 rabbitmq-server: "/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},4 C- j/ x+ ?% l. W
Jun  7 14:35:42 controller2 rabbitmq-server: [{rabbit_recovery_terms,open_table,0,
: f! x( ~4 K  h* k+ P; V( O" q" oJun  7 14:35:42 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,130}]},( m# j' |: t8 r+ C0 h# V: T: ]# B$ F
Jun  7 14:35:42 controller2 rabbitmq-server: {rabbit_recovery_terms,init,1,
* Z$ [: v  \  z& q6 d9 k! P2 P: \Jun  7 14:35:42 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,111}]},
8 g8 k  W5 M) \. H: d" p2 [Jun  7 14:35:42 controller2 rabbitmq-server: {gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},
$ k$ v# ^( ~2 m- ?Jun  7 14:35:42 controller2 rabbitmq-server: {proc_lib,init_p_do_apply,3,2 D. e4 ?0 i3 E' Y. w! p1 c8 `
Jun  7 14:35:42 controller2 rabbitmq-server: [{file,"proc_lib.erl"},{line,237}]}]},* n+ N+ Y% a" e8 K; y
Jun  7 14:35:42 controller2 rabbitmq-server: {child,undefined,rabbit_recovery_terms,
& F2 q( w5 W& ]* I9 g; @Jun  7 14:35:42 controller2 rabbitmq-server: {rabbit_recovery_terms,start_link,[]},
: F  o; r- I. R' E" xJun  7 14:35:42 controller2 rabbitmq-server: transient,4294967295,worker,3 z2 x" O& a6 n
Jun  7 14:35:42 controller2 rabbitmq-server: [rabbit_recovery_terms]}}}}}}}}}}: d% o( I* B# u2 _" f1 b) P1 |( @
Jun  7 14:35:42 controller2 rabbitmq-server: Log files (may contain more information):
3 \, ^+ E3 K0 g+ kJun  7 14:35:42 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2.log' Q9 p/ r  v, m5 G$ h4 k1 n+ w6 [
Jun  7 14:35:42 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2-sasl.log
: C7 \( S+ K$ A- h, U; UJun  7 14:35:43 controller2 systemd: Started RabbitMQ broker.
; Y& E0 k) Z  B( yJun  7 14:35:43 controller2 crmd[6475]:  notice: process_lrm_event: Operation ksc-os-rabbitmq_start_0: ok (node=controller2.ksc, call=548951, rc=0, cib-update=594840, confirmed=true)4 `1 s3 Z" h2 S" V7 f, s( o% D
Jun  7 14:35:43 controller2 rabbitmq-server: {"init terminating in do_boot",{rabbit,failure_during_boot,{could_not_start,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot,{boot_step,recovery,{badmatch,{error,{{{badmatch,{error,{not_a_dets_file,"/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},[{rabbit_recovery_terms,open_table,0,[{file,"src/rabbit_recovery_terms.erl"},{line,130}]},{rabbit_recovery_terms,init,1,[{file,"src/rabbit_recovery_terms.erl"},{line,111}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]},{child,undefined,rabbit_recovery_terms,{rabbit_recovery_terms,start_link,[]},transient,4294967295,worker,[rabbit_recovery_terms]}}}}}}}}}}}}
; y4 a5 Y0 F- g/ u' |: XJun  7 14:35:43 controller2 crmd[6475]:  notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: ok (node=controller2.ksc, call=548952, rc=0, cib-update=594841, confirmed=false), o/ B# ~1 u# Y, C
Jun  7 14:35:45 controller2 rabbitmq-server: Crash dump was written to: erl_crash.dump/ c& }7 O+ h- r7 k% h- }6 q1 ^
Jun  7 14:35:45 controller2 rabbitmq-server: init terminating in do_boot ()# j+ C9 b+ F7 n
Jun  7 14:35:45 controller2 systemd: rabbitmq-server.service: main process exited, code=exited, status=1/FAILURE
  \: i; ^) h9 K0 r# M  \- gJun  7 14:35:45 controller2 rabbitmqctl: Stopping and halting node 'rabbit@controller2' ...! j6 S8 x5 \) c$ T) ]8 r# X
Jun  7 14:35:45 controller2 rabbitmqctl: Error: unable to connect to node 'rabbit@controller2': nodedown! q" s: Z) u( E8 b0 ^0 }
Jun  7 14:35:45 controller2 rabbitmqctl: DIAGNOSTICS
& T0 F- D, e" e7 Z# F3 ZJun  7 14:35:45 controller2 rabbitmqctl: ===========
; n  e" C* Y6 E( @7 _! P( NJun  7 14:35:45 controller2 rabbitmqctl: attempted to contact: ['rabbit@controller2']
+ F1 C6 V) M  s8 g' z6 Y0 iJun  7 14:35:45 controller2 rabbitmqctl: rabbit@controller2:
6 [8 x: O) U. D8 q1 }Jun  7 14:35:45 controller2 rabbitmqctl: * connected to epmd (port 4369) on controller26 Z. r4 `- H7 L. w
Jun  7 14:35:45 controller2 rabbitmqctl: * epmd reports: node 'rabbit' not running at all
. z1 ?  q- x6 N- }Jun  7 14:35:45 controller2 rabbitmqctl: no other nodes on controller2
4 _8 a( M0 A- Q; Z" Y0 ^+ R/ d( b6 P: sJun  7 14:35:45 controller2 rabbitmqctl: * suggestion: start the node
% Y( d1 O9 \+ q5 w5 Z! J& A, qJun  7 14:35:45 controller2 rabbitmqctl: current node details:; N! z0 ]/ U+ W. c/ ^$ {9 i2 B1 I
Jun  7 14:35:45 controller2 rabbitmqctl: - node name: 'rabbitmqctl-22956@controller2'9 F" a1 b% M! q3 }7 h9 H5 {4 h
Jun  7 14:35:45 controller2 rabbitmqctl: - home dir: /var/lib/rabbitmq
$ I, n8 K0 I  v6 E1 i4 m: Z) iJun  7 14:35:45 controller2 rabbitmqctl: - cookie hash: YqnFeQ57nhtexGsA970OWw==) p. Z$ n2 x$ \- m
Jun  7 14:35:45 controller2 systemd: rabbitmq-server.service: control process exited, code=exited status=2
6 c! u- _; G' S$ @1 Y8 ]9 WJun  7 14:35:45 controller2 systemd: Unit rabbitmq-server.service entered failed state.& I5 O) l9 w3 [: L  q' @0 ^
Jun  7 14:35:45 controller2 systemd: rabbitmq-server.service failed.
' B  O4 e) E$ b0 ZJun  7 14:35:45 controller2 ocf-mysql-wss: INFO: GTID OK: 62a7fc52-52de-11e8-a423-ee3e8b5b2a73:20444521855 A1 q3 X0 X) W3 S/ H% p, \
Jun  7 14:35:45 controller2 ocf-mysql-wss: INFO: Galera GTID: 62a7fc52-52de-11e8-a423-ee3e8b5b2a73:2044452185% {  Q# g& T  C( v! V
Jun  7 14:35:49 controller2 filebeat: 2024-06-07T14:35:49.300+0800        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":890100,"time":{"ms":6}},"total":{"ticks":2986480,"time":{"ms":48},"value":2986480},"user":{"ticks":2096380,"time":{"ms":42}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":28},"info":{"ephemeral_id":"2ea13f88-dfe4-4d82-bb66-f2726d6d1965","uptime":{"ms":4384770183}},"memstats":{"gc_next":33872784,"memory_alloc":17183880,"memory_total":69311206544},"runtime":{"goroutines":133}},"filebeat":{"harvester":{"open_files":17,"running":10}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":13,"events":{"active":4126,"retry":100}}},"registrar":{"states":{"current":13}},"system":{"load":{"1":2.84,"15":2.57,"5":2.66,"norm":{"1":0.071,"15":0.0643,"5":0.0665}}}}}}
& a6 d0 L; q/ n; {. {; B) a- fJun  7 14:35:53 controller2 dhcpd: isc-dhcpd-4.2.5+ s0 C4 w! I2 a$ T/ j( ?2 f7 K
Jun  7 14:35:53 controller2 rabbitmq-server(ksc-os-rabbitmq)[23948]: ERROR: RabbitMQ server is failed.4 ~8 g0 G( O8 e/ |0 J
Jun  7 14:35:53 controller2 crmd[6475]:  notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: unknown error (node=controller2.ksc, call=548952, rc=1, cib-update=594842, confirmed=false)
; ]' \, R) w8 X% d3 F. r2 z7 b! y, TJun  7 14:35:54 controller2 rabbitmq-server(ksc-os-rabbitmq)[23963]: ERROR: RabbitMQ server is failed.9 n9 `. u, n. z9 s& r+ K2 C+ N
Jun  7 14:35:54 controller2 rabbitmq-server(ksc-os-rabbitmq)[23963]: INFO: Resource not running.; m  X2 ^) ~8 }& {+ l
Jun  7 14:35:54 controller2 crmd[6475]:  notice: process_lrm_event: Operation ksc-os-rabbitmq_stop_0: ok (node=controller2.ksc, call=548954, rc=0, cib-update=594843, confirmed=true)# `* S; j0 U% g
Jun  7 14:35:57 controller2 filebeat: 2024-06-07T14:35:57.029+0800        ERROR        [publisher_pipeline_output]        pipeline/output.go:154        Failed to connect to backoff(elasticsearch(http://172.24.110.126:9200)): Get "http://172.24.110.126:9200": dial tcp 172.24.110.126:9200: connect: connection timed out% |7 p, \; Z. Z& e: S- K
Jun  7 14:35:57 controller2 filebeat: 2024-06-07T14:35:57.030+0800        INFO        [publisher_pipeline_output]        pipeline/output.go:145        Attempting to reconnect to backoff(elasticsearch(http://172.24.110.126:9200)) with 84327 reconnect attempt(s)% ?# m3 h% k7 ^5 J* @, B
Jun  7 14:35:57 controller2 filebeat: 2024-06-07T14:35:57.030+0800        INFO        [publisher]        pipeline/retry.go:213        retryer: send wait signal to consumer1 Q1 k* S4 l( U  L
Jun  7 14:35:57 controller2 filebeat: 2024-06-07T14:35:57.030+0800        INFO        [publisher]        pipeline/retry.go:217          done
9 R" R7 T' E3 f: l6 u$ S6 y$ p& |, ~0 ~7 |- r1 I1 J/ B
# g6 _  \2 Y# r# I: u: ~
Jun  7 14:36:06 controller2 ocf-mysql-wss: INFO: GTID OK: 62a7fc52-52de-11e8-a423-ee3e8b5b2a73:2044452424
7 M: C% }3 n8 f9 hJun  7 14:36:06 controller2 ocf-mysql-wss: INFO: Galera GTID: 62a7fc52-52de-11e8-a423-ee3e8b5b2a73:2044452424
' P. G: O, `! U3 O9 W& X- ~8 E; ~! u, n# V
0 b  I- G, e8 V
您需要登录后才可以回帖 登录 | 开始注册

本版积分规则

关闭

站长推荐上一条 /4 下一条

北京云银创陇科技有限公司以云计算运维,代码开发

QQ|返回首页|Archiver|小黑屋|易陆发现技术论坛 ( 蜀ICP备2026014127号-1 )点击这里给我发消息

GMT+8, 2026-4-8 21:29 , Processed in 0.058350 second(s), 22 queries .

Powered by Discuz! X3.4 Licensed

© 2012-2025 Discuz! Team.

快速回复 返回顶部 返回列表