|
|
Jun 7 14:35:17 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2.log
3 d, P( h! d4 {' z4 V- LJun 7 14:35:17 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2-sasl.log# p% I+ S" p) L- ?4 M1 @
Jun 7 14:35:18 controller2 systemd: Started RabbitMQ broker.# U* f( T) q" f2 |
Jun 7 14:35:18 controller2 crmd[6475]: notice: process_lrm_event: Operation ksc-os-rabbitmq_start_0: ok (node=controller2.ksc, call=548943, rc=0, cib-update=594832, confirmed=true)% f/ i* J3 H5 S
Jun 7 14:35:18 controller2 rabbitmq-server: {"init terminating in do_boot",{rabbit,failure_during_boot,{could_not_start,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot,{boot_step,recovery,{badmatch,{error,{{{badmatch,{error,{not_a_dets_file,"/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},[{rabbit_recovery_terms,open_table,0,[{file,"src/rabbit_recovery_terms.erl"},{line,130}]},{rabbit_recovery_terms,init,1,[{file,"src/rabbit_recovery_terms.erl"},{line,111}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]},{child,undefined,rabbit_recovery_terms,{rabbit_recovery_terms,start_link,[]},transient,4294967295,worker,[rabbit_recovery_terms]}}}}}}}}}}}}
0 Z: q. [: R, g& M" Q0 fJun 7 14:35:18 controller2 crmd[6475]: notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: ok (node=controller2.ksc, call=548944, rc=0, cib-update=594833, confirmed=false)
) t, S! [ x$ q e% K; e! OJun 7 14:35:19 controller2 filebeat: 2024-06-07T14:35:19.300+0800 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":890090,"time":{"ms":4}},"total":{"ticks":2986430,"time":{"ms":11},"value":2986430},"user":{"ticks":2096340,"time":{"ms":7}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":28},"info":{"ephemeral_id":"2ea13f88-dfe4-4d82-bb66-f2726d6d1965","uptime":{"ms":4384740183}},"memstats":{"gc_next":33883984,"memory_alloc":18558592,"memory_total":69310713440},"runtime":{"goroutines":133}},"filebeat":{"harvester":{"open_files":17,"running":10}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":13,"events":{"active":4126,"retry":100}}},"registrar":{"states":{"current":13}},"system":{"load":{"1":2.47,"15":2.54,"5":2.58,"norm":{"1":0.0618,"15":0.0635,"5":0.0645}}}}}}
; b$ v0 s- s( w! _Jun 7 14:35:19 controller2 rabbitmq-server: Crash dump was written to: erl_crash.dump
4 L, ]$ T+ f X: f5 TJun 7 14:35:19 controller2 rabbitmq-server: init terminating in do_boot (), ]/ h6 e/ T# z
Jun 7 14:35:19 controller2 systemd: rabbitmq-server.service: main process exited, code=exited, status=1/FAILURE
. P: a- r) X1 }. ~: j, f; LJun 7 14:35:20 controller2 rabbitmqctl: Stopping and halting node 'rabbit@controller2' ...: O" d/ ?' p! f5 v
Jun 7 14:35:20 controller2 rabbitmqctl: Error: unable to connect to node 'rabbit@controller2': nodedown
6 ?" d, t% [6 T7 z; c+ }+ A" yJun 7 14:35:20 controller2 rabbitmqctl: DIAGNOSTICS [, B7 ~* s+ f+ Q0 w
Jun 7 14:35:20 controller2 rabbitmqctl: ===========: g$ E" i) g# z! z& V
Jun 7 14:35:20 controller2 rabbitmqctl: attempted to contact: ['rabbit@controller2']# j! x+ U5 y# O$ A: H+ A4 [4 ]5 Y
Jun 7 14:35:20 controller2 rabbitmqctl: rabbit@controller2:, R x% C) q% a6 B
Jun 7 14:35:20 controller2 rabbitmqctl: * connected to epmd (port 4369) on controller2' Z; M# g5 f l7 r4 g6 p; O- E7 e
Jun 7 14:35:20 controller2 rabbitmqctl: * epmd reports: node 'rabbit' not running at all6 w {' V3 V6 n9 N& P' P+ |* Z5 s
Jun 7 14:35:20 controller2 rabbitmqctl: no other nodes on controller2
. V6 H% F) [% WJun 7 14:35:20 controller2 rabbitmqctl: * suggestion: start the node
- v% `/ F* ~9 m% U" ^Jun 7 14:35:20 controller2 rabbitmqctl: current node details:
; _7 y2 u% O3 b* x, }Jun 7 14:35:20 controller2 rabbitmqctl: - node name: 'rabbitmqctl-19643@controller2'! K% I9 e( e7 X
Jun 7 14:35:20 controller2 rabbitmqctl: - home dir: /var/lib/rabbitmq
" N4 N# \, t5 _& ?$ iJun 7 14:35:20 controller2 rabbitmqctl: - cookie hash: YqnFeQ57nhtexGsA970OWw==
" Q7 W: U) O4 c- E( kJun 7 14:35:20 controller2 systemd: rabbitmq-server.service: control process exited, code=exited status=2+ p) M. o! B2 c" K& u7 a% r
Jun 7 14:35:20 controller2 systemd: Unit rabbitmq-server.service entered failed state.
1 u4 u7 O8 i" q: ]" @/ `# [' PJun 7 14:35:20 controller2 systemd: rabbitmq-server.service failed. n* }. G! Q' h' f! {4 h% L
2 O& ?( _. b8 M# m1 n: x' ?, h$ V2 b; m. S* \( ?8 S( w
- a( t0 x* `) P
Jun 7 14:35:23 controller2 dhcpd: isc-dhcpd-4.2.5
6 V3 D. U( E4 c/ @2 YJun 7 14:35:25 controller2 systemd: Starting RabbitMQ broker...
( P: x: T! B1 o8 DJun 7 14:35:25 controller2 ocf-mysql-wss: INFO: GTID OK: 62a7fc52-52de-11e8-a423-ee3e8b5b2a73:2044451984
. w8 g w2 q" d2 a0 UJun 7 14:35:25 controller2 ocf-mysql-wss: INFO: Galera GTID: 62a7fc52-52de-11e8-a423-ee3e8b5b2a73:2044451984
1 y4 ~' k8 Z7 m% zJun 7 14:35:26 controller2 rabbitmqctl: Waiting for 'rabbit@controller2' ...' m6 \5 K$ L1 Y9 w
Jun 7 14:35:26 controller2 rabbitmqctl: pid is 20016 ...6 X* ^ r& b, J; E, ~" `7 m8 b
Jun 7 14:35:27 controller2 rabbitmq-server: RabbitMQ 3.4.0. Copyright (C) 2007-2014 GoPivotal, Inc.
# o* T" U! A$ J D& VJun 7 14:35:27 controller2 rabbitmq-server: ## ## Licensed under the MPL. See http://www.rabbitmq.com/+ E& U B0 `: F! C
Jun 7 14:35:27 controller2 rabbitmq-server: ## ##, k! S' w8 f& I: @2 ~
Jun 7 14:35:27 controller2 rabbitmq-server: ########## Logs: /var/log/rabbitmq/rabbit@controller2.log
$ ]+ Y5 M& ^0 v' _$ ?# _Jun 7 14:35:27 controller2 rabbitmq-server: ###### ## /var/log/rabbitmq/rabbit@controller2-sasl.log( V7 W1 t* e1 V, j7 e2 E
Jun 7 14:35:27 controller2 rabbitmq-server: ##########3 i8 t# B- D1 d# p
Jun 7 14:35:27 controller2 rabbitmq-server: Starting broker...1 N$ T7 J. ^5 j3 s
Jun 7 14:35:27 controller2 rabbitmq-server: BOOT FAILED
, p" j$ p7 n# {9 GJun 7 14:35:27 controller2 rabbitmq-server: ===========
4 Q8 f; Z$ _, F% ~5 T ~7 ^Jun 7 14:35:27 controller2 rabbitmq-server: Error description:! @# Z t: a0 F! O3 ~
Jun 7 14:35:27 controller2 rabbitmq-server: {boot_step,recovery,
- q# f4 S) t8 }6 wJun 7 14:35:27 controller2 rabbitmq-server: {badmatch,. G5 v7 g6 I; _0 C0 w" |' ]5 S, j9 n
Jun 7 14:35:27 controller2 rabbitmq-server: {error,
* V( j/ b( _7 P. x8 sJun 7 14:35:27 controller2 rabbitmq-server: {{{badmatch,% A& O( P: L& ~$ H2 M( E' x3 j
Jun 7 14:35:27 controller2 rabbitmq-server: {error,
- Q* A$ S9 f$ _9 _Jun 7 14:35:27 controller2 rabbitmq-server: {not_a_dets_file,
* e& Z; x' Z; L. @( k" L5 A bJun 7 14:35:27 controller2 rabbitmq-server: "/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},
# b" Z6 O: D# p6 ~) z8 R) NJun 7 14:35:27 controller2 rabbitmq-server: [{rabbit_recovery_terms,open_table,0,
7 a5 k/ D$ c! Z# i5 W3 y! }: ?Jun 7 14:35:27 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,130}]},* ~, r; ~! _" W1 R3 E
Jun 7 14:35:27 controller2 rabbitmq-server: {rabbit_recovery_terms,init,1,/ _8 c- n: [; c5 {
Jun 7 14:35:27 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,111}]},
/ D$ R- i G; n- [# q, b% {8 DJun 7 14:35:27 controller2 rabbitmq-server: {gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},( V& A0 `3 v7 g$ r' d3 M i* e" ^
Jun 7 14:35:27 controller2 rabbitmq-server: {proc_lib,init_p_do_apply,3,
3 L% L% d2 Z2 M: }Jun 7 14:35:27 controller2 rabbitmq-server: [{file,"proc_lib.erl"},{line,237}]}]},2 E5 U% `# o$ a
Jun 7 14:35:27 controller2 rabbitmq-server: {child,undefined,rabbit_recovery_terms,0 [. b; f* U" P* k. s* l& c' c
Jun 7 14:35:27 controller2 rabbitmq-server: {rabbit_recovery_terms,start_link,[]},8 G+ u' g* P2 @; x" K; E
Jun 7 14:35:27 controller2 rabbitmq-server: transient,4294967295,worker,
' _, W8 p/ h6 |Jun 7 14:35:27 controller2 rabbitmq-server: [rabbit_recovery_terms]}}}}}1 x5 E' V5 k! k2 n
Jun 7 14:35:27 controller2 rabbitmq-server: Log files (may contain more information):
- z9 S# h3 u; kJun 7 14:35:27 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2.log0 _" F$ b1 b8 x, g
Jun 7 14:35:27 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2-sasl.log1 Y4 F- M0 n2 J
Jun 7 14:35:27 controller2 rabbitmq-server: Stack trace:2 x8 V0 v% T* v& _$ y0 w
Jun 7 14:35:27 controller2 rabbitmq-server: [{rabbit_queue_index,start,1,
8 e7 s F. B; c# x! PJun 7 14:35:27 controller2 rabbitmq-server: [{file,"src/rabbit_queue_index.erl"},{line,359}]},
) s2 q/ P( @& M& L7 z( qJun 7 14:35:27 controller2 rabbitmq-server: {rabbit_variable_queue,start,1,8 U- v' j" x4 Q2 n# ^/ a1 q% Q* ^
Jun 7 14:35:27 controller2 rabbitmq-server: [{file,"src/rabbit_variable_queue.erl"},
! a+ l# t. h, f4 t- K* C8 l) M! ?9 TJun 7 14:35:27 controller2 rabbitmq-server: {line,402}]},0 r1 R% ^9 U- ]! [' a
Jun 7 14:35:27 controller2 rabbitmq-server: {rabbit_amqqueue,recover,0,[{file,"src/rabbit_amqqueue.erl"},{line,216}]},
: L C" L# m5 W- j# ~Jun 7 14:35:27 controller2 rabbitmq-server: {rabbit,recover,0,[{file,"src/rabbit.erl"},{line,656}]},
5 U! [, H( w, S% F9 b3 YJun 7 14:35:27 controller2 rabbitmq-server: {rabbit,'-run_step/3-lc$^1/1-1-',2,[{file,"src/rabbit.erl"},{line,539}]},
8 C7 i2 M% ?3 b. N0 W1 kJun 7 14:35:27 controller2 rabbitmq-server: {rabbit,run_step,3,[{file,"src/rabbit.erl"},{line,538}]},7 u' Y4 X: ~0 w- Q6 Q3 [' g
Jun 7 14:35:27 controller2 rabbitmq-server: {rabbit,'-run_boot_steps/1-lc$^0/1-0-',1,
$ D x) l5 a3 ?: W/ L7 CJun 7 14:35:27 controller2 rabbitmq-server: [{file,"src/rabbit.erl"},{line,525}]}," I+ T5 D! h4 g% T/ P
Jun 7 14:35:27 controller2 rabbitmq-server: {rabbit,run_boot_steps,1,[{file,"src/rabbit.erl"},{line,525}]}]3 I2 b( k# K' v& p+ k* B' _( c% G
# Y R3 J1 q- e8 h5 a& U' l7 v( S
2 P9 P9 t9 O& n/ n
1 N; Q$ ^: D! F- p1 R! G0 rJun 7 14:35:28 controller2 rabbitmq-server(ksc-os-rabbitmq)[20504]: ERROR: RabbitMQ server is failed.. e5 |1 q) g* ~; U
Jun 7 14:35:28 controller2 crmd[6475]: notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: unknown error (node=controller2.ksc, call=548944, rc=1, cib-update=594834, confirmed=false)
# a5 c9 c, t9 f3 Q9 b X- K( DJun 7 14:35:28 controller2 rabbitmq-server: BOOT FAILED
0 b& i2 a9 P# J; H4 r) }5 C$ O( Q# VJun 7 14:35:28 controller2 rabbitmq-server: ===========" r" t! j4 r" K9 z
Jun 7 14:35:28 controller2 rabbitmq-server: Error description:
9 A% b/ a5 r4 d5 `6 V. WJun 7 14:35:28 controller2 rabbitmq-server: {could_not_start,rabbit,
+ g9 e. m& C. o: LJun 7 14:35:28 controller2 rabbitmq-server: {bad_return,2 Q' h" ]. K4 x1 a
Jun 7 14:35:28 controller2 rabbitmq-server: {{rabbit,start,[normal,[]]},4 I5 T2 g Z6 L- c1 p8 V, l
Jun 7 14:35:28 controller2 rabbitmq-server: {'EXIT',& z/ B3 c1 L- u _+ M& }8 c
Jun 7 14:35:28 controller2 rabbitmq-server: {rabbit,failure_during_boot,% X4 {7 g& u! V2 z# Y
Jun 7 14:35:28 controller2 rabbitmq-server: {boot_step,recovery,
9 P4 I4 r" Y$ k' PJun 7 14:35:28 controller2 rabbitmq-server: {badmatch,- E7 q$ ]3 c# |% g9 U. E& G
Jun 7 14:35:28 controller2 rabbitmq-server: {error,9 S$ A7 J0 R4 b/ `7 S; f- s0 C# \
Jun 7 14:35:28 controller2 rabbitmq-server: {{{badmatch,+ ^6 M% _& @2 I7 T, ~2 U( B( V
Jun 7 14:35:28 controller2 rabbitmq-server: {error,
1 M/ t/ S# F$ L$ gJun 7 14:35:28 controller2 rabbitmq-server: {not_a_dets_file,
( o0 W$ P5 M1 X$ ~: r; ^Jun 7 14:35:28 controller2 rabbitmq-server: "/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},
- _- q' e. Z1 b1 l# \7 @Jun 7 14:35:28 controller2 rabbitmq-server: [{rabbit_recovery_terms,open_table,0,! f1 M! h3 S- C/ V3 @
Jun 7 14:35:28 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,130}]},: F6 T3 x' N8 V( l, M
Jun 7 14:35:28 controller2 rabbitmq-server: {rabbit_recovery_terms,init,1,
( J/ L6 M% w/ L1 |6 a) B( K$ V* p4 p* W0 S; b2 c$ F p/ ~. S
Jun 7 14:35:28 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,111}]},
# Y7 o4 p0 G/ `& pJun 7 14:35:28 controller2 rabbitmq-server: {gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},
; }3 v/ T' b) f- Z8 C3 I. e# Y6 rJun 7 14:35:28 controller2 rabbitmq-server: {proc_lib,init_p_do_apply,3,
+ @! @' c5 }2 nJun 7 14:35:28 controller2 rabbitmq-server: [{file,"proc_lib.erl"},{line,237}]}]},% v; K- Q) ~) E, m
Jun 7 14:35:28 controller2 rabbitmq-server: {child,undefined,rabbit_recovery_terms,2 W& d8 E" j8 n* V8 I8 ?
Jun 7 14:35:28 controller2 rabbitmq-server: {rabbit_recovery_terms,start_link,[]},
* F H8 L" `* R2 i) FJun 7 14:35:28 controller2 rabbitmq-server: transient,4294967295,worker,
5 ~* x+ G8 ?' N4 z; GJun 7 14:35:28 controller2 rabbitmq-server: [rabbit_recovery_terms]}}}}}}}}}}. K2 ]8 l6 d, F2 \6 R4 m
Jun 7 14:35:28 controller2 rabbitmq-server: Log files (may contain more information):
7 p; w) o, O4 B' `Jun 7 14:35:28 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2.log; R% w( ^- f) x L
Jun 7 14:35:28 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2-sasl.log
# Q" Z; t, Y9 z' x1 d0 w" kJun 7 14:35:28 controller2 rabbitmq-server(ksc-os-rabbitmq)[20526]: ERROR: RabbitMQ server is failed.
8 p9 L* p/ ]6 s2 PJun 7 14:35:28 controller2 rabbitmq-server(ksc-os-rabbitmq)[20526]: INFO: Resource not running.+ ?" d& n" y) E! A1 M7 J
Jun 7 14:35:28 controller2 crmd[6475]: notice: process_lrm_event: Operation ksc-os-rabbitmq_stop_0: ok (node=controller2.ksc, call=548946, rc=0, cib-update=594835, confirmed=true)
$ M# `: }( ?$ F8 _Jun 7 14:35:28 controller2 rabbitmq-server(ksc-os-rabbitmq)[20544]: ERROR: RabbitMQ server is failed.$ f3 L+ N# y5 W. f& s, |
6 G* Z9 w( x1 R' `* EJun 7 14:35:29 controller2 systemd: Started RabbitMQ broker.
) s6 ^8 [5 ]! t/ P$ Z( tJun 7 14:35:29 controller2 rabbitmq-server: {"init terminating in do_boot",{rabbit,failure_during_boot,{could_not_start,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot,{boot_step,recovery,{badmatch,{error,{{{badmatch,{error,{not_a_dets_file,"/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},[{rabbit_recovery_terms,open_table,0,[{file,"src/rabbit_recovery_terms.erl"},{line,130}]},{rabbit_recovery_terms,init,1,[{file,"src/rabbit_recovery_terms.erl"},{line,111}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]},{child,undefined,rabbit_recovery_terms,{rabbit_recovery_terms,start_link,[]},transient,4294967295,worker,[rabbit_recovery_terms]}}}}}}}}}}}}/ n3 B% Z) \1 ?+ ~% W/ ^
Jun 7 14:35:29 controller2 crmd[6475]: notice: process_lrm_event: Operation ksc-os-rabbitmq_start_0: ok (node=controller2.ksc, call=548947, rc=0, cib-update=594836, confirmed=true)! y$ l9 N" ]* X P5 S
Jun 7 14:35:29 controller2 crmd[6475]: notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: ok (node=controller2.ksc, call=548948, rc=0, cib-update=594837, confirmed=false)
( Q; C* R9 z* a; d* R# [Jun 7 14:35:31 controller2 rabbitmq-server: Crash dump was written to: erl_crash.dump
4 y r- u3 V2 d) ?9 i. aJun 7 14:35:31 controller2 rabbitmq-server: init terminating in do_boot ()( s9 n4 Y' ?; H u
Jun 7 14:35:31 controller2 systemd: rabbitmq-server.service: main process exited, code=exited, status=1/FAILURE5 u1 K9 Z& c3 T1 V
Jun 7 14:35:31 controller2 rabbitmqctl: Stopping and halting node 'rabbit@controller2' ...
" W8 T7 m5 D6 s1 @; C4 V* u8 zJun 7 14:35:31 controller2 rabbitmqctl: Error: unable to connect to node 'rabbit@controller2': nodedown; o( q+ ?! o9 k% }1 F& R, Q
Jun 7 14:35:31 controller2 rabbitmqctl: DIAGNOSTICS
5 j4 |' k" `; `9 v& _. L7 F+ n8 w) ?Jun 7 14:35:31 controller2 rabbitmqctl: ===========
6 V+ B) D( R; s R& a. X% F/ UJun 7 14:35:31 controller2 rabbitmqctl: attempted to contact: ['rabbit@controller2']
8 a b$ N1 X N7 ^* N- n5 ~* PJun 7 14:35:31 controller2 rabbitmqctl: rabbit@controller2:4 v/ W9 E" U; b+ e2 A y
Jun 7 14:35:31 controller2 rabbitmqctl: * connected to epmd (port 4369) on controller2
0 e, I) [6 K' w" D' E' DJun 7 14:35:31 controller2 rabbitmqctl: * epmd reports: node 'rabbit' not running at all
9 E" H* Z2 o" O8 p+ F5 ?Jun 7 14:35:31 controller2 rabbitmqctl: no other nodes on controller2
/ h( X) V+ x k1 YJun 7 14:35:31 controller2 rabbitmqctl: * suggestion: start the node
2 d; C, u+ y) JJun 7 14:35:31 controller2 rabbitmqctl: current node details:
9 N; J8 H* r5 [+ O1 ]6 JJun 7 14:35:31 controller2 rabbitmqctl: - node name: 'rabbitmqctl-20636@controller2'
' o& Z7 O& m$ ?. U3 DJun 7 14:35:31 controller2 rabbitmqctl: - home dir: /var/lib/rabbitmq
; m& ^3 z o1 G5 |' j& ~4 QJun 7 14:35:31 controller2 rabbitmqctl: - cookie hash: YqnFeQ57nhtexGsA970OWw==5 ]# w' W5 u7 U$ \
Jun 7 14:35:31 controller2 systemd: rabbitmq-server.service: control process exited, code=exited status=2' w5 X6 u5 l! O& Q* [/ L
Jun 7 14:35:31 controller2 systemd: Unit rabbitmq-server.service entered failed state.9 L, v& u& f, U. Z( S! V' Z
Jun 7 14:35:31 controller2 systemd: rabbitmq-server.service failed.
0 |6 p6 \1 {" s, X, fJun 7 14:35:35 controller2 filebeat: 2024-06-07T14:35:35.162+0800 ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(http://172.24.110.127:9200)): Get "http://172.24.110.127:9200": dial tcp 172.24.110.127:9200: connect: connection timed out/ ^7 p0 {' R# d7 M; S, I0 w: q% l4 R
Jun 7 14:35:35 controller2 filebeat: 2024-06-07T14:35:35.162+0800 INFO [publisher_pipeline_output] pipeline/output.go:145 Attempting to reconnect to backoff(elasticsearch(http://172.24.110.127:9200)) with 84247 reconnect attempt(s)& E2 |/ L: M2 H/ i6 V* K; I
Jun 7 14:35:35 controller2 filebeat: 2024-06-07T14:35:35.162+0800 INFO [publisher] pipeline/retry.go:213 retryer: send wait signal to consumer
& ?- M' z. T: H9 c& B9 Y: aJun 7 14:35:35 controller2 filebeat: 2024-06-07T14:35:35.162+0800 INFO [publisher] pipeline/retry.go:217 done
+ g! R3 c) _* H* BJun 7 14:35:37 controller2 filebeat: 2024-06-07T14:35:37.219+0800 ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(http://172.24.110.125:9200)): Get "http://172.24.110.125:9200": dial tcp 172.24.110.125:9200: connect: connection timed out' B+ f+ j$ P! w/ h" }
Jun 7 14:35:37 controller2 filebeat: 2024-06-07T14:35:37.219+0800 INFO [publisher_pipeline_output] pipeline/output.go:145 Attempting to reconnect to backoff(elasticsearch(http://172.24.110.125:9200)) with 84378 reconnect attempt(s)# d/ f4 }% r" l0 {0 ^
Jun 7 14:35:37 controller2 filebeat: 2024-06-07T14:35:37.219+0800 INFO [publisher] pipeline/retry.go:213 retryer: send wait signal to consumer
! w2 E5 {, l2 A7 mJun 7 14:35:37 controller2 filebeat: 2024-06-07T14:35:37.219+0800 INFO [publisher] pipeline/retry.go:217 done0 G- k; I4 Y2 q% G0 l
Jun 7 14:35:39 controller2 startKingCloudOSAgent: 2024-06-07 14:35:39.186 5108 INFO kingcloudos.agent.tasks.agent_tasks [-] Start to report status of agent
+ a' U' u; m( ]Jun 7 14:35:39 controller2 startKingCloudOSAgent: 2024-06-07 14:35:39.211 5108 INFO kingcloudos.agent.tasks.agent_tasks [-] report agent status with {u'binary': u'startKingCloudOSAgent', u'host': u'controller2.ksc', u'uuid': u'9f541bc8-e6db-4816-ae39-0eb60cc58166', u'report_count': 3145039, u'state': u'up'}# V: K7 l* }/ N: Z8 s* f
Jun 7 14:35:39 controller2 rabbitmq-server(ksc-os-rabbitmq)[22417]: ERROR: RabbitMQ server is failed.
. e( Y& Y H9 _# Y( iJun 7 14:35:39 controller2 crmd[6475]: notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: unknown error (node=controller2.ksc, call=548948, rc=1, cib-update=594838, confirmed=false)
?, W5 a8 l- D4 ]1 x3 ?% p3 rJun 7 14:35:39 controller2 rabbitmq-server(ksc-os-rabbitmq)[22429]: ERROR: RabbitMQ server is failed.0 z" k5 ?: L0 F: l8 V
Jun 7 14:35:39 controller2 rabbitmq-server(ksc-os-rabbitmq)[22429]: INFO: Resource not running.
8 V7 P2 q! D9 e, s+ |( o, `( sJun 7 14:35:39 controller2 crmd[6475]: notice: process_lrm_event: Operation ksc-os-rabbitmq_stop_0: ok (node=controller2.ksc, call=548950, rc=0, cib-update=594839, confirmed=true)
3 w! d+ D: ]" ]# n* mJun 7 14:35:39 controller2 rabbitmq-server(ksc-os-rabbitmq)[22448]: ERROR: RabbitMQ server is failed.7 @2 i5 i; f3 Q4 P* R
Jun 7 14:35:39 controller2 systemd: Starting RabbitMQ broker...
2 L! |: A! r p$ u( c" z' n6 SJun 7 14:35:40 controller2 rabbitmqctl: Waiting for 'rabbit@controller2' ...
' p% t, y* h$ i" GJun 7 14:35:40 controller2 rabbitmqctl: pid is 22597 ...$ a8 t" w0 b3 d0 w
Jun 7 14:35:41 controller2 rabbitmq-server: RabbitMQ 3.4.0. Copyright (C) 2007-2014 GoPivotal, Inc." U" k G6 Y0 V& b
Jun 7 14:35:41 controller2 rabbitmq-server: ## ## Licensed under the MPL. See http://www.rabbitmq.com/8 a" E) @0 L1 }) y2 s
Jun 7 14:35:41 controller2 rabbitmq-server: ## ##( M( S3 a- E! d4 ?1 F. i2 L. g! u
Jun 7 14:35:41 controller2 rabbitmq-server: ########## Logs: /var/log/rabbitmq/rabbit@controller2.log
1 ]+ i3 L! `1 w; fJun 7 14:35:41 controller2 rabbitmq-server: ###### ## /var/log/rabbitmq/rabbit@controller2-sasl.log% F9 H. W' k' E# U: c) r
Jun 7 14:35:41 controller2 rabbitmq-server: ##########7 D( V8 A" T! H [: t6 }3 p
Jun 7 14:35:41 controller2 rabbitmq-server: Starting broker...
+ P* l- S/ m- `4 B# x* Q8 ?Jun 7 14:35:41 controller2 rabbitmq-server: BOOT FAILED
Z+ r+ ~# s2 L: z {* kJun 7 14:35:41 controller2 rabbitmq-server: ===========& C" s& T4 U* R8 [7 Y6 |
Jun 7 14:35:41 controller2 rabbitmq-server: Error description:1 u# c1 C2 k' ~7 J
Jun 7 14:35:41 controller2 rabbitmq-server: {boot_step,recovery,
/ t+ @4 I" t. L& wJun 7 14:35:41 controller2 rabbitmq-server: {badmatch,& q5 e' G5 ?3 Z
Jun 7 14:35:41 controller2 rabbitmq-server: {error,8 t( ] Y& D- L: h1 R
Jun 7 14:35:41 controller2 rabbitmq-server: {{{badmatch,. b) Y. O+ e, n" [- Z/ L+ ^9 w- Y
Jun 7 14:35:41 controller2 rabbitmq-server: {error,
( y( U. V1 }+ K2 ^) VJun 7 14:35:41 controller2 rabbitmq-server: {not_a_dets_file,) d" s6 e8 }! g8 V% I8 y/ g* {, J4 h) b
Jun 7 14:35:41 controller2 rabbitmq-server: "/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},
/ h* g" M! R& ~# z) v. e0 |Jun 7 14:35:41 controller2 rabbitmq-server: [{rabbit_recovery_terms,open_table,0,
. {" S8 G0 K" O4 r- Q7 ]- j9 @Jun 7 14:35:41 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,130}]},
+ d" }- b# o8 a6 i* G5 n8 ^% c' ^2 F. kJun 7 14:35:41 controller2 rabbitmq-server: {rabbit_recovery_terms,init,1,9 Q+ P# ^. J1 M- I' x
Jun 7 14:35:41 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,111}]},+ c4 {! }. k2 n! v1 g8 h g
Jun 7 14:35:41 controller2 rabbitmq-server: {gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},
5 F5 a! l* g8 L+ M! h$ wJun 7 14:35:41 controller2 rabbitmq-server: {proc_lib,init_p_do_apply,3,/ }, Y/ Y/ \6 w6 T2 o
Jun 7 14:35:41 controller2 rabbitmq-server: [{file,"proc_lib.erl"},{line,237}]}]},
8 J8 d' p/ Z! g; o0 n* g6 nJun 7 14:35:41 controller2 rabbitmq-server: {child,undefined,rabbit_recovery_terms,
7 W) M# V0 t' ^: X2 M+ v- sJun 7 14:35:41 controller2 rabbitmq-server: {rabbit_recovery_terms,start_link,[]},
u) t3 _1 I. J: C' W9 hJun 7 14:35:41 controller2 rabbitmq-server: transient,4294967295,worker,0 ?' E; `# M1 z# T6 x7 z# Q
Jun 7 14:35:41 controller2 rabbitmq-server: [rabbit_recovery_terms]}}}}}; [# j! {, d5 g S5 C& S
Jun 7 14:35:41 controller2 rabbitmq-server: Log files (may contain more information):( t% s, w4 F8 C1 X+ G- X
Jun 7 14:35:41 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2.log
" n4 v, p8 K. ?4 P3 `( RJun 7 14:35:41 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2-sasl.log
3 Y5 y# e1 y/ L4 I/ s; F" L$ bJun 7 14:35:41 controller2 rabbitmq-server: Stack trace:8 _& p6 v0 O4 ?; s: Y: L
Jun 7 14:35:41 controller2 rabbitmq-server: [{rabbit_queue_index,start,1,
8 r: P+ ^5 `4 U. U; V! j* o$ a3 hJun 7 14:35:41 controller2 rabbitmq-server: [{file,"src/rabbit_queue_index.erl"},{line,359}]},9 R; J+ ^& r. K r, ?9 g
Jun 7 14:35:41 controller2 rabbitmq-server: {rabbit_variable_queue,start,1,0 Q8 t9 M& c8 b0 z: ]4 X# o
Jun 7 14:35:41 controller2 rabbitmq-server: [{file,"src/rabbit_variable_queue.erl"},, h0 A0 e! @# V9 H! O
Jun 7 14:35:41 controller2 rabbitmq-server: {line,402}]},
9 b1 I( r( V. \! x" AJun 7 14:35:41 controller2 rabbitmq-server: {rabbit_amqqueue,recover,0,[{file,"src/rabbit_amqqueue.erl"},{line,216}]},
, m- A, i6 d9 z0 h: D9 Q+ ~; [8 BJun 7 14:35:41 controller2 rabbitmq-server: {rabbit,recover,0,[{file,"src/rabbit.erl"},{line,656}]},
- ?4 g5 v4 A& @, C; y5 E: _) EJun 7 14:35:41 controller2 rabbitmq-server: {rabbit,'-run_step/3-lc$^1/1-1-',2,[{file,"src/rabbit.erl"},{line,539}]},
, N: r2 \/ |) S9 @Jun 7 14:35:41 controller2 rabbitmq-server: {rabbit,run_step,3,[{file,"src/rabbit.erl"},{line,538}]},
2 M7 r. s; l% v/ uJun 7 14:35:41 controller2 rabbitmq-server: {rabbit,'-run_boot_steps/1-lc$^0/1-0-',1,- s% v6 |& z( Y2 M6 {: c& h3 {
Jun 7 14:35:41 controller2 rabbitmq-server: [{file,"src/rabbit.erl"},{line,525}]},
! O- e: ?. U' R/ a; wJun 7 14:35:41 controller2 rabbitmq-server: {rabbit,run_boot_steps,1,[{file,"src/rabbit.erl"},{line,525}]}]# U+ U: Z v) H$ y" G; L O! Y
Jun 7 14:35:42 controller2 rabbitmq-server: BOOT FAILED
. f: B/ `6 r N$ ~, bJun 7 14:35:42 controller2 rabbitmq-server: ===========
7 P7 Y7 ?5 L: G" cJun 7 14:35:42 controller2 rabbitmq-server: Error description:8 F6 Q$ w4 O0 c3 Y/ m1 x
Jun 7 14:35:42 controller2 rabbitmq-server: {could_not_start,rabbit,+ n1 Q. N" @7 ]: Q' c, i) \
Jun 7 14:35:42 controller2 rabbitmq-server: {bad_return,4 [" z4 X- W8 B5 v% _9 \
Jun 7 14:35:42 controller2 rabbitmq-server: {{rabbit,start,[normal,[]]},
" ^1 a1 g2 w( F3 ]) ^; R9 DJun 7 14:35:42 controller2 rabbitmq-server: {'EXIT',/ G; _; }+ M% ?+ i2 n# |) F7 k( ?) Q
Jun 7 14:35:42 controller2 rabbitmq-server: {rabbit,failure_during_boot,0 s- S \: H6 |& d5 j" j
Jun 7 14:35:42 controller2 rabbitmq-server: {boot_step,recovery," i' l" \. [0 O: c5 L2 T
Jun 7 14:35:42 controller2 rabbitmq-server: {badmatch,) c: }' h* ~+ E
Jun 7 14:35:42 controller2 rabbitmq-server: {error,3 `0 l+ a# f+ ~6 I, ~* c! I
Jun 7 14:35:42 controller2 rabbitmq-server: {{{badmatch,) Q, C* N7 Z$ |& h0 e Q
Jun 7 14:35:42 controller2 rabbitmq-server: {error,5 }6 y$ T3 Y9 H) Y3 G# L$ C- Y2 B _
Jun 7 14:35:42 controller2 rabbitmq-server: {not_a_dets_file,- ~/ D& Q* k" _4 [2 s. }* E
Jun 7 14:35:42 controller2 rabbitmq-server: "/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},
4 w1 n" z7 x1 w6 V; M" l! NJun 7 14:35:42 controller2 rabbitmq-server: [{rabbit_recovery_terms,open_table,0,& |% |# Z2 F, l1 i+ C4 X; O
Jun 7 14:35:42 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,130}]},# L% ]5 A& J v. f, C) h* v
Jun 7 14:35:42 controller2 rabbitmq-server: {rabbit_recovery_terms,init,1,
v" B6 x- R; J. O/ q1 iJun 7 14:35:42 controller2 rabbitmq-server: [{file,"src/rabbit_recovery_terms.erl"},{line,111}]},
q8 B' V- \- l' F% @Jun 7 14:35:42 controller2 rabbitmq-server: {gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},
, _. D P: ?$ @: T$ J: d j+ s- bJun 7 14:35:42 controller2 rabbitmq-server: {proc_lib,init_p_do_apply,3,& r) R3 n* y$ W+ s9 ~2 e
Jun 7 14:35:42 controller2 rabbitmq-server: [{file,"proc_lib.erl"},{line,237}]}]},
7 H9 R% y( s+ iJun 7 14:35:42 controller2 rabbitmq-server: {child,undefined,rabbit_recovery_terms,1 u4 y6 Y- I! p. J" G2 l
Jun 7 14:35:42 controller2 rabbitmq-server: {rabbit_recovery_terms,start_link,[]},
6 S% a2 ~; }) B+ K, oJun 7 14:35:42 controller2 rabbitmq-server: transient,4294967295,worker," v7 F. s' @% j+ b$ l7 V% s
Jun 7 14:35:42 controller2 rabbitmq-server: [rabbit_recovery_terms]}}}}}}}}}}" p( @7 m. Q0 j; }0 O
Jun 7 14:35:42 controller2 rabbitmq-server: Log files (may contain more information):) X' q+ C# G2 ?: x- P# U
Jun 7 14:35:42 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2.log9 q; n5 |* q( R k; V7 [8 d' R6 f
Jun 7 14:35:42 controller2 rabbitmq-server: /var/log/rabbitmq/rabbit@controller2-sasl.log6 z) I* t5 l0 L7 r& @ X; \
Jun 7 14:35:43 controller2 systemd: Started RabbitMQ broker.
) L" w* ^# _1 q- [; T. Z1 u7 rJun 7 14:35:43 controller2 crmd[6475]: notice: process_lrm_event: Operation ksc-os-rabbitmq_start_0: ok (node=controller2.ksc, call=548951, rc=0, cib-update=594840, confirmed=true)
4 Z0 l" i2 M2 l4 U8 ?Jun 7 14:35:43 controller2 rabbitmq-server: {"init terminating in do_boot",{rabbit,failure_during_boot,{could_not_start,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot,{boot_step,recovery,{badmatch,{error,{{{badmatch,{error,{not_a_dets_file,"/var/lib/rabbitmq/mnesia/rabbit@controller2/recovery.dets"}}},[{rabbit_recovery_terms,open_table,0,[{file,"src/rabbit_recovery_terms.erl"},{line,130}]},{rabbit_recovery_terms,init,1,[{file,"src/rabbit_recovery_terms.erl"},{line,111}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,306}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]},{child,undefined,rabbit_recovery_terms,{rabbit_recovery_terms,start_link,[]},transient,4294967295,worker,[rabbit_recovery_terms]}}}}}}}}}}}}
6 g$ Y0 c0 T/ i0 p @7 y: UJun 7 14:35:43 controller2 crmd[6475]: notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: ok (node=controller2.ksc, call=548952, rc=0, cib-update=594841, confirmed=false)
K, }1 m5 t ?' P( h WJun 7 14:35:45 controller2 rabbitmq-server: Crash dump was written to: erl_crash.dump
0 p* x8 S( ?' V6 a1 N. {Jun 7 14:35:45 controller2 rabbitmq-server: init terminating in do_boot ()
7 |) B+ e! E& C0 `$ S) J2 c( eJun 7 14:35:45 controller2 systemd: rabbitmq-server.service: main process exited, code=exited, status=1/FAILURE
. s2 b! a! t$ t, `' e5 ZJun 7 14:35:45 controller2 rabbitmqctl: Stopping and halting node 'rabbit@controller2' ..., S( w! q/ n8 q: d
Jun 7 14:35:45 controller2 rabbitmqctl: Error: unable to connect to node 'rabbit@controller2': nodedown9 i8 @9 y: @8 N; l8 ~7 w& _
Jun 7 14:35:45 controller2 rabbitmqctl: DIAGNOSTICS
# q t/ a$ b* O2 X nJun 7 14:35:45 controller2 rabbitmqctl: ===========; K! o+ F' W9 u, \' f
Jun 7 14:35:45 controller2 rabbitmqctl: attempted to contact: ['rabbit@controller2']# [, m8 r' y2 I* b8 m6 I" s7 {
Jun 7 14:35:45 controller2 rabbitmqctl: rabbit@controller2:' l o+ ]" l" p: |* O s
Jun 7 14:35:45 controller2 rabbitmqctl: * connected to epmd (port 4369) on controller2
+ `1 f$ y6 Q- J4 D) hJun 7 14:35:45 controller2 rabbitmqctl: * epmd reports: node 'rabbit' not running at all
8 P3 _( }% K8 JJun 7 14:35:45 controller2 rabbitmqctl: no other nodes on controller2
$ L% a. O5 B$ J# r# `4 KJun 7 14:35:45 controller2 rabbitmqctl: * suggestion: start the node
! ~% z O, M% B% p! ?Jun 7 14:35:45 controller2 rabbitmqctl: current node details:
' j( j2 _$ D& n/ }Jun 7 14:35:45 controller2 rabbitmqctl: - node name: 'rabbitmqctl-22956@controller2'
3 U* ?( K" k" q+ k/ aJun 7 14:35:45 controller2 rabbitmqctl: - home dir: /var/lib/rabbitmq
3 v+ I7 e/ k* C3 y8 q! JJun 7 14:35:45 controller2 rabbitmqctl: - cookie hash: YqnFeQ57nhtexGsA970OWw==
5 t2 Z- l5 ]: T' h/ N, }3 O! kJun 7 14:35:45 controller2 systemd: rabbitmq-server.service: control process exited, code=exited status=2: u5 x1 w" ^3 w* y0 Z& C3 r6 C
Jun 7 14:35:45 controller2 systemd: Unit rabbitmq-server.service entered failed state.* \8 h" E; c5 d b# {. i4 c0 D
Jun 7 14:35:45 controller2 systemd: rabbitmq-server.service failed.! y: {4 h! s$ P" p! z
Jun 7 14:35:45 controller2 ocf-mysql-wss: INFO: GTID OK: 62a7fc52-52de-11e8-a423-ee3e8b5b2a73:2044452185
2 y" o! P5 T2 \Jun 7 14:35:45 controller2 ocf-mysql-wss: INFO: Galera GTID: 62a7fc52-52de-11e8-a423-ee3e8b5b2a73:2044452185
( P5 `! q. Q0 s' T& b: oJun 7 14:35:49 controller2 filebeat: 2024-06-07T14:35:49.300+0800 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":890100,"time":{"ms":6}},"total":{"ticks":2986480,"time":{"ms":48},"value":2986480},"user":{"ticks":2096380,"time":{"ms":42}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":28},"info":{"ephemeral_id":"2ea13f88-dfe4-4d82-bb66-f2726d6d1965","uptime":{"ms":4384770183}},"memstats":{"gc_next":33872784,"memory_alloc":17183880,"memory_total":69311206544},"runtime":{"goroutines":133}},"filebeat":{"harvester":{"open_files":17,"running":10}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":13,"events":{"active":4126,"retry":100}}},"registrar":{"states":{"current":13}},"system":{"load":{"1":2.84,"15":2.57,"5":2.66,"norm":{"1":0.071,"15":0.0643,"5":0.0665}}}}}}1 v0 H) t+ {' j9 {- z! [8 N
Jun 7 14:35:53 controller2 dhcpd: isc-dhcpd-4.2.5% Q2 C. e+ ]0 U4 f& Y3 p% L
Jun 7 14:35:53 controller2 rabbitmq-server(ksc-os-rabbitmq)[23948]: ERROR: RabbitMQ server is failed.
7 r. W2 w/ {. N9 eJun 7 14:35:53 controller2 crmd[6475]: notice: process_lrm_event: Operation ksc-os-rabbitmq_monitor_10000: unknown error (node=controller2.ksc, call=548952, rc=1, cib-update=594842, confirmed=false)
3 o8 Q9 D) H" n W1 D/ [' ], IJun 7 14:35:54 controller2 rabbitmq-server(ksc-os-rabbitmq)[23963]: ERROR: RabbitMQ server is failed. C7 o9 R% X/ _+ N$ y
Jun 7 14:35:54 controller2 rabbitmq-server(ksc-os-rabbitmq)[23963]: INFO: Resource not running.
" a: a1 D5 Z( ^% iJun 7 14:35:54 controller2 crmd[6475]: notice: process_lrm_event: Operation ksc-os-rabbitmq_stop_0: ok (node=controller2.ksc, call=548954, rc=0, cib-update=594843, confirmed=true)
* x7 T! d- G+ I6 `Jun 7 14:35:57 controller2 filebeat: 2024-06-07T14:35:57.029+0800 ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(http://172.24.110.126:9200)): Get "http://172.24.110.126:9200": dial tcp 172.24.110.126:9200: connect: connection timed out
2 g" \) \% a9 Y- B' l% l0 TJun 7 14:35:57 controller2 filebeat: 2024-06-07T14:35:57.030+0800 INFO [publisher_pipeline_output] pipeline/output.go:145 Attempting to reconnect to backoff(elasticsearch(http://172.24.110.126:9200)) with 84327 reconnect attempt(s)5 ~) a" C$ ]) @1 e
Jun 7 14:35:57 controller2 filebeat: 2024-06-07T14:35:57.030+0800 INFO [publisher] pipeline/retry.go:213 retryer: send wait signal to consumer q- p7 M- j% n) |5 C; A: K. \' ?
Jun 7 14:35:57 controller2 filebeat: 2024-06-07T14:35:57.030+0800 INFO [publisher] pipeline/retry.go:217 done
* o, M# B9 S1 \7 f9 f/ G' e9 D& Q2 [, w& z- B( I9 K
# n7 [% r/ a# m8 z) c, H
Jun 7 14:36:06 controller2 ocf-mysql-wss: INFO: GTID OK: 62a7fc52-52de-11e8-a423-ee3e8b5b2a73:20444524244 {) A8 n# P7 P" y5 q4 v
Jun 7 14:36:06 controller2 ocf-mysql-wss: INFO: Galera GTID: 62a7fc52-52de-11e8-a423-ee3e8b5b2a73:20444524244 ?4 e; H- J, M, B5 j
4 C9 Q$ W% q! v5 j& z* o9 ]0 k' C5 B
: k) s D- O# t6 J0 I; o
|
|
发表于 2024-6-7 15:00:19