|
|
楼主 |
发表于 2021-10-18 11:12:53
|
显示全部楼层
[root@controller1 _data]# kolla-ansible -i multinode mariadb_recovery& Y, l/ f/ f9 D. w% g8 R
Attempting to restart mariadb cluster : ansible-playbook -i multinode -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e action=deploy -e common_run=true /usr/share/kolla-ansible/ansible/mariadb_recovery.yml , `( F8 o$ J4 w
/usr/lib/python2.7/site-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.6 `1 U) A3 U6 \
from cryptography.exceptions import InvalidSignature* R% S( U8 G, V [& W
[WARNING]: Unable to parse /var/lib/docker/volumes/mariadb/_data/multinode as an inventory source
, D) a: V+ S, X `9 L[WARNING]: No inventory was parsed, only implicit localhost is available2 b! U% [8 y0 W; }
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'- n- } P4 h3 T6 K% {. V$ V7 M3 @
[WARNING]: Could not match supplied host pattern, ignoring: mariadb$ R; @/ ^& P9 y) h5 H* c
# w3 S! j" S7 A/ H h2 u4 KPLAY [Detect openstack_release variable] **************************************************************************************************************************
! P& }6 s* C. a( `3 Jskipping: no hosts matched. v9 J$ k- h7 j0 _8 M' T
PLAY [Recover mariadb] ********************************************************************************************************************************************) z! N4 u* E+ J1 W( j, {2 n
skipping: no hosts matched7 E0 O& {) n2 W9 [) y2 K
PLAY RECAP ********************************************************************************************************************************************************
: ~+ ?0 r* Q$ [/ J9 \[root@controller1 _data]# cd ' B" q# b8 C9 e; W( [' U
[root@controller1 ~]# kolla-ansible -i multinode mariadb_recovery- b* g1 u \4 f4 ]
Attempting to restart mariadb cluster : ansible-playbook -i multinode -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e action=deploy -e common_run=true /usr/share/kolla-ansible/ansible/mariadb_recovery.yml . m* L: u2 n- |. R( I
/usr/lib/python2.7/site-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release./ c' h9 V& p% s) D v
from cryptography.exceptions import InvalidSignature: @3 w. ?2 {" c: d
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details% l, J N4 j9 A
[WARNING]: Found variable using reserved name: action
$ G6 B6 t, ?) g1 g. d( u& I$ a$ k7 h; V1 I8 ^$ z
PLAY [Detect openstack_release variable] **************************************************************************************************************************
e0 q, ]* Z* j3 ]% V5 O( Z4 sTASK [Get current kolla-ansible version number] *******************************************************************************************************************3 ~0 ]" f# X9 D# ]- k# U/ _$ d7 A
skipping: [controller1] j2 `3 r8 h3 z+ }8 }; O
skipping: [controller2]
( G) J+ h7 |& |6 @skipping: [controller3]
' H5 I' Y2 I1 c: V0 [3 QTASK [Set openstack_release variable] *****************************************************************************************************************************. m) @) Z* T5 I" G0 }1 H8 c6 G- h
skipping: [controller1]0 R# u) L& d0 D$ r+ o% L
skipping: [controller2]
: ]2 }" F; k% b8 [skipping: [controller3]
. M3 ^% Y; @) c# w- Q5 q7 ^PLAY [Recover mariadb] ********************************************************************************************************************************************
/ B8 E! a* Q8 e' b9 ^9 y5 e" `TASK [Gathering Facts] ********************************************************************************************************************************************3 a/ v% V+ x% ^ @* O% H
ok: [controller1]1 Q: `7 [4 |+ s$ e
ok: [controller3]
" ?% W) d( U2 B+ E+ dok: [controller2]
9 v1 c* V* ~1 ]; ]: b& @TASK [common : include] *******************************************************************************************************************************************
; Z% ]* R- m' x$ _% H2 s* mskipping: [controller1]8 X0 I$ l! `5 T# g1 F8 @' w
skipping: [controller2]9 k1 F! m6 c% L/ {1 v- |! |
skipping: [controller3]
* j5 Z. l; a! d9 F6 c# y7 \$ wTASK [common : Registering common role has run] *******************************************************************************************************************
' O8 a: \/ u- z |skipping: [controller1]
: X% I; ?0 e( }skipping: [controller2]
0 d( |8 p% A! U$ h6 I5 N0 ~$ hskipping: [controller3]
9 S& b" A8 S" J2 ETASK [mariadb : include] ******************************************************************************************************************************************- c7 ?, v2 \% M) B- p
included: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/deploy.yml for controller1, controller2, controller3
5 z9 _8 U8 H$ V d9 PTASK [mariadb : include] ******************************************************************************************************************************************" P& Z; q7 m7 M4 D) n+ q
included: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/config.yml for controller1, controller2, controller3
2 ^. f0 j: e/ ~+ v/ W2 g" `* U4 ]TASK [mariadb : Ensuring config directories exist] ****************************************************************************************************************
6 C; e& u( g; D, iok: [controller2] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})
4 e+ m7 t3 v# t. }& p1 E5 mok: [controller3] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})/ f- U* U+ Q. U* [7 g' a- i
ok: [controller1] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})5 g) Z V! d; G; m8 I
TASK [mariadb : Copying over config.json files for services] ******************************************************************************************************! N5 G5 [9 Z! v
ok: [controller2]* P% n( R8 |: B) b
ok: [controller3]5 Y: m- `! |/ U* p4 f2 U
ok: [controller1]$ |+ x% D! R& N! C: \: A
TASK [mariadb : Copying over galera.cnf] **************************************************************************************************************************
5 G; m2 ]! S+ ?' w' L, g+ w2 W[WARNING]: _execute_module no longer honors the tmp parameter. Action plugins should set self._connection._shell.tmpdir to share the tmpdir4 ]) S& ~' P4 O0 Y: A
[WARNING]: _execute_module no longer honors the tmp parameter. Action plugins should set self._connection._shell.tmpdir to share the tmpdir: N9 R2 N8 K8 S: I( W
[WARNING]: _execute_module no longer honors the tmp parameter. Action plugins should set self._connection._shell.tmpdir to share the tmpdir
( M( K1 Y; Y1 ?6 Z9 {1 z( Z& }5 t8 xok: [controller3]
7 C$ d6 `; K& K, ook: [controller2]
/ D, Z1 Y h8 q" Nok: [controller1]
& c" F% a7 _; Z, j& t. [TASK [mariadb : Copying over wsrep-notify.sh] *********************************************************************************************************************
% [% K' j' K( b8 v2 E+ q' W) Mok: [controller2] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})
2 ]: U8 l- W" b2 d6 y. |ok: [controller1] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})" ], E- k4 s, A
ok: [controller3] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'}); g% o+ L4 e, [ }
TASK [mariadb : Check mariadb containers] *************************************************************************************************************************
7 _; ?' e# H% X! Y9 J0 w2 Gchanged: [controller3] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})- [3 f& M, H; `, u2 w0 }
changed: [controller1] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})2 T& f O4 N7 e; V$ q2 k4 m
changed: [controller2] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})
! o, W/ F6 H+ V, {3 X6 V/ ^3 R& fTASK [mariadb : include] ******************************************************************************************************************************************$ S6 {7 g$ i4 O" Z3 M$ b* f
included: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/bootstrap.yml for controller1, controller2, controller3, y- n9 ]4 m8 p# r }1 \; V
TASK [mariadb : set_fact] *****************************************************************************************************************************************
( ]6 }% u" n4 k) I% ?ok: [controller1]
: V0 K# r; {; o3 ^" A- k; t9 m) jok: [controller2]" h% |8 n: q6 {8 m
ok: [controller3]
0 t- d6 @/ v* l* r. J" \TASK [mariadb : include] ******************************************************************************************************************************************
9 M0 V8 z) K" O2 Y, \included: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/lookup_cluster.yml for controller1, controller2, controller3
% m" ^+ g9 I6 s( X. |: l7 F( MTASK [mariadb : Cleaning up temp file on localhost] ***************************************************************************************************************
& }3 M: h: ?& w# K& Mok: [controller1 -> localhost]
( H' n" L. {4 J( r9 KTASK [mariadb : Creating temp file on localhost] ******************************************************************************************************************
* Y& d2 C+ B; l9 T y4 l8 Qok: [controller1 -> localhost]
( `" j: I) ^8 k, vTASK [mariadb : Creating mariadb volume] **************************************************************************************************************************
+ v, q7 l! [8 P; Jok: [controller1]
& @* h. [5 s1 g2 d5 _% Qok: [controller2]0 u, z( }4 X/ \ X v
ok: [controller3]' z5 T* }* U9 U& M' i1 J H3 ^
TASK [mariadb : Writing hostname of host with existing cluster files to temp file] ********************************************************************************
9 }* \9 N! |. @/ c% ][DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|changed` use `result is changed`. This feature will be removed in version & H! J# N% W7 ]2 n( E
2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
) o' x' v6 P+ t[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|changed` use `result is changed`. This feature will be removed in version
- ?( V& X! @. ^2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.' S* W- H& n; s/ w* n: s# d
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|changed` use `result is changed`. This feature will be removed in version
N5 o6 i* H5 Y( O) A& l: }2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.$ Z2 m) K. E6 x1 M
ok: [controller1 -> localhost]
' d8 m0 i0 Z, e! U! ?& gok: [controller2 -> localhost]7 J4 X* ^+ e( ~% V
ok: [controller3 -> localhost]
4 t/ |; {( s) G/ LTASK [mariadb : Registering host from temp file] ******************************************************************************************************************
0 A! B; e( [ E# f% V7 {ok: [controller1]
' u, j" J2 U* o- v0 o. |; Q% ^ok: [controller2]. V8 S1 M* x" G$ e s d$ K
ok: [controller3]
6 S1 P1 {* C pTASK [mariadb : Cleaning up temp file on localhost] ***************************************************************************************************************
# G5 G0 }* O% }0 [ok: [controller1 -> localhost]- N& [0 J* g X& _: p+ u) f3 X
TASK [mariadb : include] ******************************************************************************************************************************************( E3 Y" H1 p9 W+ F
skipping: [controller1]
. E. @% n8 H, z6 w( T* fskipping: [controller2]
) C& t* O* T- U7 C& D. ^8 g6 J6 {skipping: [controller3]$ r/ F6 H; O2 r6 k/ V% j* q2 v& J
TASK [mariadb : include] ****************************************************************************************************************************************** Z4 Z. {6 M+ V* z7 h8 v m
included: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/recover_cluster.yml for controller1, controller2, controller3. r6 u9 X( O8 N" |: V0 z
TASK [mariadb : fail] *********************************************************************************************************************************************
! ?, \4 ?* G/ X8 ~0 D; ?. }skipping: [controller1]3 f N, I/ k. a% A* `
skipping: [controller2]
5 P9 w; x5 G3 J' m4 \skipping: [controller3]
, i0 N' q$ a& B2 c; c, ~3 gTASK [mariadb : Cleaning up temp file on mariadb hosts] ***********************************************************************************************************
; d: ^: ^4 ^" O8 v. I8 A) Jok: [controller2]
2 c2 L- M; Y+ `& ?ok: [controller1]0 i. T. Y' y0 a; _ f
ok: [controller3]
5 b$ ~. {( r, `8 s7 t0 G4 N: l, _ _$ OTASK [mariadb : Cleaning up temp file on localhost] ***************************************************************************************************************# I# O( g% M$ M$ D0 C2 V
ok: [controller1 -> localhost]) f1 a4 _2 X3 j5 a' e# I8 t$ X
TASK [mariadb : Stop MariaDB containers] **************************************************************************************************************************% c, U9 P) R. O! \- l+ H
ok: [controller2]1 }" I# b5 P; V
ok: [controller1]! K) g: M$ Q# L- c7 B. O
ok: [controller3]
# B7 U( M* a4 D" YTASK [mariadb : Run MariaDB wsrep recovery] ***********************************************************************************************************************4 T" {# U9 y7 ]3 L
changed: [controller1]# o6 a7 `& m- M# G% I5 d
changed: [controller2], v1 c. R3 |6 U5 {5 c9 o! h
changed: [controller3]3 |. J4 i8 s, M4 u, l2 p
TASK [mariadb : Copying MariaDB log file to /tmp] *****************************************************************************************************************
$ l" r# ?7 ?' u' ^: ?" Pchanged: [controller3]
! Y5 t4 H& p) d1 S" g0 p/ z' Mchanged: [controller2]& {5 i1 t" ]" X% f* H j4 X: e, W
changed: [controller1]# U7 b9 r' ~. ?- g
TASK [mariadb : Get MariaDB wsrep recovery seqno] *****************************************************************************************************************7 @* N# f6 y; v3 Y! ]) r
changed: [controller2]
' h% f3 u2 ?) R- h3 J' wchanged: [controller1]
! g7 W, p- y, [* c4 D0 W7 K; ^7 o3 Tchanged: [controller3]/ C, U) z1 m5 n- A+ w! f& Y( f; f1 Z
TASK [mariadb : Removing MariaDB log file from /tmp] **************************************************************************************************************
9 b# i# q- ~' ]: V& y% `ok: [controller2]/ Z M b! x# s: T
ok: [controller1]
5 }: z8 [7 `9 Z* e5 L' h/ S/ ?/ hok: [controller3]( A% E+ Y& n8 h) r. a
TASK [mariadb : Registering MariaDB seqno variable] ***************************************************************************************************************
3 p3 U1 Z& f7 Q- j- F; _3 Cok: [controller1]
5 c7 i5 R3 ~. Y1 Ook: [controller2]. E9 j' [' o4 Z7 o) w" i
ok: [controller3]* a9 N% a) {# i# x, u% \2 x4 B
TASK [mariadb : Comparing seqno value on all mariadb hosts] *******************************************************************************************************
3 L8 U6 |0 [! ~; Q: P8 aok: [controller2] => (item=controller1)
2 m" v% s8 G8 _3 ^! i3 X7 ?4 rok: [controller1] => (item=controller1)9 v; L8 M4 B# b }% O0 Q9 U
ok: [controller3] => (item=controller1). R+ u. @" z1 N; q! Q0 H
ok: [controller2] => (item=controller2)5 l$ `4 M) W, L1 x! E6 T
ok: [controller3] => (item=controller2)4 n& G) @5 w* Y. o6 L6 x5 d
ok: [controller1] => (item=controller2)
) g( o" X+ [) j2 Z& q) `+ e5 tok: [controller2] => (item=controller3)! a. d* o: H$ s, i( _
ok: [controller3] => (item=controller3)* M% r2 n! |1 @1 U
ok: [controller1] => (item=controller3)
9 m+ i: J0 O: j$ i0 yTASK [mariadb : Writing hostname of host with the largest seqno to temp file] *************************************************************************************
6 v. s+ v. {& ~skipping: [controller3]* B @1 u1 K# L. I8 c
ok: [controller1 -> localhost]
6 L4 P7 u3 @0 I+ h$ C* `, L- mok: [controller2 -> localhost]5 @$ P& |" Q' } @' w! Q
TASK [mariadb : Registering mariadb_recover_inventory_name from temp file] ****************************************************************************************; L3 w* Y5 s- {6 O6 K
ok: [controller1]
) h( d9 K& g7 p f3 a( c* Y: b, iok: [controller2]" p1 W$ j- r0 o5 O
ok: [controller3]3 r1 X" t6 t1 Q/ l6 n2 S [) q0 _, U
TASK [mariadb : set_fact] *****************************************************************************************************************************************. @6 m, U V$ p) i( `
changed: [controller1]
) m4 l* o/ a- ?+ n0 vchanged: [controller2]/ @2 P- V" z& l0 J9 Q# q9 m
changed: [controller3]
) ?* r5 y0 b! |$ v; t! i2 W2 | FTASK [mariadb : Copying grastate.dat file from MariaDB container in bootstrap host] *******************************************************************************/ s9 G: }/ [- p4 J7 w
skipping: [controller1]8 K! W6 \% g; |2 D0 q! a
skipping: [controller3], c! G, w9 @4 p5 j2 }2 N/ C, E
ok: [controller2]
' ~2 d. u0 ~- ?. ?) \' ]* oTASK [mariadb : Set grastate.dat file from MariaDB container in bootstrap host] ***********************************************************************************
. Y% J8 Y7 E" n# {9 pskipping: [controller1]( i% c. @) Q. T; J* q4 t
skipping: [controller3]; M1 T2 ?& D2 c0 n8 c3 k& V6 e
changed: [controller2]+ U" C7 j2 q) _2 A K
TASK [mariadb : Copying grastate.dat file to mariadb container] ***************************************************************************************************- j5 T" Y r3 D [5 b! I; \
skipping: [controller1]% d' H. C) R. x, H, L9 M
skipping: [controller3]
% F9 i0 E, I9 qok: [controller2]& X' {8 K# G* P) l
TASK [mariadb : Starting first MariaDB container] *****************************************************************************************************************' }4 U6 s6 W8 U% p& E( M
skipping: [controller1], `8 ]5 r+ j( ?( K% F
skipping: [controller3]
* S+ ?3 J2 x. U9 schanged: [controller2]
" v. m, Y) N. e8 f9 `4 _TASK [mariadb : Wait for first MariaDB container] *****************************************************************************************************************5 o6 k+ t( H8 Z6 O, p" K
skipping: [controller1]( Y j1 B" W7 ~/ a( c
skipping: [controller3]
2 p0 ^( y% b& [# K5 d0 y6 R& S[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|success` use `result is success`. This feature will be removed in version ( h" g( T* r, k @' n. l
2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg." g; k4 l% E6 f. I
FAILED - RETRYING: Wait for first MariaDB container (10 retries left).5 q6 E- J/ @: T3 E
ok: [controller2]/ p# q/ c; Y8 {6 n+ ^: H: r4 U
TASK [mariadb : Set first MariaDB container as primary] ***********************************************************************************************************
/ Q4 y2 _7 t+ p1 [( _$ f- j; Q& Yskipping: [controller1]
' g+ q! w0 ]) l j5 Vskipping: [controller3]$ C. L: g; k2 { }% ?- k, s
changed: [controller2]0 {: B S0 U( K
TASK [mariadb : Wait for MariaDB to become operational] ***********************************************************************************************************' D* m# X8 x( L# \
skipping: [controller1]4 C- K0 ~* ]/ N5 o$ h4 t$ H
skipping: [controller3]
. W! R( {' q' Y( n8 P. H. h5 |ok: [controller2]
9 Q& ~' z) N) ?6 |6 VTASK [mariadb : Restart slave MariaDB container] ******************************************************************************************************************
; v" L/ @( ~% R( o, [5 \4 j0 d9 _skipping: [controller2]* }6 N! W6 k' `4 g# B
changed: [controller3]
3 ~5 |9 g, F7 S l, D; `" d& }changed: [controller1]
4 T) u1 M* I8 }4 j; aTASK [mariadb : Wait for slave MariaDB] *************************************************************************************************************************** L B) R* D) o( r; I6 d! I- M
skipping: [controller2]# B7 J, x7 Y$ q$ U4 j
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|success` use `result is success`. This feature will be removed in version
3 v/ X6 e& U# E2 |# Y2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
. d' H3 m3 c- W' U6 e$ E6 kFAILED - RETRYING: Wait for slave MariaDB (10 retries left).( m+ F; x m, H5 {# M
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|success` use `result is success`. This feature will be removed in version $ Y' g8 I* X1 A7 R, c9 B
2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
5 s A' K) I: H) ?% mFAILED - RETRYING: Wait for slave MariaDB (10 retries left).% k: c' h0 ?6 h6 F& @
ok: [controller3]
6 w! X1 J5 B% rok: [controller1]
* o& v/ ?9 Z. S: G% ?( C9 F% N9 u2 k[WARNING]: flush_handlers task does not support when conditional0 i I) K& n0 k9 z- p6 r& u
7 q' Z& \* ]. S( }& E A
RUNNING HANDLER [mariadb : restart slave mariadb] *****************************************************************************************************************
2 m" H9 N% n" s' w9 b. Yskipping: [controller3]( U; d/ {" ^% M, N3 \* r# J
skipping: [controller1]
, ^6 u- } H$ L, u3 v! y3 r/ L, Uskipping: [controller2]
$ r4 o8 g% I+ K& YRUNNING HANDLER [mariadb : restart master mariadb] ****************************************************************************************************************
6 P! U t. x$ V, {1 a6 R1 lskipping: [controller3]
- P. w0 X. R! vskipping: [controller1]
( v7 l$ p; ?$ Uskipping: [controller2]( \% I* l, Y9 J
TASK [mariadb : include] ******************************************************************************************************************************************# t k+ C6 J, d% V0 U9 s$ \' n
included: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/register.yml for controller1, controller2, controller3
* p& H9 L) B7 k1 hTASK [mariadb : Creating haproxy mysql user] **********************************************************************************************************************6 }, t9 `/ {7 @! X) `2 c+ w8 U
ok: [controller1]) B7 ^$ r( @' M$ V) s
TASK [mariadb : Cleaning up facts] ********************************************************************************************************************************
& t$ c% y. `1 Q; I% @6 Z) Lok: [controller1]2 o# R4 V: P9 E2 U
ok: [controller2]! g5 z7 Q2 O) j1 c
ok: [controller3]$ e( c6 f" }* G! ^
TASK [mariadb : include] ******************************************************************************************************************************************
3 P& e; ^! ^3 x7 x/ Y7 i/ J5 @9 ]included: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/check.yml for controller1, controller2, controller3- M/ U( N$ S3 G3 c: T; ^
TASK [mariadb : Waiting for MariaDB service to be ready through VIP] **********************************************************************************************1 Z+ Y6 Q1 a& ~
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|success` use `result is success`. This feature will be removed in version
; e, {: f. @ Y% T. m y/ }( K9 _! @2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg." r- E% _# K* g7 S. e. _
ok: [controller1]
: }% C' ?2 Y! k6 }! f- V1 o[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|success` use `result is success`. This feature will be removed in version 9 X8 b+ O, l* [! P! S
2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.4 X( N9 I) Z! ~+ }
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|success` use `result is success`. This feature will be removed in version ) P S2 U& f" C9 j# f
2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
% h4 h) `. f$ z- Nok: [controller2]* i6 k# Z1 ^: o- ~! N% ?* {
ok: [controller3]
4 ?- k( _2 b% D) u1 Y `! }* z3 LPLAY RECAP ********************************************************************************************************************************************************; u/ g$ V; q3 G) \- g5 L
controller1 : ok=37 changed=6 unreachable=0 failed=0 skipped=15 rescued=0 ignored=0
8 K9 v; l6 a6 scontroller2 : ok=37 changed=8 unreachable=0 failed=0 skipped=10 rescued=0 ignored=0 9 T1 O; N3 K* l
controller3 : ok=31 changed=6 unreachable=0 failed=0 skipped=16 rescued=0 ignored=0
2 q/ v) r5 ]0 j5 s. w
: d2 K5 H ~# q: N4 I; s' R* l# i恢复正常。
: F0 e6 U1 [- T% q- F, H& g. ^; C
; K4 d- f4 l1 n/ K! K9 z) D7 j: R
有时候通过一些特殊手段可以帮助我们解决一些特别的问题,与其长时间尝试老方法,不如来点新的方式试试或许不一样的效果。8 d0 Z2 x9 H9 b2 ?9 N
/ {3 Z* o! W9 t$ p/ _/ h+ a& _
, F5 d k J n; l% x我们首先是因为数据库恢复出问题,无法恢复数据,, 检查数据的库的值,
9 s7 b9 F1 L6 mcat /var/lib/docker/volumes/mariadb/_data/grastate.dat 中seqno值,若该值在所有节点中存在唯一得最大值
. C" W |+ v& U! g6 I然后我们保留这个节点上数据,
5 \+ ?/ S) Z! M; o' ?0 B删除其他节点上的数据文件# I& }& ~) T% ^- A$ _# B; a1 s
然后通过scp或者rsync同步保留节点的数据文件 (根据个人喜好同步文件,本人建议使用rsync), R( q; s: n5 |* c% d
然后再执行kolla-ansible数据库mariadb_recovery恢复命令:$ g, ?* \+ u& h1 F
' n# _! j4 {# r7 C" `
# `4 [" w0 b6 p( H; c" C1 x* |8 z. p: X' p$ v- z
( m- d! @& }" p5 N, f/ x1 T
差不多就可以完成数据库集群恢复了。
6 ~4 O; T. z/ d9 N S/ E: q5 } |
|