|
|
楼主 |
发表于 2021-10-18 11:12:53
|
显示全部楼层
[root@controller1 _data]# kolla-ansible -i multinode mariadb_recovery: R, O" Q" v% ~ k4 r5 D& \- H
Attempting to restart mariadb cluster : ansible-playbook -i multinode -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e action=deploy -e common_run=true /usr/share/kolla-ansible/ansible/mariadb_recovery.yml 0 V' i# I& W( s6 v& D/ Z, v! e
/usr/lib/python2.7/site-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
0 ?8 E! h7 V, ?& H8 ~+ K from cryptography.exceptions import InvalidSignature; e4 L% @$ I+ x! q* d
[WARNING]: Unable to parse /var/lib/docker/volumes/mariadb/_data/multinode as an inventory source V1 m' q$ O$ e1 |
[WARNING]: No inventory was parsed, only implicit localhost is available
$ {' ~% k( o' m( C' y$ w& w[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'. ~7 T" b; F: t/ `
[WARNING]: Could not match supplied host pattern, ignoring: mariadb
/ e5 u v- h8 q' a3 Q) z; P: j( i" a( \! p1 C3 f; Z3 f5 ? O; u
PLAY [Detect openstack_release variable] *************************************************************************************************************************** K7 q. z+ \. @& v6 V: O
skipping: no hosts matched/ V8 I9 e" T8 ~8 l; _" w! W: l
PLAY [Recover mariadb] ********************************************************************************************************************************************$ D% s# D; T1 A: y1 r, r2 P4 B
skipping: no hosts matched
, f& p, M- ~* `9 Z( EPLAY RECAP ********************************************************************************************************************************************************9 T" m9 _ I( W) ?
[root@controller1 _data]# cd * }+ F, e4 |9 t1 R1 j9 M& ^8 Y8 q! v: @
[root@controller1 ~]# kolla-ansible -i multinode mariadb_recovery1 c( b i4 E5 R8 X/ b, z. Z
Attempting to restart mariadb cluster : ansible-playbook -i multinode -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e action=deploy -e common_run=true /usr/share/kolla-ansible/ansible/mariadb_recovery.yml 2 i8 j4 B$ v- h% b
/usr/lib/python2.7/site-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release." E5 j0 b* P( o8 y5 _# L# Q
from cryptography.exceptions import InvalidSignature# l; x1 K7 E1 }2 u; T0 ~; E
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details6 `% Z4 g4 I: s5 q
[WARNING]: Found variable using reserved name: action
& k- D& y, d5 ^7 X3 c
! Z2 F0 C2 S( sPLAY [Detect openstack_release variable] **************************************************************************************************************************
& f# X9 x: s+ l* w3 UTASK [Get current kolla-ansible version number] *******************************************************************************************************************
/ f$ R2 L+ ] Y& j+ c- o; q9 }5 _skipping: [controller1]
8 k2 U) g6 B! m( Z3 {skipping: [controller2]: Y* u4 g8 U$ O
skipping: [controller3]
; _& d. @* q) i" ITASK [Set openstack_release variable] *****************************************************************************************************************************
7 m, I5 \2 r, P: T# yskipping: [controller1]
" W9 r0 e: J. ?: \: c4 Yskipping: [controller2]
3 {' D a+ R) X- s+ E( Nskipping: [controller3]
; f2 I1 j+ ]* }2 ?PLAY [Recover mariadb] ********************************************************************************************************************************************
* F; y4 K% Q/ f4 f/ D. r8 i8 o" r' p# |TASK [Gathering Facts] ********************************************************************************************************************************************
1 w' I& e2 Z8 @ fok: [controller1]; G0 u: B7 i* }+ T# K! T! v
ok: [controller3], g) q1 }! N' z; E) C8 w
ok: [controller2] F/ s3 Q/ o/ l; A/ g& G" A
TASK [common : include] *******************************************************************************************************************************************7 g+ h( Y# I- _- ]% |1 a& l+ f
skipping: [controller1]
) K- z4 {# W( k" F, zskipping: [controller2]
: L; ?# ]2 \9 W: `skipping: [controller3]- c; A+ l; `& _$ z$ s9 ^7 c
TASK [common : Registering common role has run] *******************************************************************************************************************3 |. T$ b) A& q$ |
skipping: [controller1]: ~" m) T5 C" n0 U" R
skipping: [controller2]& I) ~) J. G# M
skipping: [controller3]
/ L, M, C, B6 v& j5 PTASK [mariadb : include] ******************************************************************************************************************************************5 i: j: u* h8 q4 `
included: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/deploy.yml for controller1, controller2, controller34 h: m8 l$ a" x8 V
TASK [mariadb : include] ******************************************************************************************************************************************
6 y. N6 B8 M, V& R L- T9 D; @included: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/config.yml for controller1, controller2, controller3
8 Q( e5 N* F2 v& b G$ TTASK [mariadb : Ensuring config directories exist] ****************************************************************************************************************( e+ E2 H* `: @; P/ \
ok: [controller2] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})+ L: c0 V S" J7 F% X/ k8 Z5 J. \4 N
ok: [controller3] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})# r9 T# Z% b4 `
ok: [controller1] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})
) ^9 T% j! j; S! J/ w8 W) z. V, ITASK [mariadb : Copying over config.json files for services] ******************************************************************************************************
2 \; v, a- c) P) c0 L" W, kok: [controller2]
- P6 y% S X7 `% }ok: [controller3]7 _( x9 c) k. c T9 ]1 _
ok: [controller1]
' l# k' T3 r$ X1 r* l" fTASK [mariadb : Copying over galera.cnf] **************************************************************************************************************************8 W& B n e8 l* \
[WARNING]: _execute_module no longer honors the tmp parameter. Action plugins should set self._connection._shell.tmpdir to share the tmpdir; @, c! c, Y: c9 Y9 n- B$ `8 m
[WARNING]: _execute_module no longer honors the tmp parameter. Action plugins should set self._connection._shell.tmpdir to share the tmpdir7 }9 l( |( c u2 q \
[WARNING]: _execute_module no longer honors the tmp parameter. Action plugins should set self._connection._shell.tmpdir to share the tmpdir! h) s# h1 m& d+ T1 E
ok: [controller3], c: S# E" R- V5 i6 c- _4 L
ok: [controller2]2 u$ z' d2 X3 w2 e
ok: [controller1]
- K7 f: f% X5 q2 v9 CTASK [mariadb : Copying over wsrep-notify.sh] *********************************************************************************************************************( y. w8 G" ]$ n5 B5 j: q( E
ok: [controller2] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})
9 P9 h/ W( \% R2 S; r) r7 d$ Y. _ok: [controller1] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})
; n6 P$ D; z4 {/ V: N Iok: [controller3] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})
& h' E( t3 [: H: |; _TASK [mariadb : Check mariadb containers] *************************************************************************************************************************# q7 T1 Z9 Q6 E' A- L+ Z+ k! W
changed: [controller3] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})
9 ^$ N" ?. f# F% U3 Z" B! bchanged: [controller1] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})
! g& k1 g2 }$ R+ pchanged: [controller2] => (item={'value': {u'enabled': True, u'image': u'kolla/centos-binary-mariadb:queens', u'group': u'mariadb', u'volumes': [u'/etc/kolla//mariadb/:/var/lib/kolla/config_files/:ro', u'/etc/localtime:/etc/localtime:ro', u'mariadb:/var/lib/mysql', u'kolla_logs:/var/log/kolla/'], u'container_name': u'mariadb'}, 'key': u'mariadb'})
( m# u, P+ [! O. v/ GTASK [mariadb : include] ******************************************************************************************************************************************8 p P. k# B H4 J f0 i0 p+ D, d
included: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/bootstrap.yml for controller1, controller2, controller3
9 {# e( q% s$ G4 N# {' qTASK [mariadb : set_fact] *****************************************************************************************************************************************
8 @+ y* B/ s$ Q, k! N% f9 r' h) Jok: [controller1]* c2 @( K# Q" |$ r ?) z
ok: [controller2]$ u# |8 R% V2 R# x5 S! y6 @$ E* D
ok: [controller3] ^: J4 U6 [3 Y; |
TASK [mariadb : include] ******************************************************************************************************************************************. ~* E6 e& |, ^1 J
included: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/lookup_cluster.yml for controller1, controller2, controller3
, b" ~8 P9 Y% s# FTASK [mariadb : Cleaning up temp file on localhost] ***************************************************************************************************************
/ j: ?3 I: o' z5 A1 M6 d. t2 Gok: [controller1 -> localhost]: U3 y, }. b7 [$ Q* h
TASK [mariadb : Creating temp file on localhost] ******************************************************************************************************************: D( U2 E8 V+ r. S% r1 G ^
ok: [controller1 -> localhost]0 N/ z6 t8 m' W* b* w
TASK [mariadb : Creating mariadb volume] ************************************************************************************************************************** _; y( `. I, l3 b0 C5 e# q+ K
ok: [controller1]
1 d2 K5 I- s, ~ok: [controller2]. k- `4 L3 R2 d C3 J
ok: [controller3]5 s9 m9 W: c6 c* z$ k" N. n
TASK [mariadb : Writing hostname of host with existing cluster files to temp file] ********************************************************************************& J! H( q: d4 w! Z
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|changed` use `result is changed`. This feature will be removed in version + E4 ~- c6 F0 y% \' Y A2 k
2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
; O) j9 [7 j& \6 C7 t[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|changed` use `result is changed`. This feature will be removed in version % H* _5 x9 b Z+ j+ Z; m; g
2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
2 ^& C3 M+ @# b4 E0 ?* X Y q[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|changed` use `result is changed`. This feature will be removed in version 7 f# B$ ?1 C! W( Z! X7 l
2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.; ^" b( q, B6 ?$ K. j$ o
ok: [controller1 -> localhost]
( E5 h4 Z; y8 j V( l- sok: [controller2 -> localhost]
& S! [0 g7 _% dok: [controller3 -> localhost]5 f. S0 S2 ]4 o" `. u$ x& M
TASK [mariadb : Registering host from temp file] ******************************************************************************************************************: s0 n' z# } t4 u
ok: [controller1]- e2 {) F. s" q" f( e
ok: [controller2]4 x) u- R+ ?/ J7 Q( ]- W K
ok: [controller3]8 Q$ @ |5 S9 s- i) a: x; v" R; c% r
TASK [mariadb : Cleaning up temp file on localhost] ***************************************************************************************************************
9 `0 W6 N4 M- j; E+ _ok: [controller1 -> localhost]
8 w5 o0 {: [- O n% I$ MTASK [mariadb : include] ******************************************************************************************************************************************
* q0 G3 j" h* F" I5 {4 `; R/ a \skipping: [controller1]
5 f* O. _ n' k7 |skipping: [controller2]
& W6 b: ]& u; o3 y1 uskipping: [controller3]
( G0 s5 j+ E$ }. L" @& q! a- YTASK [mariadb : include] ******************************************************************************************************************************************
% A! N0 v2 X/ S: K) T: w/ vincluded: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/recover_cluster.yml for controller1, controller2, controller3
1 Y/ n! Z' t7 @' [5 bTASK [mariadb : fail] *********************************************************************************************************************************************
) O& @8 H8 `9 W" h: X0 oskipping: [controller1]
, a& h3 s: I$ y4 c* Dskipping: [controller2]! G* l0 Y2 G% d1 n9 U1 \
skipping: [controller3]
& A. x) e! {( p. STASK [mariadb : Cleaning up temp file on mariadb hosts] ***********************************************************************************************************( y6 q9 Q: D# D! y& e
ok: [controller2]% n' R9 ^1 u& C) A
ok: [controller1]% I* g$ x: O" p$ ~
ok: [controller3]
+ x9 t8 }9 d7 E9 YTASK [mariadb : Cleaning up temp file on localhost] ***************************************************************************************************************( d% l, ?2 \8 [' |! I
ok: [controller1 -> localhost]% u# H$ m/ U, f( g% |7 d
TASK [mariadb : Stop MariaDB containers] **************************************************************************************************************************
6 M! d" h: l7 M# @. h7 e3 uok: [controller2]
1 ]5 U" Z% u1 A% H6 D) [( ^7 uok: [controller1]7 B7 ]$ {, D# E9 D6 U) s7 u
ok: [controller3]
7 s/ ?! h0 m5 Z7 {5 C1 zTASK [mariadb : Run MariaDB wsrep recovery] ***********************************************************************************************************************' m3 l9 g% w! Y
changed: [controller1]
0 q, l" V- q5 z" |changed: [controller2]5 S5 @ j4 @ j+ Y: g
changed: [controller3]
+ q3 s. P; U7 D5 b2 KTASK [mariadb : Copying MariaDB log file to /tmp] *****************************************************************************************************************+ D8 W/ N+ `$ k7 @% g; R" m
changed: [controller3]+ ^4 b' \' l" z" j) x% N$ D$ T
changed: [controller2]
2 t. ^7 C# h' E4 T: W7 P" v8 O+ l2 Z7 achanged: [controller1]& @# \- \. P/ {; y9 H% ?- H# _
TASK [mariadb : Get MariaDB wsrep recovery seqno] *****************************************************************************************************************
+ l* `6 @8 p4 ~1 Fchanged: [controller2]
' S: R7 p( V+ @9 u* f, Kchanged: [controller1]
# U- \5 l% {7 ichanged: [controller3]1 T3 V9 x" D- e
TASK [mariadb : Removing MariaDB log file from /tmp] **************************************************************************************************************- h5 n& D& p# j- }) Y. E2 v
ok: [controller2]
" a( M( y; n9 z8 Y8 Hok: [controller1]
6 ]1 k: V+ s x' n/ {- Zok: [controller3]
, u' b7 L" h6 U8 \: I0 BTASK [mariadb : Registering MariaDB seqno variable] ***************************************************************************************************************. B+ U# b2 I7 `- Q1 R9 D8 A& O" O
ok: [controller1]
: P- c. w, n3 Xok: [controller2]3 u5 K |6 O Z* B% U, G' {! s
ok: [controller3]% y: M0 u, E6 s: P( n5 _1 j$ y
TASK [mariadb : Comparing seqno value on all mariadb hosts] *******************************************************************************************************& n- w3 e$ h2 n& V0 Q2 C, M9 z
ok: [controller2] => (item=controller1)8 m. G8 i {. E
ok: [controller1] => (item=controller1)' x& f2 o: l( S V7 F/ S7 S
ok: [controller3] => (item=controller1)% `* J- _) d8 J8 O/ \- Y) w5 x$ j
ok: [controller2] => (item=controller2)
% O& q) y7 y% E U: Jok: [controller3] => (item=controller2)
0 |1 v, [' x+ R" _; r( dok: [controller1] => (item=controller2)
4 C$ y( }2 `; E+ U F) \9 Wok: [controller2] => (item=controller3)# r. f% i L& ^: }. K
ok: [controller3] => (item=controller3)
8 V' s6 J2 ^' n w5 Wok: [controller1] => (item=controller3)! B# ~5 B/ c* l
TASK [mariadb : Writing hostname of host with the largest seqno to temp file] *************************************************************************************
! X4 d0 [) w. T% O7 S iskipping: [controller3]
: ~& r. e* B2 X9 N) w3 P% l) h, }ok: [controller1 -> localhost]/ ?3 X: Z7 v' l; K( f+ a- W
ok: [controller2 -> localhost]
6 o( k# ^, a' [( u' i& XTASK [mariadb : Registering mariadb_recover_inventory_name from temp file] ****************************************************************************************$ ~$ H' s- N9 a" c" A/ e' }% o
ok: [controller1]
7 b$ v' B# L" I. B0 ~. k+ e8 P* u f4 jok: [controller2]
( k# l5 P7 g$ u% E( t3 Q+ Tok: [controller3]3 q& d, Q0 P$ X. a; y
TASK [mariadb : set_fact] *****************************************************************************************************************************************) b" J# e3 j. e5 [8 ?7 c, ]" V
changed: [controller1]
; \8 X3 B5 M: l% X _2 k. C& K( kchanged: [controller2]7 l ^! v( p4 |" G# R
changed: [controller3]4 R" [7 I, W8 a' @
TASK [mariadb : Copying grastate.dat file from MariaDB container in bootstrap host] *******************************************************************************/ I* E* c! q" H
skipping: [controller1]5 x* K4 P& v4 l$ R! S
skipping: [controller3]
. f0 n F+ ^1 ?( P6 U6 G, [/ a- aok: [controller2]6 p; T0 O; S1 O7 K4 V
TASK [mariadb : Set grastate.dat file from MariaDB container in bootstrap host] ***********************************************************************************
+ P" w. @$ `1 [ a: D* O7 B% Lskipping: [controller1]
/ d3 ?2 }$ i$ g% m) ` Eskipping: [controller3]
. e) f. d) k0 r5 v' ~) g( |changed: [controller2]
' ~& d, h- Y( l1 U9 nTASK [mariadb : Copying grastate.dat file to mariadb container] *************************************************************************************************** P* }( d1 @+ |% F3 y
skipping: [controller1]1 K8 V3 Q) ~" |9 {( B' E* F9 \7 N
skipping: [controller3]
# g# E5 m- {! T- w& M9 iok: [controller2]& d+ [% v* E# x5 U4 C; B2 ~
TASK [mariadb : Starting first MariaDB container] *****************************************************************************************************************
3 a2 |# m. u" h( A, iskipping: [controller1]
G8 ^6 t& j, G1 o; G2 O! x; @skipping: [controller3]
* @6 G4 N" F+ h( k: e. Wchanged: [controller2]5 B# U+ n- h; @# S: L
TASK [mariadb : Wait for first MariaDB container] *****************************************************************************************************************7 j* d; v4 n; T4 d6 o. v' {
skipping: [controller1]( m3 {4 ` n! u& Y/ N
skipping: [controller3]
& i+ P1 `# l, O0 o, B4 Y[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|success` use `result is success`. This feature will be removed in version
6 @9 y7 C0 w0 C/ D2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
5 I- Y5 c' p& N% s6 k Q8 uFAILED - RETRYING: Wait for first MariaDB container (10 retries left).# V& O+ Q$ N8 m4 m
ok: [controller2]" r6 } L0 g6 _# U. d c
TASK [mariadb : Set first MariaDB container as primary] ***********************************************************************************************************" ~: c+ A c2 }" n7 y+ h3 n
skipping: [controller1]
' x1 {( f9 I. r2 Pskipping: [controller3]
. X7 \# W8 q( echanged: [controller2]. _6 V) o0 N3 {* o6 @% Q
TASK [mariadb : Wait for MariaDB to become operational] ***********************************************************************************************************
4 H5 ^0 j( D z6 Pskipping: [controller1]+ {4 P2 F* M) }" M+ L3 x
skipping: [controller3]7 v2 G1 i* T4 s! J+ e& o
ok: [controller2]- S7 u# ^6 [" C& ]* J! K9 b
TASK [mariadb : Restart slave MariaDB container] ******************************************************************************************************************
5 j2 D" a/ z7 ^. Q) e7 P3 L) pskipping: [controller2]
$ |* L3 r2 i$ t2 X) Nchanged: [controller3]( p# n9 N! L% S8 | Q
changed: [controller1]
, K) o6 Y1 P8 y/ kTASK [mariadb : Wait for slave MariaDB] ***************************************************************************************************************************- o: Y8 |# ~, V k" |; S% U, D
skipping: [controller2]
0 g) Y- S7 q6 S, c; i1 t$ I( m[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|success` use `result is success`. This feature will be removed in version
! f9 `- n- x9 p2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg./ z& u5 ^, l( J7 Q# F$ P
FAILED - RETRYING: Wait for slave MariaDB (10 retries left). l& g# ~1 ?6 Z% X
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|success` use `result is success`. This feature will be removed in version
0 D% R$ A* p, I* e, v2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.+ ]( T; N7 |# [4 C% f' Z& [4 s
FAILED - RETRYING: Wait for slave MariaDB (10 retries left).3 Q! g7 u8 M: b0 S9 m Y3 q
ok: [controller3]0 k _0 r% _, P% U
ok: [controller1]$ |/ E7 s$ y0 O. o6 e
[WARNING]: flush_handlers task does not support when conditional
# h6 t* a- ~4 J* y6 i3 M* ?0 X. x
RUNNING HANDLER [mariadb : restart slave mariadb] *****************************************************************************************************************- {. k' h# H( e1 P( ~
skipping: [controller3]
( N: ]5 h' j% o Wskipping: [controller1]
2 A0 N3 J7 {" E7 M/ Dskipping: [controller2]* z( }& ~3 L- J/ ?+ n, L
RUNNING HANDLER [mariadb : restart master mariadb] **************************************************************************************************************** g: Y" E) q$ O% C O/ b( w
skipping: [controller3]* Z6 @+ W1 ^" c
skipping: [controller1]
. Z5 f' Y F' iskipping: [controller2]& M8 q4 |/ }. f$ u0 l
TASK [mariadb : include] ******************************************************************************************************************************************
, t- @4 N3 A" s, R, u d9 kincluded: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/register.yml for controller1, controller2, controller3
8 N, p5 f$ O( ZTASK [mariadb : Creating haproxy mysql user] **********************************************************************************************************************
' ^- s3 [$ t( \/ x3 eok: [controller1]
, |- d* ^4 G0 }4 v% D- i* KTASK [mariadb : Cleaning up facts] ********************************************************************************************************************************3 u! y; Z' B5 @4 O0 y0 X8 _
ok: [controller1]
/ i9 q! ]2 m1 V* e8 y$ xok: [controller2]
* O0 @+ G; B' Pok: [controller3]
* _7 \ ~7 \7 ?TASK [mariadb : include] ******************************************************************************************************************************************
5 U# L6 x$ Q g- cincluded: /usr/share/kolla-ansible/ansible/roles/mariadb/tasks/check.yml for controller1, controller2, controller3
# h [ `- q5 O/ h6 l: DTASK [mariadb : Waiting for MariaDB service to be ready through VIP] **********************************************************************************************) G' M" v) g) V1 h3 g \/ D
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|success` use `result is success`. This feature will be removed in version 6 m! @1 n w1 S' }& N
2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.1 I& ~& j1 G3 Y& e3 m8 G- V: q" w
ok: [controller1]8 R0 ?2 F7 E. L0 L0 x6 Q
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|success` use `result is success`. This feature will be removed in version # M0 ~$ l6 E0 M+ B
2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.- ]8 y5 k) ~. i& J$ x+ e6 G! G* j
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|success` use `result is success`. This feature will be removed in version
# c' Z& N3 E& E3 T3 h! A2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.9 l8 g( _9 y- T) t; S" t6 h7 F1 @
ok: [controller2]* S$ L1 R) n3 J1 @3 R
ok: [controller3]
. z7 ]7 P- @4 Z3 h3 J' A1 [PLAY RECAP ********************************************************************************************************************************************************0 j+ r( }" I& f" m" b8 n1 k
controller1 : ok=37 changed=6 unreachable=0 failed=0 skipped=15 rescued=0 ignored=0
7 X8 }6 e) A, H8 h$ _. z. ~' Ycontroller2 : ok=37 changed=8 unreachable=0 failed=0 skipped=10 rescued=0 ignored=0 2 r& o! N7 O8 I
controller3 : ok=31 changed=6 unreachable=0 failed=0 skipped=16 rescued=0 ignored=0
9 n# g' O$ V' j5 p B* U) N' K$ u: n) L% D) b1 ?* a
恢复正常。
b1 ]/ r& ~2 V& v/ n8 j
& p; _! V* J6 q9 e1 ?5 Z8 k& J8 E$ r' M. [' K; O
有时候通过一些特殊手段可以帮助我们解决一些特别的问题，与其长时间尝试老方法，不如来点新的方式试试或许不一样的效果。2 o# g; ^3 o$ g& o- d/ P1 `
$ _7 L _" D# b: z8 C9 R
' W3 z+ @1 F+ v3 M我们首先是因为数据库恢复出问题，无法恢复数据，， 检查数据的库的值，
+ V( `9 y, m+ l X, i: acat /var/lib/docker/volumes/mariadb/_data/grastate.dat 中seqno值，若该值在所有节点中存在唯一得最大值2 a3 U2 F( I v, g2 ?% [
然后我们保留这个节点上数据，( T( y% I0 ]: Q0 ~/ C/ j
删除其他节点上的数据文件0 E/ i( T8 J- e [
然后通过scp或者rsync同步保留节点的数据文件 （根据个人喜好同步文件，本人建议使用rsync）
3 N% m! H- P! @4 i然后再执行kolla-ansible数据库mariadb_recovery恢复命令：
! Q1 U8 x0 ^( A) Q; W5 b
& {- d$ K: T7 n5 v& F+ E3 [, l) I7 Z
+ h2 I& k/ o9 Y$ u. Z/ v s
3 q* w- l x! A4 T7 g& z差不多就可以完成数据库集群恢复了。; W! G8 D- `; I+ C
|
|
发表于 2021-10-18 11:03:46