|
|
楼主 |
发表于 2023-12-22 11:54:34
|
显示全部楼层
crm options user hacluster
; r. F- |1 ^4 }7 _% PCOPY( s. V0 p( x! V) W% N/ S
Note that you need to set up /etc/sudoers so that sudo does not ask for a password.- V# Q+ A( d' F
" W' Q, i& i( u7.1 crmsh—Overview% ~, p' ~7 ~) }/ t" ^
1 R' P U5 g7 r1 R# {- g+ l4 GThe crm command has several subcommands which manage resources, CIBs, nodes, resource agents, and others. It offers a thorough help system with embedded examples. All examples follow a naming convention described in Appendix B.- G0 a# f! Z9 X, i, |# V# Z1 L
; K% @0 F2 p' [8 ^: R7 hTipTip: Interactive crm Prompt
. T: l( ^& z( l) Q# U# k# A9 GBy using crm without arguments (or with only one sublevel as argument), the crm shell enters the interactive mode. This mode is indicated by the following prompt:
3 V9 l: e0 X7 K; a& `/ h7 E8 @2 C/ }$ U3 y& z2 _
COPY) f+ k5 z8 c7 n; E
For readability reasons, we omit the host name in the interactive crm prompts in our documentation. We only include the host name if you need to run the interactive shell on a specific node, like alice for example:; Q3 H F3 e5 m- Q3 H; x; K8 L
3 y. B; h) K1 z0 j" ~
COPY
+ J8 }& p3 m/ n7.1.1 Getting Help8 |; Z4 a: C$ c) `
" q5 ]1 O3 B. N& V6 HHelp can be accessed in several ways:+ O, p# c9 m) g7 ]9 m6 q
- S; I" j8 S ?. r) w- i6 xTo output the usage of crm and its command line options:
6 f, ~/ m" z6 s/ E( S
7 J! Y$ t, |8 H' Ecrm --help1 e, c; b: r; X; N
COPY
) \7 h- S5 |+ {/ \/ VTo give a list of all available commands:: r& j) \( X1 x8 F3 ?6 V
y( @- s; a: I" f) i
crm help$ s. Y3 ]. x. |# O% x8 o
COPY9 `* A& n8 |9 M6 w% R. B
To access other help sections, not just the command reference:
: N( `) S: C0 d& I+ @( L3 K) Z D" s! p8 c$ x2 x& W ^6 r2 A- O: j2 c
crm help topics! z* T/ \. N, f0 _% M2 L- m
COPY# t5 Z/ B1 m; ]( e2 f$ W5 H( V
To view the extensive help text of the configure subcommand:
* p: i" Y7 ?1 c2 j a3 t4 P3 g# W: }" o$ B. D" ], R
crm configure help- g. z9 V6 _+ D0 t+ s
COPY
. _# |. T# b. ~0 M( VTo print the syntax, its usage, and examples of the group subcommand of configure:) x! S: o! Y: d
( r& W6 ^4 C4 K1 h. H
crm configure help group
7 P; P/ N# s" G* F$ I* V6 hCOPY: B3 t0 B! M5 f. b
This is the same:
9 `3 j Z5 d+ }5 x$ z$ Z' Y+ j7 b
5 }" X6 Z0 d( f$ o5 h$ Zcrm help configure group
3 D, p0 Z8 T4 ]8 j2 g& f! DCOPY
; m! {( T) w1 t# z% dAlmost all output of the help subcommand (do not mix it up with the --help option) opens a text viewer. This text viewer allows you to scroll up or down and read the help text more comfortably. To leave the text viewer, press the Q key.- n) v: h# y/ ^9 M$ w6 p9 o# r
+ W/ T6 _2 V! B+ |4 `+ P* STipTip: Use Tab Completion in Bash and Interactive Shell
6 u# I; P8 V! q/ Z+ ^8 o) eThe crmsh supports full tab completion in Bash directly, not only for the interactive shell. For example, typing crm help config→| will complete the word like in the interactive shell.. T7 d- u4 b2 `/ Z6 E6 m: m
( J* e* ~0 H3 y' L
7.1.2 Executing crmsh's Subcommands6 W; F! m' `: N3 L, q5 [- j9 S1 P3 h
/ \: n& ]4 p; N; |8 E* l
The crm command itself can be used in the following ways:1 c+ }4 W+ M6 N# ^6 e
; p v) R4 N3 Q0 s. h5 ?1 v3 L
Directly: Concatenate all subcommands to crm, press Enter and you see the output immediately. For example, enter crm help ra to get information about the ra subcommand (resource agents).& x' q* |) m. Y
. a3 F6 `" n3 I# _3 U
It is possible to abbreviate subcommands as long as they are unique. For example, you can shorten status as st and crmsh will know what you have meant.
^# ^6 L1 |' [3 t; ?; O" x' u
4 U" J* D+ e* B3 o, C! BAnother feature is to shorten parameters. Usually, you add parameters through the params keyword. You can leave out the params section if it is the first and only section. For example, this line:( Q5 r- Y3 D% H$ b. w
) T }. K9 Y. j9 W
crm primitive ipaddr IPaddr2 params ip=192.168.0.55
6 Q6 N5 @& w$ C9 D; x) n# KCOPY% O4 ?% |4 w3 _
is equivalent to this line:1 _+ z; i( g3 D2 m' `
; d0 x( z* O9 |crm primitive ipaddr IPaddr2 ip=192.168.0.552 _6 [) G* J9 D$ b. T
COPY0 A$ S) h1 n g# o
As crm Shell Script: Crm shell scripts contain subcommands of crm. For more information, see Section 7.1.4, “Using crmsh's Shell Scripts”." H; v" {% z" ]! \6 f
: d& `$ K5 H4 N) w6 N* UAs crmsh Cluster Scripts:These are a collection of metadata, references to RPM packages, configuration files, and crmsh subcommands bundled under a single, yet descriptive name. They are managed through the crm script command.8 S/ H% T% w( b: T
" `; S$ V3 d0 V: T
Do not confuse them with crmsh shell scripts: although both share some common objectives, the crm shell scripts only contain subcommands whereas cluster scripts incorporate much more than a simple enumeration of commands. For more information, see Section 7.1.5, “Using crmsh's Cluster Scripts”.
7 o5 K! {* r* g2 `$ T! U
$ f% L+ x) ]2 A6 ^Interactive as Internal Shell: Type crm to enter the internal shell. The prompt changes to crm(live). With help you can get an overview of the available subcommands. As the internal shell has different levels of subcommands, you can “enter” one by typing this subcommand and press Enter.- W8 D! R t$ F
3 I* A/ F1 t# c
For example, if you type resource you enter the resource management level. Your prompt changes to crm(live)resource#. If you want to leave the internal shell, use the commands quit, bye, or exit. If you need to go one level back, use back, up, end, or cd.
1 N4 y) l! B1 A0 @3 E1 T
" t4 s3 t M4 J7 t& `5 eYou can enter the level directly by typing crm and the respective subcommand(s) without any options and press Enter.
- z/ q ^# |9 v1 W; H* E. F' o! r% l3 |2 I8 C% \/ ?) k
The internal shell supports also tab completion for subcommands and resources. Type the beginning of a command, press →| and crm completes the respective object.# U# S3 Q- C& f
- J8 E+ N# O* @2 K
In addition to previously explained methods, crmsh also supports synchronous command execution. Use the -w option to activate it. If you have started crm without -w, you can enable it later with the user preference's wait set to yes (options wait yes). If this option is enabled, crm waits until the transition is finished. Whenever a transaction is started, dots are printed to indicate progress. Synchronous command execution is only applicable for commands like resource start.' v9 I* S- K, k& {1 t: x# |' I( P
5 A1 f2 Q, h3 ZNoteNote: Differentiate Between Management and Configuration Subcommands
. T) K( j5 H4 F& n+ m# k9 u( v: o- DThe crm tool has management capability (the subcommands resource and node) and can be used for configuration (cib, configure).; n* _1 N+ N- o
. g$ n2 j; g( M& I3 \3 {# jThe following subsections give you an overview of some important aspects of the crm tool.
& i$ z U% l1 f) t
# h6 D7 I2 }; d; h; s" q7.1.3 Displaying Information about OCF Resource Agents: @* n4 p4 |) V
( L/ S2 v5 H4 }! F) JAs you need to deal with resource agents in your cluster configuration all the time, the crm tool contains the ra command. Use it to show information about resource agents and to manage them (for additional information, see also Section 5.3.2, “Supported Resource Agent Classes”):( ?+ t9 `/ f. E3 h
- `# a2 i: v e% gcrm ra
7 C, d8 [$ S) `$ O4 jCOPY
# P/ O# V% a1 zThe command classes lists all classes and providers:
6 X; P! C3 z# V8 @- H L3 L4 F2 r7 A3 i. r+ q
classes
3 D/ X9 k- e! r- n: h5 s# h3 k lsb3 r5 j; N* r4 l1 V* j
ocf / heartbeat linbit lvm2 ocfs2 pacemaker
, R9 a3 d/ n, A" _ service3 {, H9 t0 M3 C) |* i" ?; J
stonith; Q& ~* V$ O2 s
systemd
9 a w3 v# |4 _9 e+ ]- I5 sCOPY
5 F" p1 g N% {" V+ HTo get an overview of all available resource agents for a class (and provider) use the list command:2 p. n. D6 A" O! D$ {
% j: E8 f ]4 d2 H% }list ocf
% A4 W2 [! K! [' ]( s* Q5 PAoEtarget AudibleAlarm CTDB ClusterMon+ Q' T1 `1 e8 Q: z5 d k
Delay Dummy EvmsSCC Evmsd
( O1 V" `- q! l1 }4 o0 y4 @Filesystem HealthCPU HealthSMART ICP
- i+ A: U1 c/ I; z4 a: C0 nIPaddr IPaddr2 IPsrcaddr IPv6addr6 U5 ~1 B/ O9 p" n6 P& b& i* c
LVM LinuxSCSI MailTo ManageRAID
4 @2 X' I3 `6 p& q6 [& ^ManageVE Pure-FTPd Raid1 Route9 j( Z% n$ C/ C* a
SAPDatabase SAPInstance SendArp ServeRAID9 R; S3 }7 y" n# k' v: s0 X! W
...4 ]: j# e" D. o; F0 m; i! D& Y8 h% i5 Z
COPY
. T3 @9 k, Y1 @5 }4 @An overview of a resource agent can be viewed with info:+ f# @) }1 a" k$ U' l( N
+ E# v/ _5 m4 c$ B0 G( ainfo ocf:linbit:drbd. m# d% [# x3 @! L& n
This resource agent manages a DRBD* resource# u. S8 {8 v% ^9 I0 f5 Y5 z; O
as a master/slave resource. DRBD is a shared-nothing replicated storage
0 j. g C* A4 V1 Ddevice. (ocf:linbit:drbd)
' O% X! O7 ]$ Q( \0 k2 L n3 P3 {/ Y$ ?" f: N; B' M* c! E+ X% Y7 {
Master/Slave OCF Resource Agent for DRBD: W* V v3 y, M6 b) s1 m9 Q
4 k6 }- n$ T; }Parameters (* denotes required, [] the default):
: t/ N+ F# |/ h/ s0 a2 k3 [& M+ v& L6 J: A, }. N7 f" E
drbd_resource* (string): drbd resource name
+ e' d+ F3 s0 `% W8 Y0 ` The name of the drbd resource from the drbd.conf file.& z3 k! s0 P3 n# J3 w
w! o4 m- Z f! p( z5 v0 d
drbdconf (string, [/etc/drbd.conf]): Path to drbd.conf* q9 |8 @3 a- K6 p. `
Full path to the drbd.conf file.; ? a/ a# e- O, G
0 U! W3 v7 U% i. k& IOperations' defaults (advisory minimum):8 f% O" U7 \1 L7 Y2 ?
1 S. r* ]9 O0 v' y5 y start timeout=240
8 r4 d& S: h2 }1 T6 D: M, |( | promote timeout=90: M1 v# A/ A: |; M1 g
demote timeout=90/ q& g' C, S- w$ `0 y% p/ X
notify timeout=90$ w- O/ m8 l: X
stop timeout=100' ^, |, x D( D8 v1 Q: U
monitor_Slave_0 interval=20 timeout=20 start-delay=1m
" H! S3 |' Y' ?6 s( ? monitor_Master_0 interval=10 timeout=20 start-delay=1m
* Z9 I T! n) QCOPY. o/ Q* u5 t) E5 ^) |6 x
Leave the viewer by pressing Q.
5 R+ Z1 w) x$ Q5 U7 j. B! ]
+ X& o8 S/ l' t8 u+ [TipTip: Use crm Directly+ l {" Q' a* q: i6 ]# J
In the former example we used the internal shell of the crm command. However, you do not necessarily need to use it. You get the same results if you add the respective subcommands to crm. For example, you can list all the OCF resource agents by entering crm ra list ocf in your shell.
& Z( L% Z9 F: M7 C. x$ {$ t* T9 |- u% w2 T' b! B, M3 G
7.1.4 Using crmsh's Shell Scripts
. K' P& n* Q& A- ?# p2 H% c5 B5 z' M( L; A3 B& V0 b
The crmsh shell scripts provide a convenient way to enumerate crmsh subcommands into a file. This makes it easy to comment specific lines or to replay them later. Keep in mind that a crmsh shell script can contain only crmsh subcommands. Any other commands are not allowed.
_1 N9 F/ {# c2 B, w ]7 I
* J3 e( W k7 OBefore you can use a crmsh shell script, create a file with specific commands. For example, the following file prints the status of the cluster and gives a list of all nodes:3 ~1 {1 X* p4 F6 ~. e
. ]' U4 q9 m8 _+ mEXAMPLE 7.1: A SIMPLE CRMSH SHELL SCRIPT
3 H1 K6 Z- z2 j2 F5 q
# U4 x- H" x# W$ T0 v# A small example file with some crm subcommands
$ p$ s4 O7 {+ k+ |% {( nstatus6 y# U% z5 ]7 @$ L
node list
" r6 U8 K) s" Z1 m8 [- A; G9 C0 LCOPY
# I2 U* M) H: t; c$ z2 w" fAny line starting with the hash symbol (#) is a comment and is ignored. If a line is too long, insert a backslash (\) at the end and continue in the next line. It is recommended to indent lines that belong to a certain subcommand to improve readability.8 S) |3 D' X" M( y/ n% c
. S( \: F/ N& `2 J* @4 H5 KTo use this script, use one of the following methods:; u- u8 |6 ^4 f7 l# K
0 A8 p" p/ k @& T
crm -f example.cli
5 y7 f$ n8 m# R+ a, Y, O; m2 o. h1 vcrm < example.cli7 M& `+ ?7 D3 }3 @
COPY
+ _8 |) g9 m) ^5 Z: x, [7.1.5 Using crmsh's Cluster Scripts1 N9 z/ J, n) L; ?% P v; l
) T9 M! w5 N( F; FCollecting information from all cluster nodes and deploying any changes is a key cluster administration task. Instead of performing the same procedures manually on different nodes (which is error-prone), you can use the crmsh cluster scripts.3 g( ^- I; g( |$ I# e B, w
. t: E7 _# A I9 q- ]7 o+ [8 MDo not confuse them with the crmsh shell scripts, which are explained in Section 7.1.4, “Using crmsh's Shell Scripts”.
2 @2 ~' p, u5 m
: T" L- N8 Z- N5 K9 q" DIn contrast to crmsh shell scripts, cluster scripts performs additional tasks like:0 f3 K5 k4 u# l5 X8 b0 @" ~
# h# ?- ~2 \* y4 j7 \
Installing software that is required for a specific task.
& ]1 d0 b6 y- A4 {& A8 X; }$ X, [7 T9 f6 F4 \# p! M! |. Q; k- _
Creating or modifying any configuration files.
) y' i: F- z) J1 A/ x7 m' y G" X6 f% ?# B9 l
Collecting information and reporting potential problems with the cluster.
+ I; p$ K7 Q( N7 s9 D
8 N5 U, M& |+ D) xDeploying the changes to all nodes.0 c; e5 a1 d* g7 W& n
- j( j* A3 }0 s. K& d* o: e9 `6 j0 E
crmsh cluster scripts do not replace other tools for managing clusters—they provide an integrated way to perform the above tasks across the cluster. Find detailed information at http://crmsh.github.io/scripts/.
: B1 g( c( n. f+ B3 v. R
s2 Z) b9 ?6 v% H5 Y" b* B7.1.5.1 Usage
' a6 m* B% U. H& u" H! A+ a9 I9 f) y& p' ]' l, v$ g, Z
To get a list of all available cluster scripts, run:
! D* o& t2 G# M4 t6 C9 ?) t( X+ i" |' a( g- o2 }) s ^9 Q+ V
crm script list
: y, A. Y H/ W8 l, lCOPY
5 J! g t( m+ e" d+ `To view the components of a script, use the show command and the name of the cluster script, for example:
/ E) B. \! S7 c3 F& W$ n
4 l1 O* V$ r$ ~* J" Qcrm script show mailto$ ` B+ Z$ _/ w2 B
mailto (Basic)
; z+ |2 Q& H; yMailTo8 N% Y2 C+ x- M) `, n* k
/ s: k6 x c' S& A This is a resource agent for MailTo. It sends email to a sysadmin
/ z; V1 c$ }4 |% p; G$ Xwhenever a takeover occurs.
# h8 R( ?8 g8 D0 N" Y5 h% i! z, }5 _ p
1. Notifies recipients by email in the event of resource takeover. X q% A" y$ ~, Z1 s8 g% n. q! `
) n1 W- E) v+ n. _
id (required) (unique): v3 {' k3 e1 ^4 ?6 \0 A" ]) x0 D1 }
Identifier for the cluster resource
2 r5 R/ f7 j: ]( i2 o' X email (required)7 x6 ^4 Z2 _. j
Email address4 g9 J$ f/ p; C9 D) e% H9 W4 U, N
subject
1 J5 d) x3 D/ o% w% | Subject
+ V# m8 T* ~8 R' q( u: n& E7 dCOPY
/ h7 h% j) u; Q, P- c; [ QThe output of show contains a title, a short description, and a procedure. Each procedure is divided into a series of steps, performed in the given order.
$ ]6 o1 ]) r$ @3 j& @& N7 [4 G; c M* T1 U
Each step contains a list of required and optional parameters, along with a short description and its default value. e5 y- c( M* S( K6 I
- O5 c$ H1 |/ d9 b$ i8 u* b
Each cluster script understands a set of common parameters. These parameters can be passed to any script:6 a4 _2 W+ q% m9 Z- ]. e
$ g+ S4 `0 |! p' j$ STABLE 7.1: COMMON PARAMETERS- E; @2 L6 B; n2 F3 J5 Y' r7 ^6 e/ Q& n/ b
* |4 o/ ^2 ~; P/ g# x9 h7 R6 B2 \1 `4 W
Parameter Argument Description8 G* a8 M7 n' E" q
action INDEX If set, only execute a single action (index, as returned by verify)
9 u# W# D% ~% J+ i0 ydry_run BOOL If set, simulate execution only (default: no)
$ G6 Q2 u B% f9 J* ]+ ?nodes LIST List of nodes to execute the script for* i- M3 [0 w0 g/ v* `% v
port NUMBER Port to connect to! y( v' W8 B0 E
statefile FILE When single-stepping, the state is saved in the given file
' T0 }# t: ^2 _7 osudo BOOL If set, crm will prompt for a sudo password and use sudo where appropriate (default: no)' r! @- [1 ]/ d9 A
timeout NUMBER Execution timeout in seconds (default: 600)
3 s' o+ Y$ n; w6 Q( a4 }- `user USER Run script as the given user5 M& c4 O& [! H* I0 [9 N
7.1.5.2 Verifying and Running a Cluster Script
- b, ^# J; U# G* E5 h8 y4 c, c" }" \. F2 J- J# g1 I1 d" Y! @
Before running a cluster script, review the actions that it will perform and verify its parameters to avoid problems. A cluster script can potentially perform a series of actions and may fail for various reasons. Thus, verifying your parameters before running it helps to avoid problems., E* C) P( t% g/ V6 e/ g
O5 R- f5 z7 Q* A2 X, H! g, u; C9 J) xFor example, the mailto resource agent requires a unique identifier and an e-mail address. To verify these parameters, run:+ x7 [5 Y8 n9 ]8 a& r" W
3 v* q, e, _3 W: S4 _
crm script verify mailto id=sysadmin email=tux@example.org' q; S0 T8 b7 y
1. Ensure mail package is installed* o7 j1 K; E( c8 k0 p
" R- _ K3 f! B mailx i$ f$ ]7 a8 S
- C, q4 l) Z2 A+ m* p
2. Configure cluster resources0 T" O* X4 o! ?( e7 {1 k5 \
$ t1 d5 W* G& e' t7 I- B! P4 I5 c& v
primitive sysadmin MailTo
6 I& i& n- S* ~7 P' ? m email="tux@example.org"
; ?+ E& S: k2 I2 t op start timeout="10"
( F: P- x) _$ f! V op stop timeout="10"
2 ~* x. N8 E, A, e op monitor interval="10" timeout="10"
: h! `: j8 L# Z$ k9 f" M- X# b1 [4 Z
clone c-sysadmin sysadmin
. o5 c1 n0 O+ N8 @COPY- B# F8 E K2 Z. Y( u
The verify prints the steps and replaces any placeholders with your given parameters. If verify finds any problems, it will report it. If everything is ok, replace the verify command with run:3 N" B' i2 G Z- `# D8 F
6 s7 p1 Y# ] t& X6 mcrm script run mailto id=sysadmin email=tux@example.org/ E. L3 j7 P5 W5 d
INFO: MailTo
% _0 ~ b4 T$ Z! ?' {INFO: Nodes: alice, bob
/ u/ K t p3 o5 E0 B D) Z. d) hOK: Ensure mail package is installed
& Q* ?" ~0 w" MOK: Configure cluster resources
) j% k: I3 V/ mCOPY
9 T+ q }8 c3 Y; j7 x9 s! L& I: JCheck whether your resource is integrated into your cluster with crm status:" }5 W/ m" {% |" h: y5 `
0 Z( O8 v. a/ O2 }( r( p
crm status B& {' k6 X/ ^! a/ q3 Z9 i
[...]
7 r2 Y0 v( X+ G3 F/ D Clone Set: c-sysadmin [sysadmin]& D3 {: j1 {7 k0 s( a
Started: [ alice bob ]6 y' m8 f* H) \! k0 V/ ?
COPY4 N% { o3 E0 y- O+ f# [' N3 u) Q, O
7.1.6 Using Configuration Templates* L9 g2 @. ^+ s- j( j+ W7 w
* Q! s9 T. B0 b& i; H
NoteNote: Deprecation Notice
0 n+ s3 K: b; J9 t8 sThe use of configuration templates is deprecated and will be removed in the future. Configuration templates will be replaced by cluster scripts, see Section 7.1.5, “Using crmsh's Cluster Scripts”.
6 u) I# f5 u5 c* F: g; m
w/ ]# Z) A6 _1 r2 A b' K3 eConfiguration templates are ready-made cluster configurations for crmsh. Do not confuse them with the resource templates (as described in Section 7.4.3, “Creating Resource Templates”). Those are templates for the cluster and not for the crm shell.
; y) Z! ]2 T; X6 ~. F7 [3 n- |5 T# h9 ?$ k: v# Q. O" [
Configuration templates require minimum effort to be tailored to the particular user's needs. Whenever a template creates a configuration, warning messages give hints which can be edited later for further customization.
8 f4 N+ G# g( q* A8 `. v$ a' X/ Q9 b6 z* C, k
The following procedure shows how to create a simple yet functional Apache configuration:. N* j4 o8 H! e0 V8 ~1 k
: I+ q; q0 L' Z$ o$ S) W
Log in as root and start the crm interactive shell:
6 q! ~- Z% g" b% B( K
* v* L7 y& J- E6 E& k0 dcrm configure5 [5 T$ J! v& ?" T& v
COPY
4 F1 k- s* g) H" I: c& z k* n" ECreate a new configuration from a configuration template:1 {2 S1 Q5 V! ]8 \- B! S
8 C* s1 B. P! S" ~: u2 q! A8 P
Switch to the template subcommand:, M2 r% X* h o1 Q: c1 l
# n5 A1 D: j, I1 e& ?, M
template
. {9 _, }' Q1 `5 f% R; wCOPY1 k- o) q4 Y& x% \
List the available configuration templates:; x Q7 I/ O! V; F h
! ~% A% ?5 @. Z1 }list templates
. h, v4 ^7 ~/ k, x3 G2 A$ Qgfs2-base filesystem virtual-ip apache clvm ocfs2 gfs2
2 B: S2 q* K+ \/ Q- u ACOPY
8 H- w8 e1 e8 _Decide which configuration template you need. As we need an Apache configuration, we select the apache template and name it g-intranet:8 r3 s a" x& |
6 W% S& Z n5 o
new g-intranet apache
% V6 T9 C1 S, j' m: B. JINFO: pulling in template apache
$ H- x5 w9 o' h- a' M8 FINFO: pulling in template virtual-ip
+ S6 R; T& v2 B/ q& j6 ~$ Z- gCOPY
5 g, x8 O: B2 X/ I# t0 }/ _9 zDefine your parameters:1 P( W3 V! h: F$ `* B5 f
1 ]" S q7 Z9 H4 B F p9 k q
List the configuration you have created:
) Z6 x( ~7 `* T3 Z# j; i
, X2 O! P3 O: D& Dlist
, ~9 [& }6 Y2 Ug-intranet
5 V+ |& U R- R7 l8 p2 G3 uCOPY8 O( ]' c- U, C4 |$ p# x, F8 Z1 v! m
Display the minimum required changes that need to be filled out by you:# n) m) t1 W3 [9 i8 X9 [
4 O, O0 p/ b2 ?show
/ I6 n; ?: e; N( d4 E' KERROR: 23: required parameter ip not set
/ V' H; B* V. [ oERROR: 61: required parameter id not set
( P" P5 n" I4 k9 ~0 {' }ERROR: 65: required parameter configfile not set6 ^: d; S% r. n' D0 ]7 s/ z4 Q, s
COPY
% X" J) z7 b% `5 k4 pInvoke your preferred text editor and fill out all lines that have been displayed as errors in Step 3.b:
2 i. o. T$ _ L
6 N/ m! i8 g% }! kedit6 k3 Y3 N+ O" N- {* T5 R: a
COPY
% h9 U9 q* { `7 Q3 ?8 f3 o8 {% FShow the configuration and check whether it is valid (bold text depends on the configuration you have entered in Step 3.c):' Q6 X0 O& z2 N& ~/ v* ?$ E3 Y: M
! O8 A. N; M' W7 }( B
show
! ]9 I4 o: S8 `2 b& h1 x9 O" T5 gprimitive virtual-ip ocf:heartbeat:IPaddr \
/ }. z* y9 O* \ params ip="192.168.1.101"
4 b* f, s. t/ f3 |primitive apache apache \$ @4 j7 N- j/ L/ G1 h' M
params configfile="/etc/apache2/httpd.conf"
& x8 N# i% T$ {7 h6 V monitor apache 120s:60s
/ W3 i ]1 C+ Z5 i0 x0 T2 v+ tgroup g-intranet \( S& G, M8 J8 A5 ?/ U
apache virtual-ip% F) p6 U& M% S% H" d
COPY3 l0 ]( ~, q9 z
Apply the configuration:! c7 l# h5 J6 w! d8 q& i8 ^+ _- \
/ e9 Z. T. f/ v( N$ Q* B o
apply
0 v5 D5 I$ u8 D0 b; C7 acd ..
& I& [# w. R( j# \# n, x3 Yshow
8 o: a5 B; `/ A- n5 @COPY
( X6 b/ f) v y& X; D$ ASubmit your changes to the CIB:7 k0 _# `2 \/ @5 I# W( `
M5 s$ R* J0 r' C0 Y
commit2 q4 r& j; ^: n4 K! m
COPY
. @8 |+ c' z; @/ e d8 @It is possible to simplify the commands even more, if you know the details. The above procedure can be summarized with the following command on the shell:
- B( m# `* R* M' D- W$ Z1 k8 U- `+ M
, c( p- z$ P; x! d0 ccrm configure template \
9 [% B, U4 G* G) p1 B5 u5 R new g-intranet apache params \
: {- `/ E, L1 R$ z2 } configfile="/etc/apache2/httpd.conf" ip="192.168.1.101"5 J- i8 q' r/ p+ S! E5 q" R
COPY8 ^5 z/ F# O' W) u" j: G- r
If you are inside your internal crm shell, use the following command:
4 H- l; T3 C3 M( e! K9 | O4 g/ P& q+ P# H
new intranet apache params \
# a7 b, M8 c$ S6 ]$ L3 B! H/ S8 Q) _ configfile="/etc/apache2/httpd.conf" ip="192.168.1.101"
! Q+ O% f2 H1 [1 {COPY. K# V! i: A7 f7 C( ]9 g' t
However, the previous command only creates its configuration from the configuration template. It does not apply nor commit it to the CIB. f$ x1 ?7 e- O9 s& {2 y
! N, X' i) E# G0 j/ ?' h: B$ j7.1.7 Testing with Shadow Configuration- Y5 z( ]. q+ r+ \8 d+ `4 T9 O
5 S I4 c1 { }/ p E" D
A shadow configuration is used to test different configuration scenarios. If you have created several shadow configurations, you can test them one by one to see the effects of your changes.
/ e4 M/ l& N- G9 ?; S
D; S4 w- I% C- \The usual process looks like this:
% }) j+ k" s g( q; m6 U6 o4 H
( J: E, O$ [1 s, i! ~; P' {5 z, fLog in as root and start the crm interactive shell:# e& I M- u' x& |
# E' ^# C8 x& C- E* z6 @crm configure
- H* i4 z- f+ i/ FCOPY
4 ^' l7 Z( R/ @Create a new shadow configuration:
. Z/ z2 ^( H2 @+ ?* n- k; h5 a) M1 R2 Y9 [- J5 ^8 T, p
cib new myNewConfig
* {- C F% D. c3 oINFO: myNewConfig shadow CIB created% g$ u' t( M0 P# D/ b2 v2 M$ a
COPY
" A% F. \- K$ r k! D" K) F6 uIf you omit the name of the shadow CIB, a temporary name @tmp@ is created.
S2 J6 j- s, h% @" U U( |3 ?/ M$ [; a2 @
If you want to copy the current live configuration into your shadow configuration, use the following command, otherwise skip this step:
( ~' e3 @! L! i5 B( \' _9 H f {, ]% Q
crm(myNewConfig)# cib reset myNewConfig
D0 R* i, U- N! o$ x. B2 i ^COPY
# w! K. q1 X5 A! F' c" N% j, R6 iThe previous command makes it easier to modify any existing resources later.
- k3 h. [& h& P8 u) W0 S, h+ y
( P* g* k$ S4 M' PMake your changes as usual. After you have created the shadow configuration, all changes go there. To save all your changes, use the following command:
& T# |" W" s/ S3 K8 e8 y6 l; G# j$ O( a5 i3 z! w2 d
crm(myNewConfig)# commit
/ ~+ U2 A2 H/ J2 s! I6 O2 I* ECOPY" W- `- N$ Y$ h: P
If you need the live cluster configuration again, switch back with the following command:
0 v, v( K: [2 Y: t% w3 D) j5 \- n- k/ S) k" ~( ]6 {4 Q, P
crm(myNewConfig)configure# cib use live
$ ]7 _5 E* [1 l/ _COPY) e! Y! c/ U! ?8 s) r/ K) p
7.1.8 Debugging Your Configuration Changes6 l' A3 H( E& N N
, b; Z2 N& {% n' p) JBefore loading your configuration changes back into the cluster, it is recommended to review your changes with ptest. The ptest command can show a diagram of actions that will be induced by committing the changes. You need the graphviz package to display the diagrams. The following example is a transcript, adding a monitor operation:6 k" y% K7 s& J* k" a m/ h
% T! \1 \7 d0 M& C( |$ B$ V& y; J
crm configure8 I# [- S2 J3 n8 }
show fence-bob
! [/ G+ F. v* S. q. r! Rprimitive fence-bob stonith:apcsmart \
8 e& t* d' {2 W- l! n- h params hostlist="bob"
, M% b+ \) G7 N5 A6 Umonitor fence-bob 120m:60s
8 R+ y6 c5 [5 \% a0 D& D+ Lshow changed
^' X* [2 a; eprimitive fence-bob stonith:apcsmart \/ a. G7 x* O5 i8 Y+ {
params hostlist="bob" \
`. x6 ]: v* K" b( P3 |4 | op monitor interval="120m" timeout="60s"
# { h; \) \0 f" M% Jptest' q, J' {0 E" Q" A E: M
commit
# z# I* r* B2 T2 O6 KCOPY
! a! B* l# J4 d7.1.9 Cluster Diagram" q% Z) r A( T; L3 z% C8 Q, s4 K
+ {: f1 N0 i! BTo output a cluster diagram, use the command crm configure graph. It displays the current configuration on its current window, therefore requiring X11.
$ T* u% ]7 J n J6 X, U- i% e, u' i7 g8 Z
If you prefer Scalable Vector Graphics (SVG), use the following command:
4 { N. ]$ [5 T+ z. O/ B, s% k' Y ?( D' d( N5 @6 P
crm configure graph dot config.svg svg9 k- v% r/ M& E/ T2 ?6 R
COPY& Y9 k$ X1 g; \0 t5 s0 L
7.2 Managing Corosync Configuration
2 `' U& [" V' W
/ ~2 j. w$ ?4 w9 g$ M! }& }Corosync is the underlying messaging layer for most HA clusters. The corosync subcommand provides commands for editing and managing the Corosync configuration.
4 j# b0 [8 G: G% U/ J% G
' r6 P2 R2 T' u9 ^For example, to list the status of the cluster, use status:9 ` i" r( e/ d6 p* H
8 u9 t) Q# b! Z$ ]( ^/ B
crm corosync status
; R- |: D& |) c; f5 vPrinting ring status.
$ f0 _9 j4 \( K" {Local node ID 175704363
- G4 U4 i; c4 k* X9 @7 yRING ID 0
6 J( h/ a, E- U. Z" h4 o1 v" D id = 10.121.9.43
2 K. L3 @* O9 f4 f status = ring 0 active with no faults) [6 `* ?& ~; k' P" N+ o
Quorum information3 f: o3 k0 Y7 ?" ?0 i! W: @$ K* o6 e
------------------
. w; N7 n( H# g i* y% vDate: Thu May 8 16:41:56 2014! I- Q6 P- ?& J5 G/ X
Quorum provider: corosync_votequorum' X, x. A8 Y4 `
Nodes: 2
. C; v- F( M6 H3 R/ H# P, }Node ID: 175704363- T4 M- R* r2 ?/ I6 r! a4 o
Ring ID: 4032
$ ?- F5 t q- C- w' V% uQuorate: Yes
# Q% j- ^; K6 z/ F, x: I
8 y: {* L9 V" IVotequorum information
1 Z9 Y. ] V4 M. r \ h6 y/ t R----------------------3 F1 @. g3 g1 C8 \" w% c! y9 V u1 [
Expected votes: 24 J' M+ |& I9 [( t: y* U, Z
Highest expected: 2
/ D: `6 D) C3 |* ?* U3 LTotal votes: 2
' |& k& i: X! A7 pQuorum: 2
4 D% I4 Q. A/ D: ]1 QFlags: Quorate
9 _( a0 n& u+ l7 ?# ~9 B
8 A3 `" f7 S, wMembership information# `& {& m E$ V. n2 @
----------------------
9 Q0 m! \7 o) ^" a. T Nodeid Votes Name9 E9 ^; k7 J- k' R( x: ^
175704363 1 alice.example.com (local)9 R$ c$ H& c+ Y3 _$ n
175704619 1 bob.example.com
% M' o K8 b/ L2 L; N6 \- ?COPY
" ~4 S/ L! H! E, b: d6 yThe diff command is very helpful: It compares the Corosync configuration on all nodes (if not stated otherwise) and prints the difference between:; j& Z- ?; C% `/ ]4 O
0 M$ c' F9 K* n6 s: e! F9 l
crm corosync diff( f3 \( N# E) i2 B, C
--- bob4 n4 f2 @3 B, D- | k
+++ alice
, {9 e1 C( [. Q( Q@@ -46,2 +46,2 @@
; N3 o3 C7 [3 @8 s; V3 |) i$ P/ b; l- expected_votes: 24 t0 j" B1 _2 @5 w* a
- two_node: 16 ] \# ]4 X4 @3 O% }- G
+ expected_votes: 1
! v) g3 P- L8 k' F+ two_node: 0; f* `6 e' P. ?- \: f C/ ? }# H
COPY
+ \6 s1 j0 z- aFor more details, see http://crmsh.nongnu.org/crm.8.html#cmdhelp_corosync.
( ^8 @/ w/ u* i5 n
9 E" R& `$ g2 w6 a" ^7.3 Configuring Global Cluster Options: c# u8 w* w$ ^! J
- q9 I, A8 P9 o& W V$ SGlobal cluster options control how the cluster behaves when confronted with certain situations. The predefined values can usually be kept. However, to make key functions of your cluster work correctly, you need to adjust the following parameters after basic cluster setup:
4 z+ x* O- c* ~9 E, o- X& e
2 W$ T& i6 h, K5 v, @4 _PROCEDURE 7.1: MODIFYING GLOBAL CLUSTER OPTIONS WITH crm
R" Q9 g- A- m) {" K: w% {: |* }* p$ Z9 e. P" U
Log in as root and start the crm tool:0 ?3 b t/ P2 c: {. p
A* b! a& K( o3 U( i9 ?: k1 M9 Ncrm configure
( D, u9 V; d1 X. K/ TCOPY
m+ a( w" _/ q: FUse the following commands to set the options for two-node clusters only:8 T# R J/ E3 I% ~$ F- x: C
9 w2 l- E/ k9 J, M4 r! M
property no-quorum-policy=stop
/ [8 C; w0 k* J: j L4 S% d9 b: iproperty stonith-enabled=true# V9 T: w4 J2 x
COPY3 x+ W) }9 W3 v/ N2 n/ X7 L; M. q
ImportantImportant: No Support Without STONITH8 {; |, ]8 a& k( B5 j; `% P
A cluster without STONITH is not supported.5 N0 f! M1 E- a+ z7 D
* t% B, ~4 O1 ? M# PShow your changes:
, O! ^. `; a) B6 F# ?
5 r s( T) H6 h/ |& P0 g0 ushow
/ K, K( W4 u( V T1 U/ b6 rproperty $id="cib-bootstrap-options" \* @# b; i6 M! q; b4 k
dc-version="1.1.1-530add2a3721a0ecccb24660a97dbfdaa3e68f51" \
" V, j* _5 [ X cluster-infrastructure="corosync" \
* `# I0 B$ c7 }2 D$ @7 t expected-quorum-votes="2" \
i/ L% n0 l7 g4 T no-quorum-policy="stop" \
% ~7 p* K2 Q+ A1 D4 i stonith-enabled="true"
9 V. D: ` W% W+ H/ D ?( g' eCOPY
4 P: U3 s2 ^1 C+ C' bCommit your changes and exit:
5 |; t: c8 l6 k
) }# r* I0 `7 S, k& P5 Ycommit
) T; D7 s$ L( r, m6 y; f' l# wexit( V6 @% I5 n+ [$ t0 V
COPY
: b. y3 p5 W5 p9 w' v6 ~7.4 Configuring Cluster Resources* V% P) u( D2 i/ a, R
" {: p8 {1 g# r4 R) aAs a cluster administrator, you need to create cluster resources for every resource or application you run on servers in your cluster. Cluster resources can include Web sites, e-mail servers, databases, file systems, virtual machines, and any other server-based applications or services you want to make available to users at all times.
' `0 [( Z7 E$ L% @' [( v& S, E; V" g; E9 P
For an overview of resource types you can create, refer to Section 5.3.3, “Types of Resources”.6 I$ L$ q; c! U
- T$ o0 t4 k q& _3 h
7.4.1 Loading Cluster Resources from a File" G, {+ W: H' e
" W Q) X9 r& j9 _5 Y" h1 p' CParts or all of the configuration can be loaded from a local file or a network URL. Three different methods can be defined:: i# S6 i- n* [9 w' n8 u# z# F
) L0 S; f# U4 Y/ j0 |( D0 ureplace: m$ B9 m+ [* Q1 K
This option replaces the current configuration with the new source configuration.
. u0 J4 p' U+ f, v) N% U! D
/ U( `( t3 m$ ]& d/ T: g2 Aupdate
, M" i9 k I, ^9 L5 hThis option tries to import the source configuration. It adds new items or updates existing items to the current configuration.
6 k% @3 y+ f7 ?) e+ O5 {
, }! V; \" I* _/ {$ {push
N/ U. k7 k8 f+ MThis option imports the content from the source into the current configuration (same as update). However, it removes objects that are not available in the new configuration.
5 C8 n4 P+ d/ P: ]* ?: l% x) [8 z
( O0 d% T' R1 r% o3 M/ vTo load the new configuration from the file mycluster-config.txt use the following syntax:3 Z" Q1 W8 k4 }6 T
* R: B. A& \7 H5 t' x+ d M
crm configure load push mycluster-config.txt! F/ v. w& E0 B$ e1 }1 z
COPY/ d( [, l: e/ s% _! S
7.4.2 Creating Cluster Resources
u$ O2 o# k7 Q- _( l0 c5 h2 F$ e0 i p+ `$ Q% y& m, c1 Q1 {: j% L
There are three types of RAs (Resource Agents) available with the cluster (for background information, see Section 5.3.2, “Supported Resource Agent Classes”). To add a new resource to the cluster, proceed as follows:0 i& ^6 i8 L% V) ^3 G# N# a
8 Y- }% a% ~1 W/ D/ x) mLog in as root and start the crm tool:
8 t' j( @2 k# u1 g( s2 C9 `+ a* \0 e/ n* t5 h4 `* ^9 G* k
crm configure3 R: R+ u. a) a8 x
COPY
/ N) ~' z0 t% jConfigure a primitive IP address:
" c" n; b3 T J3 d: N
" ?# s0 Z J/ q% b6 cprimitive myIP IPaddr \4 D1 x/ Y3 c. e
params ip=127.0.0.99 op monitor interval=60s {4 K+ C3 X0 n. I
COPY
5 W- K6 `9 V: K3 c/ MThe previous command configures a “primitive” with the name myIP. You need to choose a class (here ocf), provider (heartbeat), and type (IPaddr). Furthermore, this primitive expects other parameters like the IP address. Change the address to your setup.7 K5 n7 z7 F" Q0 `: ^ |
& ]5 K' Z- c; u
Display and review the changes you have made:; Q6 {; N* ?; Z3 `* K1 A
9 T# z" r2 b& q! D! F* pshow k# M! K* D; @& W1 ^; V% K) ]
COPY
! d" r e* r/ w9 t+ UCommit your changes to take effect:
. x$ U0 M2 d; I; N' V4 d
3 ]4 ~& S L2 ^+ i" j$ dcommit: ~9 S! S+ h0 g
COPY
% z0 `9 t7 C& Y1 L7 Z; h7.4.3 Creating Resource Templates& U) q f0 N4 S( j1 H- N
+ w* Z4 `9 A6 E- e* D! H
If you want to create several resources with similar configurations, a resource template simplifies the task. See also Section 5.5.3, “Resource Templates and Constraints” for some basic background information. Do not confuse them with the “normal” templates from Section 7.1.6, “Using Configuration Templates”. Use the rsc_template command to get familiar with the syntax:* L# U/ A$ A, u8 A+ ~' X- P
5 D5 q) B# P7 b! o P
crm configure rsc_template* s0 p/ R* Z( X- A
usage: rsc_template <name> [<class>:[<provider>:]]<type>
2 A9 N, G( V' g5 L6 h$ q# T8 Z [params <param>=<value> [<param>=<value>...]]
% L( ]4 ~5 F, f% W- w [meta <attribute>=<value> [<attribute>=<value>...]]
: e. T4 h' w8 G- t$ _' u [utilization <attribute>=<value> [<attribute>=<value>...]]
$ F2 |. l& I$ m; j# V; w# { [operations id_spec' t1 c1 i; w+ v _" M# l9 x
[op op_type [<attribute>=<value>...] ...]]
+ I5 w$ Q. r* w% ?2 W: hCOPY
( q: r" B. W" w* o y- X# r+ b$ bFor example, the following command creates a new resource template with the name BigVM derived from the ocf:heartbeat:Xen resource and some default values and operations:
0 l t/ e. k c8 }6 }" s' S
. ^; A" |1 N) brsc_template BigVM ocf:heartbeat:Xen \
6 M: _% d5 D9 p; [/ E params allow_mem_management="true" \
2 q0 E0 ~* L% V7 q U0 y op monitor timeout=60s interval=15s \
: \) G( t, b1 M0 v2 Y. @) W: f- u op stop timeout=10m \
5 H% a6 u) B1 b+ e# @ op start timeout=10m1 F) ^+ q1 R, {# I& P
COPY
0 P) r* ?8 H5 _1 k& @. c* ?Once you defined the new resource template, you can use it in primitives or reference it in order, colocation, or rsc_ticket constraints. To reference the resource template, use the @ sign:5 ?/ N: A' d$ j" h* P
. I$ j0 K. y* I e0 X( E8 [primitive MyVM1 @BigVM \
" d' w f# [8 m; v params xmfile="/etc/xen/shared-vm/MyVM1" name="MyVM1"
f' t3 d" d7 W% g/ h& n2 P! `COPY) x* a9 X I v+ X) r* a O
The new primitive MyVM1 is going to inherit everything from the BigVM resource templates. For example, the equivalent of the above two would be:
# d/ q' c5 t1 F4 L' L1 O( N
+ ] ~2 m k/ c' v( T \( Uprimitive MyVM1 Xen \
& g( ?6 q* a6 `/ u7 e params xmfile="/etc/xen/shared-vm/MyVM1" name="MyVM1" \
`# U% F6 A4 L: E: b8 u params allow_mem_management="true" \
6 S5 ?& [* c# ?7 m op monitor timeout=60s interval=15s \
) M! F" ?( a3 _2 [% e6 u) W op stop timeout=10m \+ H# ?. I9 E3 \ Q2 |2 @
op start timeout=10m9 _/ {" b* B, i
COPY
# [' c) P0 l7 y8 i6 i5 D" n; j& OIf you want to overwrite some options or operations, add them to your (primitive) definition. For example, the following new primitive MyVM2 doubles the timeout for monitor operations but leaves others untouched:) ]6 V" N9 y4 R+ T3 n. _- w
9 o1 M/ A# q1 m6 q9 E2 b6 \
primitive MyVM2 @BigVM \4 R: R0 O/ K/ s8 c2 b. O
params xmfile="/etc/xen/shared-vm/MyVM2" name="MyVM2" \
) r5 J. H, L9 k5 N0 z* p- d op monitor timeout=120s interval=30s
* L% J, m! z$ s4 `2 O$ D8 O1 oCOPY
) j% J- S# {$ \% _+ O- lA resource template may be referenced in constraints to stand for all primitives which are derived from that template. This helps to produce a more concise and clear cluster configuration. Resource template references are allowed in all constraints except location constraints. Colocation constraints may not contain more than one template reference., j$ P g, d" E. }7 E$ q$ T
$ ?% v$ `% _9 @7 S. Z! A$ I
7.4.4 Creating a STONITH Resource8 w# i5 W8 @2 o0 b. e
+ a/ R( e* K% _( ?
From the crm perspective, a STONITH device is just another resource. To create a STONITH resource, proceed as follows:
( n, `0 i7 K% f5 [, s# J# K, t
/ C( \ I" Y2 S0 ?Log in as root and start the crm interactive shell:
' U7 `7 g- J: x/ S: z) {
9 h* c+ G3 Q% _1 Ocrm configure
5 S4 U+ G$ v- ~! f- L* s7 pCOPY
+ `4 F8 {3 i7 u$ B% vGet a list of all STONITH types with the following command:
6 C3 u# I. K( R* s. X$ C" M. I6 o9 s; a* Q
ra list stonith/ W0 F% J" G- s+ ^9 m7 w5 u; K
apcmaster apcmastersnmp apcsmart5 K2 G+ k0 q" f
baytech bladehpi cyclades
C- u, _: w, N" s1 W( Fdrac3 external/drac5 external/dracmc-telnet% N( A' |* i2 |7 j: Q5 e" i5 H
external/hetzner external/hmchttp external/ibmrsa1 T- t9 `9 I7 l! f# o- D/ G
external/ibmrsa-telnet external/ipmi external/ippower9258
! i3 R" W, n% e7 B" Uexternal/kdumpcheck external/libvirt external/nut4 H+ K# s$ Q2 m7 U! U2 y$ @; L
external/rackpdu external/riloe external/sbd- v+ j7 T; E3 F$ u( f9 R- I) f
external/vcenter external/vmware external/xen0
) Q1 ~4 j; E' y5 j3 oexternal/xen0-ha fence_legacy ibmhmc
2 E _# T, p0 t% K/ }3 d1 |ipmilan meatware nw_rpc100s
- q s. f9 p3 @. {rcd_serial rps10 suicide. E$ O0 n8 U5 d8 {+ |& h
wti_mpc wti_nps
1 S# I1 e( s7 p" J. G1 i8 z7 u ZCOPY8 Y1 f: l5 P. m6 [
Choose a STONITH type from the above list and view the list of possible options. Use the following command:
$ _; F7 T$ H2 W+ G4 n
: U+ q. Y0 m6 gra info stonith:external/ipmi
4 }( _! X8 x+ J7 x1 TIPMI STONITH external device (stonith:external/ipmi)
" n% R7 E& B" m( y1 S( L6 l' r9 @( `0 N& p) u7 a- S) A
ipmitool based power management. Apparently, the power off i. ^4 w [7 v
method of ipmitool is intercepted by ACPI which then makes
P2 k7 W! ~( w% y" D+ W% K1 A1 Ca regular shutdown. If case of a split brain on a two-node1 d% n1 R) v& H/ T) R
it may happen that no node survives. For two-node clusters! u" d3 t y9 [5 f
use only the reset method.
& ^* x# V$ d* V2 L4 W1 m/ ~! k* w' }/ b$ i0 `
Parameters (* denotes required, [] the default):5 |/ A: t7 x& E+ ]$ ]
2 `5 \' K5 z& n6 u* \5 G5 V3 d
hostname (string): Hostname
) h6 @8 [" Z* @, A( c( c1 F+ I8 W* ~ The name of the host to be managed by this STONITH device.1 \ n* W; \! z( h. u
..." C) u, C& C6 q% [' K
COPY
1 N; e- p0 ~, }6 @' ~8 {) d6 d }Create the STONITH resource with the stonith class, the type you have chosen in Step 3, and the respective parameters if needed, for example: x9 o) H: u, |+ h6 O2 {
1 o6 a+ u/ c2 }& P7 V1 a
configure
' D6 R( J! F3 _( U- [/ Nprimitive my-stonith stonith:external/ipmi \
8 B# z/ {+ z- O4 U! e9 |# V params hostname="alice" \
r# @; e9 X9 Q' V+ M+ T. T ipaddr="192.168.1.221" \7 s9 K2 A" u' p/ |$ }( X, l/ o
userid="admin" passwd="secret" \
& y; e9 g7 ^ j$ { op monitor interval=60m timeout=120s; a; O d1 F' T: h$ j& k
COPY% j2 F' C! K- p* |" A9 ]8 g
7.4.5 Configuring Resource Constraints
; A& L, c/ q5 U5 ~
6 S* a( s0 P. f: zHaving all the resources configured is only one part of the job. Even if the cluster knows all needed resources, it might still not be able to handle them correctly. For example, try not to mount the file system on the slave node of DRBD (in fact, this would fail with DRBD). Define constraints to make these kind of information available to the cluster.# ^2 g" ?* _9 r3 m$ t. z7 _
, l6 X2 `3 W% J8 qFor more information about constraints, see Section 5.5, “Resource Constraints”.
% }$ _8 S. E( B9 Y5 R; d# n+ D& S2 n4 k! S- [9 M* g- A! f1 } H' K) {
7.4.5.1 Locational Constraints+ U' F3 Z/ g0 z9 h* u! C5 n7 l
" L* _7 M; _6 ^' g% R, aThe location command defines on which nodes a resource may be run, may not be run or is preferred to be run.
7 k! u: v& a$ |/ A
1 O+ q# h( K3 D, b2 v6 }6 d+ ^This type of constraint may be added multiple times for each resource. All location constraints are evaluated for a given resource. A simple example that expresses a preference to run the resource fs1 on the node with the name alice to 100 would be the following:4 _! d7 `' q! f d/ ~3 c: f7 w
" K% Y% ?2 G2 ]% V6 e9 slocation loc-fs1 fs1 100: alice
: j# n0 o5 }5 S' x$ }' Q9 ]COPY4 ^4 ^) J+ Y$ b! k- j8 `0 M
Another example is a location with ping:$ y$ N p* S: s' k u
$ q" w- s2 ?3 n8 ?2 d0 L
primitive ping ping \
2 ]* _- q" I" M1 f' e- d) N params name=ping dampen=5s multiplier=100 host_list="r1 r2"
& ~6 k2 w# E: k* C) ~& zclone cl-ping ping meta interleave=true9 T+ Q. e" z5 x, O" R! a9 L
location loc-node_pref internal_www \
: C: X7 A% }' l6 L( g8 K# x rule 50: #uname eq alice \
+ B+ j, E* H( O& ?! q5 E4 L+ B rule ping: defined ping
( I; E0 y, s: yCOPY* [* m4 g/ D! y
The parameter host_list is a space-separated list of hosts to ping and count. Another use case for location constraints are grouping primitives as a resource set. This can be useful if several resources depend on, for example, a ping attribute for network connectivity. In former times, the -inf/ping rules needed to be duplicated several times in the configuration, making it unnecessarily complex.& L9 Z9 ?+ A4 L/ Q( N3 ^5 E
, Z0 ^. [* l( [. U: b' S8 _$ P
The following example creates a resource set loc-alice, referencing the virtual IP addresses vip1 and vip2:
9 w* b0 K4 v- w; K; S% C' X3 k/ w ]
primitive vip1 IPaddr2 params ip=192.168.1.5- u6 N6 z8 y3 i& ]4 Q7 r8 J, L
primitive vip2 IPaddr2 params ip=192.168.1.68 V! G) g8 m z% i1 x* `0 Q$ |
location loc-alice { vip1 vip2 } inf: alice
+ A& |8 L. G; v0 v- F9 r; ~COPY
1 l( D! {) W' L d) R5 BIn some cases it is much more efficient and convenient to use resource patterns for your location command. A resource pattern is a regular expression between two slashes. For example, the above virtual IP addresses can be all matched with the following:* S! _$ q+ {- o- A6 p$ G
, V! y2 l' q% f' u7 D/ \' d
location loc-alice /vip.*/ inf: alice* y6 b: J; T7 e5 k
COPY
( X5 a+ X; z: E u* l# m! U7.4.5.2 Colocational Constraints
+ g; {! [2 v2 Y$ f: D7 @9 ~1 ?) @+ {6 g) x% R& b
The colocation command is used to define what resources should run on the same or on different hosts.' c' [/ i# P' k( e
( Z- V/ V/ y: y- j( dIt is only possible to set a score of either +inf or -inf, defining resources that must always or must never run on the same node. It is also possible to use non-infinite scores. In that case the colocation is called advisory and the cluster may decide not to follow them in favor of not stopping other resources if there is a conflict.! a! `% X6 T3 H0 K, [4 ?
3 q7 @' Q& s5 c4 O; z' m
For example, to run the resources with the IDs filesystem_resource and nfs_group always on the same host, use the following constraint:6 Z& O+ l. X7 j, D# E! v
: p) F- g1 f! s0 o2 ocolocation nfs_on_filesystem inf: nfs_group filesystem_resource5 I K8 K5 N, P: D" y; T) j
COPY
. a. @4 ~9 M6 @3 W: U) a+ J3 R: zFor a master slave configuration, it is necessary to know if the current node is a master in addition to running the resource locally.
' n/ p+ t+ v: g# t2 j( R, N
. A6 f- j+ T8 m: s$ r$ r- ]7.4.5.3 Collocating Sets for Resources Without Dependency
* R9 _5 ?3 E% x7 A5 J% ~. |# @1 o! b; o- p8 Q1 s% v& W
Sometimes it is useful to be able to place a group of resources on the same node (defining a colocation constraint), but without having hard dependencies between the resources.
+ U. O. R, z5 c& O1 A
( p# W8 Y' {# v+ i0 j" A, YUse the command weak-bond if you want to place resources on the same node, but without any action if one of them fails.+ V6 R, K4 G2 @
, E- w& e4 m$ J* i# J* N
crm configure assist weak-bond RES1 RES2
; K9 |+ D a- F/ {1 gCOPY- [1 V9 j* y' s, C% v/ P3 M4 ]
The implementation of weak-bond creates a dummy resource and a colocation constraint with the given resources automatically.
. `" G7 R* C4 Y; |. W: c
9 T8 W$ C2 K' h+ A2 r: e7.4.5.4 Ordering Constraints
/ J$ s: t/ o9 }/ T& Q+ U+ C) @, D8 l9 v- r8 a: Q% N
The order command defines a sequence of action.+ l7 ]; }- h% {% x' {7 b* J7 k
" _6 j; u& o+ n( \' z4 f. _
Sometimes it is necessary to provide an order of resource actions or operations. For example, you cannot mount a file system before the device is available to a system. Ordering constraints can be used to start or stop a service right before or after a different resource meets a special condition, such as being started, stopped, or promoted to master." o& F( h, S% R; O
9 s' S7 n/ q5 H8 y k) B
Use the following command in the crm shell to configure an ordering constraint:
& N% k1 t/ c; p3 o& d A o8 D5 G( w1 E, @! I) T6 `, p" E
order nfs_after_filesystem mandatory: filesystem_resource nfs_group
$ g9 `/ z( s1 Z) K" FCOPY# R- m' j) v) M$ n% Z! \
7.4.5.5 Constraints for the Example Configuration+ b, f0 A. r' t
Z; m7 r7 ?& B7 l
The example used for this section would not work without additional constraints. It is essential that all resources run on the same machine as the master of the DRBD resource. The DRBD resource must be master before any other resource starts. Trying to mount the DRBD device when it is not the master simply fails. The following constraints must be fulfilled:5 b+ l2 D! E5 b! O- q- J( F
* V3 N/ L* H6 x: h+ ?" H: nThe file system must always be on the same node as the master of the DRBD resource.
9 t0 W4 _. I" ]$ q' q/ {3 E9 q$ W1 a
; F9 L! L+ t2 Bcolocation filesystem_on_master inf: \
4 w) K9 _ `& x" o* ] filesystem_resource drbd_resource:Master
' I" w9 A5 h8 v1 d5 x8 yCOPY
! e) l1 b5 a7 d. \ j: JThe NFS server and the IP address must be on the same node as the file system.# v3 ~0 B, _2 u) Y0 q
9 G; _7 I u; z; y$ h7 d o' _
colocation nfs_with_fs inf: \
; c& \- V3 R- Q5 E( G0 z, ` nfs_group filesystem_resource
) O/ P" P+ K4 e4 y% s0 yCOPY
$ b, g$ j0 ^ m5 J6 iThe NFS server and the IP address start after the file system is mounted:
^: F3 z5 D) n$ J9 S2 n/ }( T+ R( I, u9 j! i% g1 m
order nfs_second mandatory: \8 j5 m5 d& L* G# |4 d
filesystem_resource:start nfs_group7 Y3 o- Y/ G$ X. S% \$ X0 t0 i
COPY
' u* o( I, i; m; ]% nThe file system must be mounted on a node after the DRBD resource is promoted to master on this node.
A5 n$ m, x. E1 M& i. g
" @7 l2 o3 d0 J( ]/ P8 x7 border drbd_first inf: \. A5 |# B& T! P2 T! T% S7 c
drbd_resource:promote filesystem_resource:start) w: I& c3 S6 l+ y7 i
COPY
. f& F! q5 Z9 D/ O _/ K+ K& q( z+ C4 @7.4.6 Specifying Resource Failover Nodes
3 c1 u' S6 w/ J: [3 @8 o- S( A) E' E6 `0 l$ u
To determine a resource failover, use the meta attribute migration-threshold. In case failcount exceeds migration-threshold on all nodes, the resource will remain stopped. For example:
* h; M, p" j- i. O: I4 ]4 d6 A R% I$ n V% d: P# j/ _
location rsc1-alice rsc1 100: alice
+ T8 |( ~, w. S# O# q/ d1 D2 W$ C' eCOPY7 b8 V7 m0 q0 B8 z/ |* L. B
Normally, rsc1 prefers to run on alice. If it fails there, migration-threshold is checked and compared to the failcount. If failcount >= migration-threshold then it is migrated to the node with the next best preference.9 S6 h3 r5 P% p; n
; c$ j ` z" k: `. `Start failures set the failcount to inf depend on the start-failure-is-fatal option. Stop failures cause fencing. If there is no STONITH defined, the resource will not migrate.: `4 Z& B2 R5 R: `# b
- F: H2 w7 D& S9 o0 {6 b1 T
For an overview, refer to Section 5.5.4, “Failover Nodes”.7 V( c- M+ k" A
6 m5 j$ j) m) O( X# l3 B
7.4.7 Specifying Resource Failback Nodes (Resource Stickiness)' V# c; P% O# I/ P5 `
* s. I+ }- a' ~A resource might fail back to its original node when that node is back online and in the cluster. To prevent a resource from failing back to the node that it was running on, or to specify a different node for the resource to fail back to, change its resource stickiness value. You can either specify resource stickiness when you are creating a resource or afterward.
3 x' R$ D) Q5 n* ^; V. Z- z) V
' ~0 @. Z; C& O q9 b$ O! X) gFor an overview, refer to Section 5.5.5, “Failback Nodes”., i) i/ t7 @; [0 e2 o0 ]" j
' ^0 l. J6 L1 K A9 e
7.4.8 Configuring Placement of Resources Based on Load Impact
5 E2 u3 S; l. [9 ]* ^6 Y5 w% j- B: v' h% l3 [/ M, N3 o
Some resources may have specific capacity requirements such as minimum amount of memory. Otherwise, they may fail to start completely or run with degraded performance.- P; W, V0 i9 ~7 M3 ]. ]
) T4 d2 ^8 y6 l9 TTo take this into account, SUSE Linux Enterprise High Availability allows you to specify the following parameters:
+ _) \/ L7 P5 J& m( g* ?: z
e3 t0 i0 T) A+ o" EThe capacity a certain node provides.
* q+ t, k% N! t' h5 e6 P. ~, ~% Z; X- y, u/ o' I. [! D* ]1 L! Y
The capacity a certain resource requires.) B: Q t0 L9 P! M- p; W
6 C$ E1 V" K7 d
An overall strategy for placement of resources.
9 O3 o7 r1 A3 j9 D4 Y- `$ R5 e
! I/ O; d1 z; e+ _For detailed background information about the parameters and a configuration example, refer to Section 5.5.6, “Placing Resources Based on Their Load Impact”.
y3 C ?# G6 N! A% A8 |4 J. T6 X+ o; X# Q
To configure the resource's requirements and the capacity a node provides, use utilization attributes. You can name the utilization attributes according to your preferences and define as many name/value pairs as your configuration needs. In certain cases, some agents update the utilization themselves, for example the VirtualDomain.7 ?7 c [5 a! H; E; ~' |7 R
& z; a6 q( L6 O; |- N
In the following example, we assume that you already have a basic configuration of cluster nodes and resources. You now additionally want to configure the capacities a certain node provides and the capacity a certain resource requires.( ]0 D6 S9 k1 \; O4 k1 @
+ L0 I# E* G. D9 E! r$ T# k
PROCEDURE 7.2: ADDING OR MODIFYING UTILIZATION ATTRIBUTES WITH crm
+ n1 L" k/ o8 l1 N, p9 I- R
, m5 z- t' H& Q" J$ B' dLog in as root and start the crm interactive shell:
& S4 |7 M! [$ R
+ T5 Q) m W- @7 dcrm configure
' `1 S1 q0 S1 {0 eCOPY! o1 V' {: P( P7 |+ y2 [1 i
To specify the capacity a node provides, use the following command and replace the placeholder NODE_1 with the name of your node:" k7 w X0 L0 D$ m
7 f ~! T- q: a+ S1 g0 B
node NODE_1 utilization hv_memory=16384 cpu=8
3 v+ ?9 T& u5 Z2 X2 H* {COPY; ?+ [, J0 X0 s2 t5 F0 w) q: c
With these values, NODE_1 would be assumed to provide 16GB of memory and 8 CPU cores to resources.
7 D S$ z& x2 E1 K$ N
5 l& s$ F: k2 _/ _, z" PTo specify the capacity a resource requires, use:- M" v4 ~, T4 e" U
9 m/ G8 [1 d3 P/ x* T0 ^5 J" I) F
primitive xen1 Xen ... \- A, o+ K8 Z8 Y! V/ R
utilization hv_memory=4096 cpu=4" `3 C6 l' e$ V# Z8 B$ e! t
COPY
5 M5 ~. w8 f, ?1 }& c; B* E7 d$ R2 |This would make the resource consume 4096 of those memory units from NODE_1, and 4 of the CPU units.' E& M) }, Q* r g m% Z6 \: C
. L5 I6 W5 E% v; Y( xConfigure the placement strategy with the property command:
\7 w3 F4 U8 f) j8 @# U1 H* K" W8 @4 b3 [, r! z
property ...* Q( D4 K3 x$ w$ h. {% o
COPY
- ~2 }# E$ a& ]/ v) P8 n8 \The following values are available:
6 g4 K5 B( q. z, E- s4 i$ D4 {1 H% u- k& s4 n$ G2 `1 W% h
default (default value)
# P3 z( H) |- x' O sUtilization values are not considered. Resources are allocated according to location scoring. If scores are equal, resources are evenly distributed across nodes.
# N* b% \! g! _9 r6 k/ L7 f* Z6 E6 V8 j( t$ z' ?$ Z
utilization( i" T+ i! Q# s7 b
Utilization values are considered when deciding if a node has enough free capacity to satisfy a resource's requirements. However, load-balancing is still done based on the number of resources allocated to a node.
8 I& b, s: ~# N3 w9 f8 r5 Z( t4 |, M
minimal
3 U o' I( x iUtilization values are considered when deciding if a node has enough free capacity to satisfy a resource's requirements. An attempt is made to concentrate the resources on as few nodes as possible (to achieve power savings on the remaining nodes).
( \- V2 b7 x* X% v9 Z1 T( h( S* D; |0 _" {
balanced8 ]) c9 W2 `/ H9 }, a6 P1 O8 a. t$ r. `
Utilization values are considered when deciding if a node has enough free capacity to satisfy a resource's requirements. An attempt is made to distribute the resources evenly, thus optimizing resource performance.
/ n3 n- ~+ S4 p' o$ v* f4 Q8 R( \, D: Y
NoteNote: Configuring Resource Priorities
3 c5 k! M4 ~; A5 f# k* }5 Y: XThe available placement strategies are best-effort—they do not yet use complex heuristic solvers to always reach optimum allocation results. Ensure that resource priorities are properly set so that your most important resources are scheduled first.& b. B" S# S2 w& q
( c! q6 C0 U) RCommit your changes before leaving crmsh:
: ?8 A+ {- B+ V5 _ r1 \8 Y3 C! L* \+ V8 u8 b$ O
commit7 [/ T& ^* x6 V, X: ~
COPY
1 s* j3 G0 ^3 a+ h7 \! CThe following example demonstrates a three node cluster of equal nodes, with 4 virtual machines:6 y) F1 F0 G0 U K$ }5 H
+ E! S9 g# N+ j( X* u: Y7 _9 Jnode alice utilization hv_memory="4000"
6 t/ b O/ e6 }( t A8 Y. nnode bob utilization hv_memory="4000"4 V/ Z! H( C" d' a
node charlie utilization hv_memory="4000"
7 D2 r- V, D( }/ G, a" d9 oprimitive xenA Xen \
4 \5 e- R9 K( K/ j utilization hv_memory="3500" meta priority="10" \
" I) q0 c: w5 ?+ G1 a0 k params xmfile="/etc/xen/shared-vm/vm1"$ f. Q5 i6 d$ l$ S) ?' z, a( B2 {
primitive xenB Xen \; |0 w( H% \2 Y% m. H5 B1 p
utilization hv_memory="2000" meta priority="1" \. H0 M5 ^% ] u6 j4 N
params xmfile="/etc/xen/shared-vm/vm2"( l; h- M* |) L `7 t& n
primitive xenC Xen \+ V$ O7 X4 j, A
utilization hv_memory="2000" meta priority="1" \, V q2 r1 B f7 C' e" U! h
params xmfile="/etc/xen/shared-vm/vm3"
9 S8 C! Y& m, I4 t3 zprimitive xenD Xen \! p& V4 o- U) g+ v! f8 p/ _
utilization hv_memory="1000" meta priority="5" \. p+ D8 @7 E) Q' K2 \, A
params xmfile="/etc/xen/shared-vm/vm4"* Z* J8 h# \: B) L) h! z
property placement-strategy="minimal"
2 g2 w/ C$ X: y/ @" ^! b3 j JCOPY
( @7 }: P0 T* M- AWith all three nodes up, xenA will be placed onto a node first, followed by xenD. xenB and xenC would either be allocated together or one of them with xenD.4 {( v2 S, |% N% S& k* H
* X9 D T I1 R/ t5 ~If one node failed, too little total memory would be available to host them all. xenA would be ensured to be allocated, as would xenD. However, only one of xenB or xenC could still be placed, and since their priority is equal, the result is not defined yet. To resolve this ambiguity as well, you would need to set a higher priority for either one.3 }2 X# C, |5 y L' x0 e
4 B, Y' ?2 J7 {' `7.4.9 Configuring Resource Monitoring
) ?, K& m' H% x/ m; h) I" @3 X A. l* |" ~ L" A
To monitor a resource, there are two possibilities: either define a monitor operation with the op keyword or use the monitor command. The following example configures an Apache resource and monitors it every 60 seconds with the op keyword:: D/ m0 ~4 E5 p# Z+ }# n5 [
" L* t ]/ E9 A! Zprimitive apache apache \2 }6 _/ c/ C" ]: p/ I' W0 X9 m
params ... \
) L$ t7 m. V- e4 u op monitor interval=60s timeout=30s: w% O. i% Q( f6 m: [6 K1 x9 _
COPY
3 w& ?4 m$ k" w. w' H5 yThe same can be done with:3 j$ Y" |( I+ v* x* n7 c$ O9 f# f
0 A& m. T* H- @( nprimitive apache apache \* U7 m0 u4 U4 V" W4 k
params ...
* |- U& r( D' L$ k/ J# w& nmonitor apache 60s:30s
$ X! a4 |/ |5 H, E" OCOPY
2 m3 u# G. k3 u; U, k/ oFor an overview, refer to Section 5.4, “Resource Monitoring”.9 _- E$ ?" W; l* B6 y
7 F0 \9 k; a0 g# U1 L' Q7.4.10 Configuring a Cluster Resource Group
! {! `, G6 T3 b5 {% q" N* ^' ~* k: e/ l; L+ o
One of the most common elements of a cluster is a set of resources that needs to be located together. Start sequentially and stop in the reverse order. To simplify this configuration we support the concept of groups. The following example creates two primitives (an IP address and an e-mail resource):
1 \- t1 L0 ~& q% I+ @9 j
5 `& Z, F0 S `' xRun the crm command as system administrator. The prompt changes to crm(live).( _7 u& u9 k1 f! v3 u3 g+ I/ G% U
/ {5 q# |" F. D/ ]: bConfigure the primitives:2 `) ?9 n( y1 `% U ]0 [6 w
' w! _( ?: o1 P2 fconfigure
: U! z9 E7 L4 ^$ J& q7 w- Rprimitive Public-IP ocf:heartbeat:IPaddr \. X$ J& k4 @* X \2 r$ [/ f4 g
params ip=1.2.3.4 id= Public-IP
) o- w5 b j7 \) xprimitive Email systemd:postfix \
1 R: C; V& w+ W9 }3 _' M K6 L: c params id=Email ^* n: s$ w2 E i, e
COPY
# ^! ~ L4 V# u4 rGroup the primitives with their relevant identifiers in the correct order:
% ^, ]* H- h7 Z8 C
* f4 r2 O$ q( ygroup g-mailsvc Public-IP Email# a1 r: S9 Z- S) u& C/ V
COPY6 h2 Y) ^# k: t r7 i" H( S0 x
To change the order of a group member, use the modgroup command from the configure subcommand. Use the following commands to move the primitive Email before Public-IP. (This is just to demonstrate the feature):
" y2 e; B. E9 J8 h; S
) L. B3 Y1 W# U% J( K9 Y$ ^% I6 B1 kmodgroup g-mailsvc add Email before Public-IP; N# ]3 g' u; ~( o: g
COPY
) g; m$ O+ q/ n. q$ |% j7 N2 wTo remove a resource from a group (for example, Email), use this command:
4 O3 N2 a+ V1 Y' a0 o7 l# q1 I% R# [- P `0 I
modgroup g-mailsvc remove Email
% a. u$ g7 G7 l! u+ ]: {2 N$ Y0 ^, }COPY
0 z; n0 H. }( k; d& N3 I! bFor an overview, refer to Section 5.3.5.1, “Groups”.3 r8 [. e6 ]! r" k
f8 W8 ~2 {8 \% h) D- F/ t6 Y/ p+ p7.4.11 Configuring a Clone Resource
- F2 X2 H8 L5 r
) O) Z( B- q. H4 pClones were initially conceived as a convenient way to start N instances of an IP resource and have them distributed throughout the cluster for load balancing. They have turned out to be useful for several other purposes, including integrating with DLM, the fencing subsystem and OCFS2. You can clone any resource, provided the resource agent supports it.
, s! Q$ J6 T- g8 ^8 J- [$ ~ i6 I
Learn more about cloned resources in Section 5.3.5.2, “Clones”.
2 j8 [% J; e9 w! H6 f4 G
) V9 {7 q* d1 a4 N5 C e+ Y9 V, N7.4.11.1 Creating Anonymous Clone Resources
; i: W. h3 u- W4 n u3 Q% I5 |1 j/ ~$ J7 K+ T5 v
To create an anonymous clone resource, first create a primitive resource and then refer to it with the clone command. Do the following:2 w8 V s4 C+ z2 p+ C, U2 m
9 g1 L- }) s ~9 g8 p2 y
Log in as root and start the crm interactive shell:
' Q8 P) ]4 f3 B
7 i0 X! r& _& s* v* c; O: \crm configure6 P) M+ ?. @# I9 ~. P
COPY0 U* y1 ~* F7 ^, ?( m" ?1 B
Configure the primitive, for example:+ J6 m1 ^7 D- S) Q5 S
8 W# {7 x6 P! H! B8 rprimitive Apache apache
( j5 t: Z0 a* N8 lCOPY5 W8 f6 B! i# p7 T' I9 E
Clone the primitive:
3 \/ p+ z3 i+ P$ b' M
8 R) d. \) Y& z( g' Z4 T" Hclone cl-apache Apache8 I9 t" ]5 z8 n; G& s
COPY
: K0 X% @1 F P! u: Y7.4.11.2 Creating Stateful/Multi-State Clone Resources8 l. |, W; |* |9 Y, F+ T g
2 K' l3 e- v# h. xMulti-state resources are a specialization of clones. This type allows the instances to be in one of two operating modes, be it active/passive, primary/secondary, or master/slave.5 Z, C9 a) Q ?7 d' T P0 ~, B# U
; o9 C( S; d) X1 @2 w/ R
To create a stateful clone resource, first create a primitive resource and then the multi-state resource. The multi-state resource must support at least promote and demote operations.
* ]/ E. V/ a$ C, o3 t3 o; j8 k+ o/ Q5 Z6 j0 P& C' p
Log in as root and start the crm interactive shell:* {" o9 j2 V" u8 q+ `( M
( v& C+ g% B9 N
crm configure
" T% j+ A6 g' w) h/ ]COPY7 o: F% p0 @. {/ H) g- ]
Configure the primitive. Change the intervals if needed:5 V% P) } {, L4 \9 r& _4 e
/ I" f$ L* g3 Bprimitive my-rsc ocf:myCorp:myAppl \+ f" d# L1 B* ?2 F8 b/ V; Z
op monitor interval=60 \6 n/ U; X- v" T. r0 h
op monitor interval=61 role=Master
- t: l! ]$ k9 k4 X% JCOPY
9 p1 Z4 Y( {9 \Create the multi-state resource:' X- ^2 E0 L5 }7 Q( m. V/ r5 y* i9 O
7 h o; p. p8 Z' hms ms-rsc my-rsc
. N+ L) m2 f4 V2 b( dCOPY
% j/ D3 D2 ?" N/ ], a7.5 Managing Cluster Resources" s3 e9 G' l2 S( d: W. b
! j7 ~" ]0 p+ \1 F, \. N PApart from the possibility to configure your cluster resources, the crm tool also allows you to manage existing resources. The following subsections gives you an overview.0 a3 O! T! y8 l& y$ _8 \
& [1 w( ] r5 j9 j3 I+ { Y7.5.1 Showing Cluster Resources. ~5 r6 Z$ W7 O1 Q, v
, w8 O/ G- N) K0 q& j# [* g. nWhen administering a cluster the command crm configure show lists the current CIB objects like cluster configuration, global options, primitives, and others:0 [ V* v) q3 ^' R. f
/ X' p X4 R+ N, a( d
crm configure show, R2 [& v& ^2 ?$ k+ ]
node 178326192: alice% _3 Q7 _& D7 P) |. e4 d
node 178326448: bob: E4 ?3 z4 q R0 A7 {0 y6 k
primitive admin_addr IPaddr2 \* c1 U+ ?& m, w N" {
params ip=192.168.2.1 \- ?7 A2 H; A, j# c' m7 K" g
op monitor interval=10 timeout=206 ]- F' r2 L; g8 i M7 X8 l5 h
primitive stonith-sbd stonith:external/sbd \# K4 A/ R9 i; y
params pcmk_delay_max=30- p ?/ @+ T3 M6 C
property cib-bootstrap-options: \
8 J! ?' Z- X! B7 @3 Z4 M/ \1 @ have-watchdog=true \
/ F* q& V5 }( P: y7 F dc-version=1.1.15-17.1-e174ec8 \
8 X9 G9 W, b& t% ?3 [+ K7 n, T( y cluster-infrastructure=corosync \
, g3 M3 ~" U' h! x; z cluster-name=hacluster \" k) ?3 a( e/ M [1 m$ f
stonith-enabled=true \
' t# m% |' e& E6 {# g. P% [ placement-strategy=balanced \
: ^! h* T& l" G( Q$ m H standby-mode=true9 r$ f# g4 \4 z" r% X! B$ o/ L; N
rsc_defaults rsc-options: \
8 R' t9 V' l5 R, h T) N; ^ resource-stickiness=1 \$ c v& }4 b0 [& F3 r3 q
migration-threshold=3
. {8 q) g) T6 G1 X* P& f& z; ^. ] hop_defaults op-options: \
) E1 I% a; Q" V: @6 B1 F; N timeout=600 \) X# Y/ i! z% s6 ?: q8 y
record-pending=true
9 T7 B( R; p- G; g& pCOPY$ G3 c- s& O* D' w+ h! [5 ]
In case you have lots of resources, the output of show is too verbose. To restrict the output, use the name of the resource. For example, to list the properties of the primitive admin_addr only, append the resource name to show:; I. P' F2 j/ F0 C' L J; N7 C
* E, }# O0 R$ R6 c/ X/ Gcrm configure show admin_addr" v8 [1 c6 I7 M/ ?" V6 q
primitive admin_addr IPaddr2 \
" I# Q. g; Y9 s; a* l% F5 ~ params ip=192.168.2.1 \: {. P9 k; e$ Z! ^6 _5 u
op monitor interval=10 timeout=20
0 y, J+ V0 s* Z n HCOPY- M8 O% F9 {! q: V: M2 P# q: S
However, in some cases, you want to limit the output of specific resources even more. This can be achieved with filters. Filters limit the output to specific components. For example, to list the nodes only, use type:node:" z& G$ C& u: E9 B8 M8 A
! C! F) a# c7 F8 g* J* K0 z( o
crm configure show type:node5 c% S+ W7 T7 H% Q
node 178326192: alice
; g* K5 }" I6 `6 Y; ] Cnode 178326448: bob4 I5 j' n: ~* }( W0 f
COPY
: C# g, S- }- Y; y6 ?6 bIn case you are also interested in primitives, use the or operator:
; _) N; N8 w, b N* v d( I. F$ w( w! G2 \- Q# C6 `
crm configure show type:node or type:primitive% }5 E" }) F( I$ y
node 178326192: alice# A* z9 j3 C+ s* P0 x( P
node 178326448: bob% p* E/ D1 m' Q2 u1 R
primitive admin_addr IPaddr2 \
' _% f) k2 }! x) R0 \# P3 S params ip=192.168.2.1 \8 I5 t+ j% K, B3 O3 X
op monitor interval=10 timeout=20
5 [4 e+ f8 l8 n4 I) g. d* wprimitive stonith-sbd stonith:external/sbd \
" S3 `# b* E$ b: K& j( j$ C params pcmk_delay_max=30
, k! k. D! q) f4 L+ B7 C' H: bCOPY$ s; M2 z6 o. s4 Q- J
Furthermore, to search for an object that starts with a certain string, use this notation:% ]+ v, I, p' }, U) t5 b
. A& O q- W* @7 tcrm configure show type:primitive and and 'admin*'
, f9 h. ]) w% s) g' D: Eprimitive admin_addr IPaddr2 \
$ {' k2 A) s+ R ~ params ip=192.168.2.1 \
! r9 B* M# D& C% I op monitor interval=10 timeout=20% B U5 ?' ?0 l
COPY
% V# z! o: ~& H4 A- b/ LTo list all available types, enter crm configure show type: and press the →| key. The Bash completion will give you a list of all types.
. T5 J" a# b C. U0 G: {- c a9 A, h& l9 r8 S1 @. w6 g
7.5.2 Starting a New Cluster Resource
" A P7 y7 o' k0 N; \9 K" i' Y1 }' P3 ?& o8 K3 ~0 C( q
To start a new cluster resource you need the respective identifier. Proceed as follows:
" P$ j0 X$ n$ n q- q' k1 }" Q% s# j
Log in as root and start the crm interactive shell:
* N9 D7 L3 o, f: z6 x% E% U& M2 J# T1 B& r
crm
2 g9 ?7 P$ B( f$ z$ ^COPY
( ~1 d) ]& F- ]# T; F o/ \8 lSwitch to the resource level:" w! @: E- I, W! S' j; Z
: F$ z& s6 W5 P
resource+ `0 u& y' o' \3 V
COPY+ X# K, M, d; I; u! o! n- {
Start the resource with start and press the →| key to show all known resources:
; ^# O2 N3 t. R7 `/ h* S, ], [$ v/ }5 s0 X9 g8 q
start ID
' I2 M8 j+ O$ vCOPY0 [! J9 [$ p6 ~, ]& N
7.5.3 Stopping a Cluster Resource/ t" g) I- s X% G" D' `- B$ _
- ^1 o1 o) V9 Y' [; J2 s/ \
To stop one or more existing cluster resources you need the respective identifier(s). Proceed as follows:1 u9 ?5 C% q' ^2 u
4 |: |! `/ ]. M. ^
Log in as root and start the crm interactive shell:
% R8 N: {, L/ X @' Q8 g6 Q$ a/ L D3 e- v6 _- R" s3 N
crm
* r" a. e0 [: L( i- A- H5 _4 kCOPY3 @( e8 L3 s( Y, t/ s+ F. h8 F
Switch to the resource level:
* Q7 \7 D7 h7 C- Y& q9 {. N0 B
V, {1 B$ b2 N+ L5 A' l$ fresource9 D y9 M7 F2 {& \
COPY
7 A) ` `( W" Z) L2 @% \( @" q1 vStop the resource with stop and press the →| key to show all known resources:
* d. f7 T& o- H' G
@3 B8 d# C4 U% ` |3 jstop ID' o0 C4 I0 k" b+ f' q
COPY, l0 I: t. m8 i4 v
It's possible to stop multiple resources at once:9 O/ T* P* m! u# y9 J8 m- X$ m+ I" b
: S( W4 R0 ? Z( ?6 G" B" D
stop ID1 ID2 ...
: S, R$ v: x0 Z1 m VCOPY
# d8 n) E/ G( a/ }! @$ y* B O7.5.4 Cleaning Up Resources, v1 x& c: Z: T* m k3 v% e
( K6 |: ~' ^/ }( b8 j2 b3 D
A resource will be automatically restarted if it fails, but each failure raises the resource's failcount. If a migration-threshold has been set for that resource, the node will no longer be allowed to run the resource when the number of failures has reached the migration threshold.
" {( P2 L* W3 ~$ l) G
0 b2 [. v; N* p0 T( BOpen a shell and log in as user root.
5 R* d ?1 y; H9 N. l5 {/ h. k1 E5 J# c
Get a list of all your resources:
( J L# H5 F9 B/ h; b8 s& l9 ]7 f6 ]3 [( B" ~
crm resource list
7 j3 Q+ Y9 V1 h) V4 d ...
) L% W' |$ o7 p, wResource Group: dlm-clvm:1: U1 F( ] X5 u3 S
dlm:1 (ocf:pacemaker:controld) Started! O& s1 z2 F* w: {8 p
clvm:1 (ocf:heartbeat:clvm) Started
" {/ a7 e( v# I* ]+ g* o9 wCOPY1 t8 |1 p4 o+ N5 Q
To clean up the resource dlm, for example:
- x7 C8 P. ?' k/ F7 {3 O6 I8 ~$ O1 M+ J8 x8 G
crm resource cleanup dlm& L! y% ?" p) r# s, L. `9 w9 n
COPY
) x8 l. ?" G- _' M5 w* ]3 I2 ^7.5.5 Removing a Cluster Resource: Q# Z- s% ] ]1 }9 m
% i# B. W$ F; o5 i" N, U1 \Proceed as follows to remove a cluster resource:4 [# U& k3 B- V
) r5 c2 \% d/ ^9 c, C N/ G% qLog in as root and start the crm interactive shell:
( Q- v$ @7 @4 ^# M* ^3 H
& r! K+ r+ h/ F# L* |! qcrm configure
! `* C; S, f' z" g# i0 q8 [COPY0 x# `) K+ F" V# c; k
Run the following command to get a list of your resources:: M1 n, p; p1 i+ r5 P
" e, M0 Q3 T5 q) r3 r
resource status2 b* M9 F& `( r7 C' o" s* F4 c
COPY% ^+ N, S3 H/ X- c6 F
For example, the output can look like this (whereas myIP is the relevant identifier of your resource):8 _) j4 G8 q% {# a
# Q, p. Q, Z; `; K' s( T' RmyIP (ocf:IPaddr:heartbeat) ...6 s h% w9 l/ D6 k0 O6 W Z
COPY. T" P0 M8 R% B9 Q9 [7 J) j- s& V
Delete the resource with the relevant identifier (which implies a commit too):
4 i" r: y! @' T, U9 J2 E! V: L3 Z+ R5 @2 M- G0 Y
configure delete YOUR_ID
1 l6 V( i8 N$ iCOPY
4 f. g' I9 X( }2 ^! y, w2 aCommit the changes:
" x2 b% k' }7 i3 [0 e8 m4 s* [8 P+ b7 r
configure commit
- F! B4 J p* b6 pCOPY' E2 s z, j" b* L
7.5.6 Migrating a Cluster Resource. P5 z! W; ~+ T
9 O$ {: G3 x2 ^
Although resources are configured to automatically fail over (or migrate) to other nodes of the cluster if a hardware or software failure occurs, you can also manually move a resource to another node using either Hawk2 or the command line.% w9 S/ ?$ t" K8 f7 U/ P; C, y
; {: i6 j; x6 L6 g( F
Use the migrate command for this task. For example, to migrate the resource ipaddress1 to a cluster node named bob, use these commands:8 Y8 `9 ] Q# g
( y$ G; l0 I6 [6 B9 h) i
crm resource$ @% B m% @! F; _
migrate ipaddress1 bob& `( l+ u9 T* t) o) k
COPY0 H9 u( y* n5 F: Q' W5 d& o6 J. {
7.5.7 Grouping/Tagging Resources+ W" e; o' r% Y( S2 C" Y
( |! u& Y/ m4 {8 B9 j
Tags are a way to refer to multiple resources at once, without creating any colocation or ordering relationship between them. This can be useful for grouping conceptually related resources. For example, if you have several resources related to a database, create a tag called databases and add all resources related to the database to this tag:9 t1 H! o7 M% H4 z8 F8 r' D
9 T! K q, e# I! ]. N# }' _3 wcrm configure tag databases: db1 db2 db3
! [5 G9 Z! K2 }5 z$ eCOPY
8 N) c! G i4 @. I2 Z" |This allows you to start them all with a single command:
! G7 i! Q$ i! h7 [: F7 T8 N; q' e+ z" U
crm resource start databases# ^, T; N/ d s4 B: m
COPY
# R3 l- B% F4 n( B& \2 `9 zSimilarly, you can stop them all too:7 Q" o- t5 t9 ?5 S
2 y) ]" {7 E! ~; j7 {# xcrm resource stop databases d+ z$ A9 P+ l
COPY" _8 f3 } N, X9 w; L9 v
7.5.8 Getting Health Status
/ a2 f+ W9 L9 [7 ~% D# D4 u; T$ n, c6 z u- N0 E4 V$ J
The “health” status of a cluster or node can be displayed with so called scripts. A script can perform different tasks—they are not targeted to health. However, for this subsection, we focus on how to get the health status.
8 \4 z" g8 u d) H9 @. O9 k4 H) X+ b# C! n# M
To get all the details about the health command, use describe:
3 W3 u" c" k; w( ~0 H& h9 @$ Q- K9 z" a3 c& B* y$ ^ k' M
crm script describe health7 t! Y7 S, e" T6 V+ S
COPY& K5 R; d- I% C
It shows a description and a list of all parameters and their default values. To execute a script, use run:9 n: l2 |% R6 g n' w; g! C
4 A2 k7 y8 k" D1 ^. ?6 l# q! C c
crm script run health
+ T; j0 Y8 F6 D- q. H, j! k5 FCOPY2 ]/ w ]1 S: i
If you prefer to run only one step from the suite, the describe command lists all available steps in the Steps category.5 s4 a# _( E/ k) I9 _
1 I2 y# ?8 L; Q5 F( W d* I' U" ?# j* P
For example, the following command executes the first step of the health command. The output is stored in the health.json file for further investigation:
! z) y+ r7 J$ b- h' S! z( ~" a4 s: h7 @$ U7 z0 f* c8 `$ j& E
crm script run health& \, X6 F1 t( W: ]. J- n
statefile='health.json'2 L" \ P" O q/ g
COPY9 f5 w3 O' G `! p' r
It is also possible to run the above commands with crm cluster health.
; O( t7 ^, {5 g1 @. e3 c9 n5 _# m
8 _9 v! A, P& F* j bFor additional information regarding scripts, see http://crmsh.github.io/scripts/.
& a3 Z! w7 W% Z+ Q4 \! I
& {$ U* {7 H+ J% j- ^7.6 Setting Passwords Independent of cib.xml" q' C5 `" J I9 g9 Y3 @
& G2 \8 |* A* v, v( C8 RIn case your cluster configuration contains sensitive information, such as passwords, it should be stored in local files. That way, these parameters will never be logged or leaked in support reports.3 ^: k1 ^. ^6 _" _) q
; q8 O8 Z# V& a- ]
Before using secret, better run the show command first to get an overview of all your resources:3 k: _2 Z- P1 W9 H6 c/ r; Q
! C, q2 w, O/ K o5 [7 w4 q
crm configure show" X( ^7 B# n% P- g9 y8 z' `
primitive mydb mysql \1 Z! V4 u- @6 e8 y& |
params replication_user=admin ...
- z" d9 D; ~- K) fCOPY1 m2 g5 N, |& ~" T" N. I/ t: t( M7 `
If you want to set a password for the above mydb resource, use the following commands:% b1 {; T( a& r$ Z
; W2 {, _: S }( C3 t2 F3 V [* ^& r
crm resource secret mydb set passwd linux
. P+ }$ q( ^8 A) V; s, }INFO: syncing /var/lib/heartbeat/lrm/secrets/mydb/passwd to [your node list]
, a6 ^& F* X' N3 R6 I# T- oCOPY X4 v- _+ _. e. ?2 E+ M
You can get the saved password back with:
# S8 S! N6 b1 H5 X; B% _
! W% Y3 L9 ^# P; \crm resource secret mydb show passwd3 X+ b( ]2 h1 o/ [0 e
linux0 V4 y$ l5 [ ^$ I
COPY
K0 f; f% [! H- g g% c3 DNote that the parameters need to be synchronized between nodes; the crm resource secret command will take care of that. We highly recommend to only use this command to manage secret parameters.& a2 E. v, A% x: D1 ]" n
4 I C; w$ g2 G! S/ {2 U# C1 ?* k
7.7 Retrieving History Information8 k$ Y& l$ g! o- T r
_: n) }2 T8 t5 R0 Q
Investigating the cluster history is a complex task. To simplify this task, crmsh contains the history command with its subcommands. It is assumed SSH is configured correctly. p( U% s! `2 N
0 ]$ p" z& Q8 `9 P
Each cluster moves states, migrates resources, or starts important processes. All these actions can be retrieved by subcommands of history.4 o- y5 V1 m! j9 C: z
4 o6 \/ d+ [6 N* E! g/ L9 XBy default, all history commands look at the events of the last hour. To change this time frame, use the limit subcommand. The syntax is:
! Q, s1 w* d, c+ A5 V7 V) s. @/ S' S8 V: l
crm history1 u& B& D- V0 e7 g& ~
limit FROM_TIME [TO_TIME]
. D" g& p6 H( l5 }7 O ZCOPY
' G; ]) d. B7 a2 DSome valid examples include:7 |% \* L& }) j8 Y' q/ k+ t. s
5 L: a/ w% D4 ^3 k
limit4:00pm , limit16:00
# U" }$ }% O/ @" h( J8 pBoth commands mean the same, today at 4pm.
: i! `& K: \: U" l
* H6 F1 Y% E: r2 u% tlimit2012/01/12 6pm
3 S* a5 @1 r x5 ZJanuary 12th 2012 at 6pm/ q" `# l4 ~: l, n* D
, E' z) K7 w9 ?0 z/ r# k; H$ F: elimit"Sun 5 20:46"# g/ ]- U9 I" k" v" m W! ?
In the current year of the current month at Sunday the 5th at 8:46pm
! L7 R- r4 B- j I& q6 K$ j# v
* b; W0 s: \ M# @, bFind more examples and how to create time frames at http://labix.org/python-dateutil.' T9 z7 H: K, t! I7 w
( I" S$ \0 i! I1 }! Y
The info subcommand shows all the parameters which are covered by the crm report:( |$ A& d( `: @5 T S& I% L
$ |0 t5 H" u8 s% A" Ainfo
, n1 ^: V* {& E" \/ C3 kSource: live
/ r5 F/ V/ R* x, K7 U2 m! ]Period: 2012-01-12 14:10:56 - end+ A2 I# L$ O! C$ a# U
Nodes: alice7 Z g1 F, @6 @6 d+ S
Groups:4 L. t$ c5 w. y
Resources:
+ @# i, g y \. p. uCOPY2 y8 S6 t% [3 T7 Q
To limit crm report to certain parameters view the available options with the subcommand help.$ @4 f- p! B# v6 o+ X: y3 g
2 p* N; I4 G4 f( f A1 V& Y* C* _- B" z
To narrow down the level of detail, use the subcommand detail with a level:* z. E; B6 e) ~' A
8 S( Z5 @: Z2 E* d( w
detail 1
2 G' r! q) M n8 s7 fCOPY8 P$ L' |0 N$ a" B" n# L
The higher the number, the more detailed your report will be. Default is 0 (zero).
9 d+ H' f* L4 A; F+ h- v: ^( i# P, ^; e) u* p) v7 h
After you have set above parameters, use log to show the log messages.
+ ^+ X$ o r: _' k) L& }
5 ]7 H1 h! y# ~2 kTo display the last transition, use the following command:
# J1 ?, E- V. D4 a- t* n5 i& g4 A+ m M* p
transition -1
$ E# G' }& K2 B+ c5 yINFO: fetching new logs, please wait ...
$ \3 c9 ^& t L+ z+ BCOPY# W- M' {) _; n7 E o4 `
This command fetches the logs and runs dotty (from the graphviz package) to show the transition graph. The shell opens the log file which you can browse with the ↓ and ↑ cursor keys.
) Q `6 }; e4 _
9 [6 w( k5 R. T1 t3 t8 jIf you do not want to open the transition graph, use the nograph option:/ Y4 w$ o) ~& k( Z
, K3 s+ {$ {( ?, Q$ V7 w4 x% ~
transition -1 nograph7 f! W# V5 n3 ~, K1 E4 c/ B* x
|
|
发表于 2023-12-22 11:39:28