& ]% J# U. c! h, s* Q
pacemaker+corosync实现zabbix高可用集群
8 w1 \& m6 A: d: t8 R7 y; i; ? ! x- s' t# c+ z* z/ R9 E
4 s9 [8 X! s: m0 j - 一、pacemaker 是什么5 ]4 v4 a. ^ u+ E# n4 K0 e- ^
# x4 j% ], M# R- _& S5 A; n. u' V8 Z - 1.pacemaker 简单说明
& K* k5 O! k/ Z1 g; V7 v0 ` - 2.pacemaker 由来
! n0 t Z# P' _, o7 U - 二、pacemaker 特点4 |! e0 z4 f) G0 u" d6 Z
- 三、pacemaker 内部结构; U8 U* V! L/ V0 O$ u3 U; L
, a" p2 i7 L- G9 ]$ u- l" y6 C
- 1.群集组件说明:1 n% c$ Q, g e% g$ U, j
- 2.功能概述
3 Z, P" Q- I% w+ |# E1 w3 w* C w7 F - 四、centos6.x+pacemaker+corosync实现zabbix高可用- E D0 E' s9 h, L2 K; X3 j
{. ], q/ Z9 z7 W; s1 P4 \ - 1、环境说明
7 n0 X; }4 E* c% a$ G7 v - 五、安装pacemaker和corosync(各个节点均要运行)
0 _: w4 _; a) y7 n( { 3 u5 u. {- v' u5 _; ~; {3 x
- 1、前提条件各个节点完成主机解析工作。
% U3 g6 V; C$ b" L2 L- ^8 B - 2、各个节点的时间同步
( [" M! Z7 X" i; K' C- u - 3、各个节点完成互信工作
! B: |% C/ e( V- { - 4、关闭防火墙与SELinux6 m8 ^7 ?: z9 T. X! S
- 5、安装pacemaker+corosync+pcs; F* ~4 V) f2 ]- o7 o5 c
- 六、配置corosync# j7 a: Z% E$ }/ f
?9 F2 g5 |2 P* i- b; D' O8 M
- 1、设置变量
7 J6 o/ @3 _8 W) [% f" K - 2、更改corosync配置文件5 N; o5 ]3 M1 ?% h, j4 n9 V: d( [: m; k
- 3.生成密钥文件, X7 F, O: f. ^) z- P2 Z* D: `; S! N
- 七、安装和配置cman' N! [. O! t4 h
- 八、编辑cluster.conf
7 B: E, I( Y' H) a) ? - 九、检查配置文件并开机自启8 @. ?- m& i: X- C$ `
- 十、资源配置 t" U# p- H" s( L' X+ O
- 十一、验证- u, w: N j. } E- A) v) w
- 十二、常用命令' K' J/ x1 L- {; ?$ n
- 十三、zabbix启动脚本
2 o& ?- c# f. j4 ] : m9 [; V$ y& m( I0 {, d# {
, Z- R7 J3 R% q; k8 L
一、pacemaker 是什么 6 x& W4 [1 c" b5 N1 m" J
1.pacemaker 简单说明 9 D; r* Z0 j' G! U/ Q+ E
pacemaker(直译:心脏起搏器),是一个群集资源管理器。它实现最大可用性群集服务(亦称资源管理)的节点和资源级故障检测和恢复使用您的首选集群基础设施(OpenAIS的或Heaerbeat)提供的消息和成员能力。 7 E, O' a2 z$ ?9 U( x0 n$ I
它可以做乎任何规模的集群,并配备了一个强大的依赖模型,使管理员能够准确地表达群集资源之间的关系(包括顺序和位置)。几乎任何可以编写脚本,可以管理作为心脏起搏器集群的一部分。
/ @. c" s) w- p* |$ X& J0 i. ` 我再次说明一下,pacemaker是个资源管理器,不是提供心跳信息的,因为它似乎是一个普遍的误解,也是值得的。pacemaker是一个延续的CRM(亦称Heartbeat V2资源管理器),最初是为心跳,但已经成为独立的项目。 3 w) Z7 s- Q& D6 M P
2.pacemaker 由来
8 N; [% I( i! W4 M7 C大家都知道,Heartbeat 到了V3版本后,拆分为多个项目,其中pacemaker就是拆分出来的资源管理器。 + e/ L: Y; U1 q2 d1 S2 K; |
4 U; s/ P8 ?6 u4 A* ^ - Heartbeat 3.0拆分之后的组成部分:+ P [" S$ K8 R
- Heartbeat:将原来的消息通信层独立为heartbeat项目,新的heartbeat只负责维护集群各节点的信息以及它们之前通信;9 d/ G: X4 P* _0 y% N6 {5 [7 @
- Cluster Glue:相当于一个中间层,它用来将heartbeat和pacemaker关联起来,主要包含2个部分,即为LRM和STONITH。
% Y4 g3 l, T8 |( D& \) @ - Resource Agent:用来控制服务启停,监控服务状态的脚本集合,这些脚本将被LRM调用从而实现各种资源启动、停止、监控等等。% B$ M, t ^, q6 g5 ~ Z
- Pacemaker : 也就是Cluster Resource Manager (简称CRM),用来管理整个HA的控制中心,客户端通过pacemaker来配置管理监控整个集群。. }3 p) h3 r* o2 Z- w
+ U" E2 Z- B* D0 {/ Y1 v二、pacemaker 特点 " s* ?# T9 E( l* _, y6 y
, |; M4 E' o. L* M; U \+ ~% v s# ?' w
- 主机和应用程序级别的故障检测和恢复
" u% r( J# ~, j1 F - 几乎支持任何冗余配置& u, C' G! D2 S& j/ t" K5 L0 P& r6 V
- 同时支持多种集群配置模式, t/ b% W/ Q2 y5 p! Z
- 配置策略处理法定人数损失(多台机器失败时)
1 m. @" z0 |. p5 @4 o; }% G - 支持应用启动/关机顺序2 c! K6 {* F5 d$ ?
- 支持,必须/必须在同一台机器上运行的应用程序: l. O8 h* ]. L4 M5 E2 D
- 支持多种模式的应用程序(如主/从)
4 Q9 v3 ^$ u4 }* x+ Q - 可以测试任何故障或群集的群集状态" ]' C- j M) O8 ~# K Z T
3 E2 V' F: s! q* Q2 u- r4 S
m$ j4 W" ]* b9 u. y三、pacemaker 内部结构 3 Q0 U- u3 n( s/ U$ [/ m5 H! `
( X' i' P# t) X' m* T+ w
1.群集组件说明: 1 M* r5 v$ v6 F1 X. u) W* X
5 U, u( X$ D3 {8 p: ^ - stonithd:心跳系统。
% Q6 [2 L+ e5 @2 g8 ^- O - lrmd:本地资源管理守护进程。它提供了一个通用的接口支持的资源类型。直接调用资源代理(脚本)。# C( ]) z. ?) }: n/ m5 _6 h
- pengine:政策引擎。根据当前状态和配置集群计算的下一个状态。产生一个过渡图,包含行动和依赖关系的列表。
3 d, p! E# i. Y" i( R) l8 c& J - CIB:群集信息库。包含所有群集选项,节点,资源,他们彼此之间的关系和现状的定义。同步更新到所有群集节点。6 Q) M4 }9 T- z% f
- CRMD:集群资源管理守护进程。主要是消息代理的PEngine和LRM,还选举一个领导者(DC)统筹活动(包括启动/停止资源)的集群。
3 a% q: W! C+ H# Z - OpenAIS:OpenAIS的消息和成员层。' t3 ], v4 ~: U: A x7 ]/ H
- Heartbeat:心跳消息层,OpenAIS的一种替代。
4 C' L" q U7 l J( z2 c9 h" `# h - CCM:共识群集成员,心跳成员层。
! z7 X; m$ p# N2 M: `# a) X: Q - CMAN是红帽RHCS套件的核心部分,CCS是CMAN集群配置系统,配置cluster.conf,而cluster.conf其实就是openais的配置文件,通过CCS映射到openais。" r/ o5 j/ ^, Q M# ~5 _+ B
, E! A3 L k7 U" c0 g2.功能概述 5 x0 a! P) s2 }3 d2 E
CIB使用XML表示集群的集群中的所有资源的配置和当前状态。CIB的内容会被自动在整个集群中同步,使用PEngine计算集群的理想状态,生成指令列表,然后输送到DC(指定协调员)。Pacemaker 集群中所有节点选举的DC节点作为主决策节点。如果当选DC节点宕机,它会在所有的节点上, 迅速建立一个新的DC。DC将PEngine生成的策略,传递给其他节点上的LRMd(本地资源管理守护程序)或CRMD通过集群消息传递基础结构。当集群中有节点宕机,PEngine重新计算的理想策略。在某些情况下,可能有必要关闭节点,以保护共享数据或完整的资源回收。为此,Pacemaker配备了stonithd设备。STONITH可以将其它节点“爆头”,通常是实现与远程电源开关。Pacemaker会将STONITH设备,配置为资源保存在CIB中,使他们可以更容易地监测资源失败或宕机。
5 \7 E. `- G* m* X. s四、centos6.x+pacemaker+corosync实现zabbix高可用
! g5 q" }% ~/ I7 L s+ |1、环境说明
* m: n5 A" R1 y7 O7 w8 {$ {1 @OS:Centos 6.7x86_64 mini 2 y' J% u* e8 X
APP: Pacemaker 1.1.15 |, ]9 l7 g, ^8 e
LNMP+Zabbix 3.4.1 ) d1 ^6 ^& g+ H2 }/ M& B
corosync+pcs+cman - H' D5 b6 l m a9 `5 F7 V
IPADDR:vip-192.168.8.47/20
! b. ~7 W+ [- B zabbix01-192.168.8.61/20
, Z# W- H9 i) Y) N' Z7 _9 v, a% q zabbix02-192.168.8.63/20
% G! R# E5 o I+ a! ?# P7 B& A zabbixdb-192.168.8.120/20 " @& d2 K9 k' w4 ]2 D* S
: @% U0 N/ k, ]. u
PS:IP地址需要根据个人具体环境配置,VIP和zabbix要在同一网段。 ; L$ z* Z3 g% l% Y
拓扑结构
( R4 }) P0 w0 F* T1 r7 \& q
$ |7 l" D% E$ J/ ~( D- o# m' e0 JPS:接下来会直接介绍pacemaker和corosync的安装和配置,关于zabbix+LNMP环境的部分请参考之前发表的“zabbix3.2编译安装”或“zabbix高可用”两篇文章。 , h2 k. S9 o" i& z. c
五、安装pacemaker和corosync(各个节点均要运行) % ?. n0 R+ J8 G6 Y% |3 f
1、前提条件各个节点完成主机解析工作。
9 {: T4 @8 D' p( P8 |: yvim /etc/hosts 6 C- C C2 F6 {
# cat /etc/hosts
7 E3 ^0 k! Y# e6 n" @) i2 R# P {127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
* p" o4 K o6 @; M::1 localhost localhost.localdomain localhost6localhost6.localdomain6 % }$ B; w, q" C
192.168.8.61zabbix01.okooo.cn zabbix01 % t1 k7 n: k: O, F: q& b9 @. L1 B; D
192.168.8.63zabbix02.okooo.cn zabbix02
0 A+ `+ T* k( _- q192.168.8.120zbxdb.okooo.cn zbxdb - Z+ ]8 T9 ^7 ^: U& h- x, K
2、各个节点的时间同步 + X6 ?( b1 x. h5 K- T
ntpdate210.72.145.44
8 o# H" b+ X$ g& ^) w3、各个节点完成互信工作 # a7 ~- p1 M% l, q& h* b
ssh-keygen -t rsa -f ~/.ssh/id_rsa -P '' 0 [: D7 E8 h2 }8 h
ssh-copy-id -i.ssh/id_rsa.pub root@zabbix01/02/db.okooo.cn
) U1 S1 O4 i, ]& q3 L- O3 v4、关闭防火墙与SELinux % T. g4 @ ? u! O. b9 Z
# cat /etc/selinux/config d( Y! g- C0 p( [9 y
# This filecontrols the state of SELinux on the system.
# u; u5 \/ z8 a' @$ L4 _1 d4 \! ~6 @) s# SELINUX= cantake one of these three values: - u; k, s! P- f; Z" v/ y. _& `! q
# enforcing - SELinux security policy is enforced. ( [5 @; m' [1 S n+ H9 D9 V
# permissive - SELinux prints warnings instead of enforcing.
$ Q: }* z7 z! S8 G. r9 X# disabled - SELinux is fully disabled. % m1 g6 {) ^1 O$ F/ U
SELINUX=disabled 3 m8 D3 D$ j* h/ C, M% e
# SELINUXTYPE=type of policy in use. Possible values are: - K) Z! f7 O h: j
# targeted - Only targeted network daemons are protected. # J* s+ X h# L2 A; J
# strict - Full SELinux protection.
/ K5 b& |& H( l7 CSELINUXTYPE=targeted 3 ?' y/ `2 e8 o9 a( w5 d
: M) h/ G4 K, U- D) g
#/etc/init.d/iptables status 3 `# z; A0 b- `0 ?$ M
iptables:Firewall is not running.
% X( J' ?) t4 @0 [7 w7 E# A5、安装pacemaker+corosync+pcs
8 q7 U- r* C# |4 Oyum install -ypacemaker corosync pcs * u9 m8 s5 A- v6 \- |2 x$ j. A
六、配置corosync
+ B7 V" H7 f; W* @$ ^1、设置变量 ! o6 e+ V: ` R5 {2 z3 B' z
exportais_port=4000 $ ]. x" T& r1 W# m) J; }0 C/ s
exportais_mcast=226.94.1.1
8 @* z* f* l; H. @3 }exportais_addr=192.168.15.0 ; c: c7 y5 \9 D$ j2 D
env|grep ais
1 R% X% ~# S0 y/ P" d$ k2、更改corosync配置文件
; c$ V! o( Y. O) r! d! Z, U7 N4 C1 ?% Ecp /etc/corosync/corosync.conf.example/etc/corosync/corosync.conf
7 n0 F m; N- Psed -i.bak“s /.*mcastaddr:。* / mcastaddr:\ $ ais_mcast /g”/etc/corosync/corosync.conf
2 x4 W/ [$ M9 o/ @/ C2 q, R% Qsed -i.bak“s /.* mcastport:。* / mcastport:\ $ ais_port / g”/etc/corosync/corosync.conf
1 l" O% v% U% J5 Ased -i.bak“s /.* bindnetaddr:。* / bindnetaddr:\ $ ais_addr / g”/etc/corosync/corosync.confcat/etc/corosync/corosync.conf 0 W; Z6 o. g1 f9 b
# Please read the corosync.conf.5 manual page
9 @8 I H' Q0 }* ?compatibility: whitetank
+ h8 B" _1 U o' T, Ttotem {
4 q0 r1 `: C8 V( X4 {, A version: 2 , k$ e9 y& Z' t% s N
secauth: on #启动认证
% O1 A: J+ {- J! U; I4 Z threads: 2
8 `4 G! d! \) [! A interface { / v. C- v, T( {9 H0 X$ R$ a
ringnumber:0 - f& A3 Y: s, v. ]& |& |6 S
bindnetaddr:192.168.15.0 #修改心跳线网段 , O+ d2 m9 | I4 E
mcastaddr:226.94.1.1 #组播传播心跳信息 ( x6 \6 P" S* A* m9 G5 U' M+ @
mcastport:4000
2 r ^# F7 z- d8 z) `" g" a5 X+ v ttl:1 . g, q5 c3 E% Z
}
7 J6 F0 \1 I, h0 F}
. L. F. v% V X- y9 C; wlogging {
! m$ }3 U' f I6 y$ H. t( a& X9 U fileline: off . g7 f' Y) [" v
to_stderr: no * ~% D2 n, `+ K; m5 E7 [
to_logfile: yes ) _: T$ R/ k9 J
to_syslog: no
* z9 R2 |% @# Q logfile: /var/log/cluster/corosync.log #日志位置
# p4 M6 C) O! N5 r# k f debug: off $ T* n1 O( S7 p1 b2 N/ h1 f ~
timestamp: on 6 N+ }, z J+ s& O
logger_subsys {
$ ^8 |2 u( [% M$ R: K6 E subsys:AMF
4 @% |( y4 W! n3 {+ C" E6 ^ debug:off
$ a& }: K5 i) d8 b5 Y }
. H3 N' \3 ]. q1 W1 T/ J}
+ H. N, N: y0 z) famf { ' P9 w& a$ b% j- u* D" M
mode: disabled
7 u* |% u* l2 R5 e}
% j, }3 V" b! G; I#启用pacemaker ) G) l, f3 b1 A8 R2 V
service { & v1 W, S2 I* P8 s/ z1 U
ver: 0 : N: {+ h. n& e8 m$ M% _
name: pacemaker ; X$ X. M- m* ]! Z: w0 m
} % o$ g* N' _6 w% q( A
aisexec {
& U0 \$ J& Z* U9 b+ G9 f user: root
; l0 u o8 n" i" A& a6 t group: root
) `0 X b' H* o1 q( V2 V}
) P" A4 s# r5 d/ o1 i _% i注:用 mancorosync.conf 可以查看所有选项的意思。 " y5 s0 u$ b( a) \ n; x- ]
3.生成密钥文件
7 z* z# q& y" C- }1 m注:corosync生成key文件会默认调用/dev/random随机数设备,一旦系统中断的IRQS的随机数不够用,将会产生大量的等待时间,因此,为了节约时间,我们在生成key之前讲random替换成urandom,以便节约时间。 3 J. ?, C) Q% U, j
mv /dev/{random,random.bak} . q% B+ U1 D" i" ^; ?2 f
ln -s /dev/urandom /dev/random s, r/ S1 a4 l9 F8 H+ W; X
corosync-keygen - H+ V0 h/ C1 }9 Y i
PS:以上步骤要在所有节点上运行。
. B- o3 x) I5 P0 f4 Z! W七、安装和配置cman ; d2 r. h( o. p; y/ }
yuminstall -y cman
5 M. t8 z4 @# H" ~: d, T sed -i.sed"s/.*CMAN_QUORUM_TIMEOUT=.*/CMAN_QUORUM_TIMEOUT=0/g"/etc/sysconfig/cman. G" Z% Q! _/ h, b- N4 d3 ]7 H
# cat/etc/sysconfig/cman
/ G9 B, x# P( K#CMAN_CLUSTER_TIMEOUT -- amount of time to wait to join a cluster2 T0 V3 I( i& ]& o( G
# before givingup. If CMAN_CLUSTER_TIMEOUT is positive, then we will9 S& I6 i# v. {( J# N: A l
# waitCMAN_CLUSTER_TIMEOUT seconds before giving up and failing if
2 C3 C. f8 Q! j; H# we can't joina cluster. If CMAN_CLUSTER_TIMEOUT is zero, then we
3 G0 Q0 L' Z6 q) Y# will waitindefinitely for a cluster join. If CMAN_CLUSTER_TIMEOUT is
7 c# |5 G& f6 M9 l' d# negative, donot check to see if we have joined a cluster.
% V! P; s+ c+ l( s8 ?8 T; }1 I#CMAN_CLUSTER_TIMEOUT=5#CMAN_QUORUM_TIMEOUT -- amount of time to wait for a quorate cluster on
2 J/ c; o- L) R& i. E/ d# startup.Quorum is needed by many other applications, so we may as9 `3 v! A- L. d: u1 M
# well waithere. If CMAN_QUORUM_TIMEOUT is zero, quorum will0 ~! ^9 a! B" T; V6 j5 g: o
# be ignored.$ \/ k, C3 c" U/ T' n7 K3 {* X
CMAN_QUORUM_TIMEOUT=0#CMAN_SHUTDOWN_TIMEOUT -- amount of time to wait for cman to become a
, [, q! p I6 y6 Z S# cluster memberbefore calling 'cman_tool' leave during shutdown.
; f" C _1 `9 W7 x7 h# E0 M# The default is60 seconds( w7 P7 c% s6 f+ c, R0 q
#CMAN_SHUTDOWN_TIMEOUT=6#CMAN_NOTIFYD_START - control the startup behaviour for cmannotifyd,5 l. K. w7 Z/ z* l( M
# the variablecan take 3 values:
# o j% M+ e+ B+ J! P& p# yes | willalways start cmannotifyd
' N/ k! {0 U, |# no | willnever start cmannotifyd
% ~+ D8 n6 T9 ~0 u, t0 J. x# conditional(default) | will start cmannotifyd only if scriptlets: L! U4 o+ [5 V; M
# are found in/etc/cluster/cman-notify.d, L, Y7 R3 D( _6 V7 K: _
#CMAN_NOTIFYD_START=conditional#CMAN_SSHD_START -- control sshd startup behaviour,& t4 N" P& O# e) |+ d
# the variablecan take 2 values:
( K3 E$ ~9 U5 ^ u3 [ W# yes | cmanwill start sshd as early as possible
3 L$ f i0 V* }! c0 b0 ^# no (default) |cman will not start sshd
5 U& H' z; O% J9 l#CMAN_SSHD_START=no#DLM_CONTROLD_OPTS -- allow extra options to be passed to dlm_controld daemon.
) a @0 S) [& g/ P( i: o#DLM_CONTROLD_OPTS=""#Allow tuning of DLM kernel config.6 b- z/ C J7 F7 ~
# do NOT changeunless instructed to do so.+ J& ^/ i$ T' C: ~
#DLM_LKBTBL_SIZE=""# r4 x# s7 \* d7 G, H2 N3 e: t
#DLM_RSBTBL_SIZE=""
+ u( g+ E/ W* e& C#DLM_DIRTBL_SIZE=""9 x# g% j B" U) d2 T6 Y5 g
#DLM_TCP_PORT=""#FENCE_JOIN_TIMEOUT -- seconds to wait for fence domain join to
# u, f1 Y) }8 S' U2 b$ B$ @+ r# complete. Ifthe join hasn't completed in this time, fence_tool join! Y/ w' c5 `# G8 v9 p+ m+ J& _
# exits with anerror, and this script exits with an error. To wait
l1 b6 F, J: u2 l, f% u) M; ~5 A# indefinitelyset the value to -1.. u5 A& _6 [! v" X9 r0 @- Z
#FENCE_JOIN_TIMEOUT=20#FENCED_MEMBER_DELAY -- amount of time to delay fence_tool join to allow8 C" @6 @! N% a C
# all nodes incluster.conf to become cluster members. In seconds. l) a B0 B, F* U+ ^7 ~
#FENCED_MEMBER_DELAY=45#FENCE_JOIN -- boolean value used to control whether or not this node0 `- ^" P: [. p* p5 n- ]# ]
# should jointhe fence domain. If FENCE_JOIN is set to "no", then( X) l$ E4 [" T4 B6 {
# the scriptwill not attempt to the fence domain. If FENCE_JOIN is
# E/ U0 u* U; D. |" w5 x2 \# set to"yes", then the script will attempt to join the fence domain.
+ B# N* m C& M2 v1 h# If FENCE_JOINis set to any other value, the default behavior is
9 m) }! C* X! i# to join thefence domain (equivalent to "yes").' f- }+ [5 J$ F2 R5 x4 u' F/ Q* j
# When settingFENCE_JOIN to "no", it is important to also set" R0 n' `5 o, H
#DLM_CONTROLD_OPTS="-f0" (at least) for correct operation.
) R1 H4 u2 Y# t2 }2 e# Please notethat clusters without fencing are not
% u5 `- z( S& o' h% Z# supported byRed Hat except for MRG installations.
7 R6 ^ I" c. Z#FENCE_JOIN="yes"#FENCED_OPTS -- allow extra options to be passed to fence daemon.
( H1 M" t5 i4 R) |$ k4 l) A2 \#FENCED_OPTS=""#NETWORK_BRIDGE_SCRIPT -- script to use for xen network bridging.. N2 p4 K/ u1 q$ @
# This scriptmust exist in the /etc/xen/scripts directory.* w3 S h U2 o' O. o! X
# The defaultscript is "network-bridge".
( [& T2 ]/ ]3 L {5 |#NETWORK_BRIDGE_SCRIPT="network-bridge"#CLUSTERNAME -- override clustername as specified in cluster.conf
; F, ~- c3 t, k#CLUSTERNAME=""#NODENAME -- specify the nodename of this node. Default autodetected.
( r* F* O9 Y% N8 W& o3 |#NODENAME=""#CONFIG_LOADER -- select default config parser.
/ y/ v8 Q9 W+ B A# This can be:" Z3 T: S/ ] b2 n( u1 Z
# xmlconfig -read directly from cluster.conf and use ricci as default2 o8 g% a$ \( B8 O! H! o6 ]* f* S
# configpropagation method. (default)
3 d& A0 V2 R. K# T; N4 _# t#CONFIG_LOADER=xmlconfig#CONFIG_VALIDATION -- select default config validation behaviour.
; {2 J' B/ d5 w% n& r: `: l, T# This can be:
9 x) Q/ E. F; X( C# FAIL - Use avery strict checking. The config will not be loaded if there
# \4 g1 S4 ^. a# are any kindof warnings/errors
: H O- V0 I3 x$ Q6 s& k# WARN - Same asFAIL, but will allow the config to load (this is temporarily3 W0 {" v8 Y3 T0 C: O+ X+ \( A% @
# the defaultbehaviour); h" \3 ^8 F3 c' D* i
# NONE - Disableconfig validation. Highly discouraged& ~4 h% O" s% F! q
#CONFIG_VALIDATION=WARN#CMAN_LEAVE_OPTS -- allows extra options to be passed to cman_tool when leave5 j+ p/ A) D8 o* g! h0 H+ s. C5 e
# operation isperformed.) c& k/ ?% ~/ K, D
#CMAN_LEAVE_OPTS=""#INITLOGLEVEL -- select how verbose the init script should be.! d+ K' y: j6 E/ y% g
# Possiblevalues:$ C% a# G; X/ z# z" q
# quiet - onlyone line notification for start/stop operations
7 l/ x" L! A5 i! g. z; z$ F# terse(default) - show only required activity
6 h- P5 p |3 c1 A# full - showeverything
J# ?+ b+ A# J: a J5 }9 P#INITLOGLEVEL=terse
8 @6 p' J: k! e0 ~4 T: c八、编辑cluster.conf
5 j+ ?. T, C& ivim /etc/cluster/cluster.conf * p6 l7 @- |4 w
# cat /etc/cluster/cluster.conf" z/ w& t2 \( Z- M
<?xmlversion="1.0"?>
4 [0 d# p* [7 W8 M<clusterconfig_version="1" name="zabbixcluster">9 ~; P0 I. Y4 C* N: x
<loggingdebug="off"/>: ]+ D6 }/ v" E$ u4 |
<clusternodes>9 J- l9 s: d+ [ i1 J9 t" z# P
<clusternodename="zabbix02" nodeid="1">
* c0 s& z# ]! I, Z" a. d4 Z `<fence>1 Z7 o7 T: u( P' w C" Z
<methodname="pcmk-redirect">
7 k& O R% ?- ]/ u# U% B% {2 d<devicename="pcmk" port="zabbix02"/>1 `- O$ ]5 M$ p
</method>0 c. @3 B7 W$ C. O; _
</fence>
' J% E9 ?/ M+ u+ I: N7 h; t7 @" @" t</clusternode>
$ ?) V; Y4 g1 W* @2 R. l+ S<clusternodename="zabbix01" nodeid="2">
2 w6 H4 Y9 D7 A2 N$ i<fence>5 q, J; @) X4 Y
<methodname="pcmk-redirect">( T& }) \3 w' D; N2 E2 W* v
<devicename="pcmk" port="zabbix01"/>* G/ S! { V5 q9 |6 {, s
</method>, w2 e5 W# D9 @& }; b1 V# V' c) S
</fence>/ J7 h9 h1 D/ ^/ ^ l5 U0 T
</clusternode>
1 p0 h! v G; b</clusternodes>
0 r; m7 P$ ~3 T1 ~<fencedevices>' g' V3 ]3 \$ d- Z
<fencedevicename="pcmk" agent="fence_pcmk"/>* {8 K" v* q' l0 s) I, ~7 f
</fencedevices> J2 B9 s+ l& C9 @+ L6 o) z
<cman><multicast addr="226.94.1.1"/></cman> => when runningmore clusters in same network change multicast address
( M$ x" H; A1 E3 L" ^5 t+ w</cluster> ) Y: a# s3 e/ y8 e
九、检查配置文件并开机自启 1 G% C/ B0 H2 V; L6 E, r8 T" v
ccs_config_validate
& i6 w% i/ \+ e- t M6 j9 [service cman start
: V- n. V/ L! h0 \& P9 vcman_tool nodes
6 A8 S! i& I# b3 \service pacemaker start $ X1 r: k5 X: l- a
chkconfig cman on ( N3 H) D, c7 H/ [! u, x
chkconfig pacemaker on
, b7 `6 b& H$ ~; | * O- ?2 G( _, G5 ]! @( E
十、资源配置 ( Q: Y0 l; e- ]- n4 V
pcs cluster auth zabbix01 zabbix02 #节点间认证
4 b6 Q- J, A+ ~9 @2 G% epcs cluster start --all #启动集群中所有节点
5 |$ x, ?8 [- M. @! Q' ^ ^6 Vpcs resource create ClusterIP IPaddr2ip=192.168.8.47 cidr_netmask=32 op monitor interval=2s #创建一个名为ClusterIP,类型是IPadd2,VIP是192.168.8.47/32 每隔2秒检测一次的资源
7 ^, O' U. N6 D0 Npcs property set stonith-enabled=false #因我们没有STONITH设备,所以我们先关闭这个属性 2 i! {. p# x! l6 j4 c s3 ^
pcs resource create zabbix-serverlsb:zabbix_server op monitor interval=5s #创建一个名为zabbix-server、标准是lsb、应用是zabbis_server,每隔5秒检测一次的资源。lsb指/etc/init.d/下的启动脚本
0 y7 o+ N9 j; z/ i# s) Kpcs resource group add zabbix ClusterIPzabbix-server #将ClusterIP zabbix-server资源加入到zabbix资源组中 ) \/ G* X) N6 A( ]5 Z3 A
pcs property setno-quorum-policy="ignore" #忽略法定人数不足时进行仲裁 . D. ~$ W, w9 b9 o$ b
pcs property setdefault-resource-stickiness="100" #资源粘性为100
; u* W8 |% C: A6 I4 Ppcs constraint colocation addzabbix-server ClusterIP #资源共置 + v: Z- Z7 n5 C( X$ ]' L& \( t
pcs constraint order ClusterIP thenzabbix-server #资源限制,确保VIP和service在同一个节点上运行,而且VIP要在service之前完成。
# k+ Y* r9 E- R& N2 Y( Y" apcs constraint location ClusterIPprefers zabbix01 #ClusterIP更喜欢在zabbix01节点上,可用于节点故障恢复
z+ `) u9 @; J; _, c- mpcs constraint location zabbix-serverprefers zabbix01 #zabbix-server更喜欢在zabbix01节点上,可用于故障恢复
1 a z$ E3 g0 B+ B, q \- U, l十一、验证 0 Q5 T# C% n% p& F+ V
1、在zabbix01上操作停止zabbix_server 服务
0 n2 y, @1 V; z9 C$ }& Y
* a# j0 ?% Q8 Y# m/ N! T% IPS:集群可以保证服务的高可用 9 z; t( N& ?. z5 u" ]. J% ?
; ^ V7 V: K2 b; \
root@zabbix01:~
! L8 ? |- i3 G& c$ \0 }# pcs resource 0 t! t& e% {: N2 X/ X4 n% k
ResourceGroup: zabbix
' }# t/ V# q8 P! R9 Z ClusterIP (ocf::heartbeat:IPaddr2): Started zabbix01
& F& Z7 `) I" d+ D; N$ ^3 c) U zabbix-server (lsb:zabbix_server): Started zabbix01 / `# q/ \; b+ n+ Y R' X$ V
root@zabbix01:~ " B" y% @6 K. D- G; k
, U' F& W% }8 p
# ip a
1 t$ w9 B! T2 Y' J, P+ h6 J& a1: lo:<LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 6 k7 L% h: b4 Q. d5 ]+ x7 A" w- I
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 6 n! h6 q. k6 [+ @/ Z) G$ L3 c' o
inet 127.0.0.1/8 scope host lo
) K% f# F1 d% }4 g inet6 ::1/128 scope host / ?9 q5 L- W" G- z+ D/ T
valid_lft forever preferred_lft forever 6 ^1 `+ T+ J) z
2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000 8 G. z. e8 `- E7 y
link/ether 00:50:56:bb:68:49 brd ff:ff:ff:ff:ff:ff / o) }, d; l3 T+ S8 }+ t2 B" @
inet 192.168.8.61/20 brd 192.168.15.255 scope global eth0 " \. K1 ^: _* m- z# ~
inet 192.168.8.47/32 brd 192.168.15.255 scope global eth0
8 ~& D' P7 Y7 r! f inet6 fe80::250:56ff:febb:6849/64 scope link
$ O, M! R. G! D4 Y# J valid_lft forever preferred_lft forever ' z7 `, e7 i) h, u
PS:此时VIP和资源均在zabbix01上运行
/ a4 I% ]0 t o 4 S8 k+ m; B" u5 L5 j
# u" C8 y4 i. _root@zabbix01:~
# \- U# F3 u F, p% d% b# ip a
1 D5 T8 l- Q# z. N1 [$ C# ? W1: lo:<LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
# k3 _' S0 ^% h4 ^3 N2 A) P! W$ g link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
. D, V2 e; E2 ~+ G$ ^3 f) Y inet 127.0.0.1/8 scope host lo
% ?/ z' [/ N! e9 v inet6 ::1/128 scope host
) o/ m' l. M1 | k. _ valid_lft forever preferred_lft forever 5 u0 v+ T" \; p1 J# G! R9 f
2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000 3 g3 K) p- Z1 d$ V5 w, `' Q4 ?
link/ether 00:50:56:bb:68:49 brd ff:ff:ff:ff:ff:ff
- F6 U3 S: a+ f3 C. `* H$ K. d; o$ V inet 192.168.8.61/20 brd 192.168.15.255 scope global eth0
( Y, K1 K% o. R. z inet6 fe80::250:56ff:febb:6849/64 scope link . U8 X1 L* k( b
valid_lft forever preferred_lft forever 6 u/ M1 J, `. j5 O6 I V
4 [5 T) s5 X* @. d2 Broot@zabbix01:~
8 G+ E3 \! s2 n$ p# ssh zabbix02"pcs resource"
5 i! p; l A- l$ S4 R, E ResourceGroup: zabbix
) Y6 b) Z+ E' y5 G( K" j& I ClusterIP (ocf::heartbeat:IPaddr2): Started zabbix02
' U/ X g+ Y) g; M2 }/ { zabbix-server (lsb:zabbix_server): Started zabbix02
3 F7 N0 \1 W' N% _3 W$ N" g$ h. G & V* ~' E7 e2 W( |5 F$ h6 ` G
root@zabbix01:~
& ?# ?0 x% e" `) H7 X0 W2 r# pcs clusterstart zabbix01
@, O3 ~ N# G: m" r1 S2 |1 Rzabbix01:Starting Cluster... 2 G2 w5 I, J4 ?+ i
root@zabbix01:~
9 u4 D1 n* i" r0 @ F5 Z7 Y# ip a
" a8 d0 L- Q) O, ^0 J1: lo:<LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
9 _/ K S$ f2 r9 U5 K/ Y; @ link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3 v7 O+ O: ]/ p+ A3 Q. t4 w inet 127.0.0.1/8 scope host lo / K! {7 \* u6 m2 B5 c
inet6 ::1/128 scope host
3 B1 i# ]8 P9 y) V valid_lft forever preferred_lft forever ' M( J7 I$ S* P6 i
2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000 : }% |% S U3 \' C
link/ether 00:50:56:bb:68:49 brd ff:ff:ff:ff:ff:ff 0 S: X5 p, J# C) I' i' b
inet 192.168.8.61/20 brd 192.168.15.255 scope global eth0 $ R$ X: n; X6 R) X8 U
inet 192.168.8.47/32 brd 192.168.15.255 scope global eth0
8 s0 X1 k9 z8 d& q; u# n: ]$ J inet6 fe80::250:56ff:febb:6849/64 scope link - N$ @9 H' M8 ?6 s, [
valid_lft forever preferred_lft forever ! F4 @$ g; n, Y% F( @, F# a8 ~
root@zabbix01:~ 2 s& Z- Q/ K2 B9 i( r5 C
# pcs resource 9 s2 G0 B% x, f% F) ^/ v" ?. {. }6 e
ResourceGroup: zabbix
* j s; F! o. {- k ClusterIP (ocf::heartbeat:IPaddr2): Started zabbix01 + B6 \1 b2 ?+ J: i* D+ l1 u; A* h' G
zabbix-server (lsb:zabbix_server): Started zabbix01
5 Y4 `! R- v* Y! x" f! K, T9 |- c' |# V1 j
! I( g3 y) D* X6 n; ]; w 2 h* d. a }0 r
PS:重启集群后资源和VIP均被zabbix02接管,但zabbix01恢复后资源与VIP又再次回到zabbix01.
! h+ g- ]3 d m4 w) ^" J; f3 M+ Z 9 v& Y) [- _5 f- C
十二、常用命令 6 E. w& y: T; k1 a5 n/ m$ b7 K: e
1、查看集群状态
1 H( A8 x4 j3 w& D, ^1 u# pcs cluster status: p. F2 f- N1 S* L' ]: C
Cluster Status:, n7 T5 C, r5 c
Stack: cman
6 P: x9 Z, P/ i4 |0 CCurrent DC:zabbix01 (version 1.1.15-5.el6-e174ec8) - partition with quorum7 Z4 F4 h7 W8 b9 N D; S
Last updated:Thu Sep 21 02:13:20 2017 Last change: Wed Sep 20 09:13:10 2017 by root viacibadmin on zabbix01( Q. [5 | I6 d; ]( k. d( }! v
2 nodes and 2resources configured
! C0 }- I7 N: FPCSD Status:" _* q$ f) ` r* l r( c9 Y
zabbix01: Online- r/ j, E# d: n3 j o+ Y: ]
zabbix02: Online
4 F7 _! Z4 k- e) i$ x2、查看配置 9 V+ m/ B$ J8 q5 G
# pcs config show4 d- T( l* f2 v% J
Cluster Name:zabbixcluster$ p1 P, @+ R# s1 D5 F7 Y, w
Corosync Nodes:
b( ~# V3 [, }2 g% o( Kzabbix02zabbix01- h' S$ S1 S1 R& l
Pacemaker Nodes:4 a0 V$ X0 i% X9 M: C6 t+ i
zabbix01zabbix02 3 E% F6 _, C4 g* E
Resources:& e$ D$ {3 o9 Y1 G( D$ w( \/ u
Group: zabbix
% ~: Y* N# W9 w% PResource:ClusterIP (class=ocf provider=heartbeat type=IPaddr2) i6 D$ Y: e& ?2 V1 k3 U6 X; X( B+ R
Attributes:ip=192.168.8.47 cidr_netmask=32* Q. I4 k Y+ s2 I5 u$ Z; t) Z
Operations:start interval=0s timeout=20s (ClusterIP-start-interval-0s)
/ |3 N! R7 E1 Z* q' ~' h- Sstop interval=0stimeout=20s (ClusterIP-stop-interval-0s)
2 W- u5 r) A; ?monitorinterval=2s (ClusterIP-monitor-interval-2s)$ d# r6 {, v- h* G6 @
Resource:zabbix-server (class=lsb type=zabbix_server)! H" J% ^3 P5 R. d0 D1 g$ s
Operations:start interval=0s timeout=15 (zabbix-server-start-interval-0s)$ ~7 i* _. a, [- F
stop interval=0stimeout=15 (zabbix-server-stop-interval-0s)4 S4 K/ z8 k+ W7 w- z
monitorinterval=5s (zabbix-server-monitor-interval-5s)
3 o; d/ \; L0 QStonith Devices:
, o' W9 O, R* C2 E5 iFencing Levels: ( N* i' D5 {. C, }
Location Constraints:
0 q- z! F; V+ v0 ^# M6 K2 `! FOrderingConstraints:) H3 o. Q E+ V/ A
ColocationConstraints:+ O4 ?( z( F% k& `
TicketConstraints:
4 x% a4 v1 m% V4 m$ B% RAlerts:
* q1 H/ h; W# v: H" Q' u$ yNo alertsdefined : _: x3 Q# z) R& |# Q# j; R4 r
Resources Defaults:( K# V8 X+ e; Q! |: X, |. X# `
No defaults set( f8 m4 f6 \. A' r. f, `
OperationsDefaults:/ }/ _+ s# o* ]2 j+ `, g
timeout: 60s " E7 T$ R/ t+ B% a8 K( d3 z
Cluster Properties:. O4 ^7 h% I8 j9 q& L1 O
cluster-infrastructure:cman$ O. |1 f( u* i( O/ |- T/ {
dc-version:1.1.15-5.el6-e174ec8+ n5 V/ w# k- v, p" ^6 d3 @3 P
default-resource-stickiness:100/ V/ [. d/ w4 U, A9 r0 p
have-watchdog:false0 b3 c7 H3 @9 p& m: D# }0 f
last-lrm-refresh:1505857479
! o* B1 b; ~9 fno-quorum-policy:ignore% p( Y# ?* O! t. u3 Y3 P
stonith-enabled:false / t- {+ C+ e' R" J, d
6 P2 Y; b- Q- L) e% V' p# `
# pcs clustercib
`/ x0 E# Z, b3 s6 G( ?# O6 i6 l<cibcrm_feature_set="3.0.10" validate-with="pacemaker-2.5"epoch="82" num_updates="182" admin_epoch="0"cib-last-written="Thu Sep 21 02:56:11 2017"update-origin="zabbix01" update-client="cibadmin"update-user="root" have-quorum="1" dc-uuid="zabbix02">
: D c9 L6 S% P% ^: s0 \! n <configuration> , ]4 O! B8 T' `/ R' [; F" J5 Y
<crm_config>
: v$ s2 _! C6 g5 {, n# p <cluster_property_set id="cib-bootstrap-options">
, Y7 B# n0 X% J0 P5 X <nvpair id="cib-bootstrap-options-have-watchdog"name="have-watchdog" value="false"/> 3 m$ c$ l, Q3 Q+ ^4 }
<nvpair id="cib-bootstrap-options-dc-version"name="dc-version" value="1.1.15-5.el6-e174ec8"/> R9 O5 V4 s$ `, v3 |
<nvpair id="cib-bootstrap-options-cluster-infrastructure"name="cluster-infrastructure" value="cman"/> . ]/ `6 S' O' [# p% \1 P+ R
<nvpair id="cib-bootstrap-options-stonith-enabled"name="stonith-enabled" value="false"/> : f) _/ R- M( ?
<nvpair id="cib-bootstrap-options-no-quorum-policy"name="no-quorum-policy" value="ignore"/>
" K9 z6 s! b( j% Z2 l7 { <nvpairid="cib-bootstrap-options-default-resource-stickiness"name="default-resource-stickiness" value="100"/> , H1 ^2 l: a7 z2 F9 D% G* P4 D
<nvpair id="cib-bootstrap-options-last-lrm-refresh"name="last-lrm-refresh" value="1505857479"/>
, s% q& N5 K' o8 h% | </cluster_property_set>
9 d$ B5 e2 C: c </crm_config>
% E* g3 `" y! N5 n4 \ <nodes> * n" r' ?0 L% r9 y0 z- y5 I
<node id="zabbix02" uname="zabbix02"/>
) B( g! f6 J4 \/ Q7 g; p$ C/ u( v <node id="zabbix01" uname="zabbix01"/>
6 C$ P& }& `4 t/ C! A </nodes>
6 Q( U( \8 d7 W& x; z4 c& M <resources> 0 x. N5 A! L: V* B" R: V$ l
<group id="zabbix"> ' a- w# e/ Z/ h# p( F+ I
<primitive class="ocf" id="ClusterIP"provider="heartbeat" type="IPaddr2"> ( L2 y1 ?8 v& A
<instance_attributesid="ClusterIP-instance_attributes"> : B. v* O$ s7 ]/ p" u6 T& C
<nvpairid="ClusterIP-instance_attributes-ip" name="ip"value="192.168.8.47"/> 0 F+ K4 U& A3 B2 ~% @5 U9 H+ Z0 k3 f
<nvpair id="ClusterIP-instance_attributes-cidr_netmask"name="cidr_netmask" value="32"/>
& w3 P* s7 [: o8 ?$ a </instance_attributes>
8 m* b/ \1 z3 `+ v+ O/ o7 D <operations>
2 C; N& C, l( W" R, n6 ^, W <op id="ClusterIP-start-interval-0s"interval="0s" name="start" timeout="20s"/> , u8 s7 u" h" ?7 n% V: \
<op id="ClusterIP-stop-interval-0s"interval="0s" name="stop" timeout="20s"/> 8 v5 h2 g+ o& b; o* u5 o6 _
<op id="ClusterIP-monitor-interval-2s"interval="2s" name="monitor"/>
& c: d+ Z% w- k# l1 l$ x3 V1 t4 j </operations>
/ x1 o5 T) e0 ]5 U9 c5 n <meta_attributesid="ClusterIP-meta_attributes"/>
( {) q( e w$ Q5 G: U! h. F5 H </primitive> 5 K" U% P4 o2 U9 M) B8 H
<primitive class="lsb" id="zabbix-server"type="zabbix_server"> : }7 k$ h+ S4 W$ N8 M$ ~/ t
<instance_attributesid="zabbix-server-instance_attributes"/> ( k( X! x# }1 o; R; A9 X+ J# U
<operations> $ x# s) X/ j7 F
<opid="zabbix-server-start-interval-0s" interval="0s"name="start" timeout="15"/>
% Y% V4 N% B$ ]4 R6 }/ L0 S <opid="zabbix-server-stop-interval-0s" interval="0s"name="stop" timeout="15"/>
" \" V2 [+ U% a& o1 B- j' F/ ^7 T$ u <opid="zabbix-server-monitor-interval-5s" interval="5s"name="monitor"/>
& S) c1 S0 C8 B5 m, u+ A1 k" J </operations>
2 C0 w) H' w, ? <meta_attributes id="zabbix-server-meta_attributes"/>
, {/ ~9 r8 b; e8 E </primitive> 9 E8 G8 l: \! n8 u/ B
</group> 1 _% `' H8 P8 F
</resources>
( k* C I+ j1 b! Y <constraints> {: u% ?, ]; h
<rsc_colocationid="colocation-zabbix-server-ClusterIP-INFINITY"rsc="zabbix-server" score="INFINITY"with-rsc="ClusterIP"/> % ^' X& r& [) S# q: E
<rsc_order first="ClusterIP" first-action="start"id="order-ClusterIP-zabbix-server-mandatory"then="zabbix-server" then-action="start"/>
; r. N; f4 E8 ?/ i <rsc_location id="location-ClusterIP-zabbix01-INFINITY"node="zabbix01" rsc="ClusterIP"score="INFINITY"/>
2 k. ~0 U7 V |, l0 T <rsc_location id="location-zabbix-server-zabbix01-INFINITY"node="zabbix01" rsc="zabbix-server"score="INFINITY"/> 7 K1 @! _( V( C! B8 E! w
</constraints> & Y0 M# \6 j S# e8 r. n* k Z
<op_defaults> ) }9 `+ a6 V$ l
<meta_attributes id="op_defaults-options">
( H( f7 _% i1 Z# J/ Y3 L3 h2 ^6 j- x <nvpair id="op_defaults-options-timeout"name="timeout" value="60s"/> : v, z0 B3 |: X/ y1 c
</meta_attributes> - s: A! W {" C' e0 x* m7 [8 v
</op_defaults>
& u$ H1 j1 F; _$ m6 G" \# \ </configuration>
_6 c3 t. ]' {: G! e0 P <status>
- g: t0 m' u$ @ x' {- G <node_state id="zabbix01" uname="zabbix01"in_ccm="true" crmd="online"crm-debug-origin="do_update_resource" join="member"expected="member"> 4 e- k( V% } I2 U# C. ]9 ^
<lrm id="zabbix01">
: T1 V* a5 L k% l& \5 M$ Z) K; O( Z$ A <lrm_resources>
. n8 L5 ?0 x* d <lrm_resource id="zabbix-server"type="zabbix_server" class="lsb">
" k" {, E: q* `5 c <lrm_rsc_op id="zabbix-server_last_0"operation_key="zabbix-server_start_0" operation="start"crm-debug-origin="do_update_resource" crm_feature_set="3.0.10"transition-key="10:56:0:76f2a258-6d10-4819-8063-54875d8f896a"transition-magic="0:0;10:56:0:76f2a258-6d10-4819-8063-54875d8f896a"on_node="zabbix01" call-id="12" rc-code="0"op-status="0" interval="0" last-run="1505940302"last-rc-change="1505940302" exec-time="44"queue-time="0"op-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/>
) l6 T1 q: B0 i2 n' z8 Y <lrm_rsc_opid="zabbix-server_monitor_5000"operation_key="zabbix-server_monitor_5000"operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.10"transition-key="11:56:0:76f2a258-6d10-4819-8063-54875d8f896a"transition-magic="0:0;11:56:0:76f2a258-6d10-4819-8063-54875d8f896a"on_node="zabbix01" call-id="13" rc-code="0"op-status="0" interval="5000"last-rc-change="1505940302" exec-time="27"queue-time="1"op-digest="873ed4f07792aa8ff18f3254244675ea"/> 2 |2 j$ {- G9 A/ q2 @+ C& g
</lrm_resource> & @1 a4 ~* u+ ?+ a- I
<lrm_resource id="ClusterIP"type="IPaddr2" class="ocf"provider="heartbeat">
# H) T2 i) _6 T8 I/ f8 ? <lrm_rsc_op id="ClusterIP_last_0"operation_key="ClusterIP_start_0" operation="start"crm-debug-origin="do_update_resource"crm_feature_set="3.0.10"transition-key="7:56:0:76f2a258-6d10-4819-8063-54875d8f896a"transition-magic="0:0;7:56:0:76f2a258-6d10-4819-8063-54875d8f896a"on_node="zabbix01" call-id="10" rc-code="0"op-status="0" interval="0" last-run="1505940302"last-rc-change="1505940302" exec-time="117"queue-time="0"op-digest="31f0afcca8f5f3ffa441c421e2c65fe1"/>
0 V" A9 b) j$ T" l$ E <lrm_rsc_op id="ClusterIP_monitor_2000"operation_key="ClusterIP_monitor_2000" operation="monitor"crm-debug-origin="do_update_resource"crm_feature_set="3.0.10"transition-key="8:56:0:76f2a258-6d10-4819-8063-54875d8f896a"transition-magic="0:0;8:56:0:76f2a258-6d10-4819-8063-54875d8f896a" on_node="zabbix01"call-id="11" rc-code="0" op-status="0"interval="2000" last-rc-change="1505940302"exec-time="160" queue-time="0"op-digest="e05b48764ef09ebc8e21989b06afa924"/> 2 @3 P1 R3 O) _; F7 |' {
</lrm_resource> : {( m N" m# Q7 Z. J* f( H
</lrm_resources> ( \5 ?0 I6 L/ a2 ~- R- j" f3 k
</lrm>
2 }8 x' W8 A& |: N8 A& S <transient_attributes id="zabbix01"> : U* c b. F1 E: D w7 ]4 t
<instance_attributes id="status-zabbix01">
. B' u5 L" {2 Y2 u2 T8 m <nvpair id="status-zabbix01-shutdown"name="shutdown" value="0"/> ! X5 S0 Q: ]' L% K
</instance_attributes>
?% G+ t* Z2 o6 t$ T5 W </transient_attributes>
; x/ {1 Y U* u$ h# @5 O- U N1 b3 e </node_state>
" b# d; w7 }; D: ^+ s <node_state id="zabbix02" uname="zabbix02"crmd="online" crm-debug-origin="do_update_resource"in_ccm="true" join="member" expected="member"> ! B! W) ~5 h6 j* T7 C, N
<transient_attributes id="zabbix02">
5 y( u" p3 U. b& l! |4 @$ i <instance_attributes id="status-zabbix02">
& z. Q. f# i& y& D( G5 T <nvpair id="status-zabbix02-shutdown"name="shutdown" value="0"/>
7 y, I) p; A, h2 {8 H% o </instance_attributes> & F8 g- ?8 s Q0 d8 G2 }
</transient_attributes> 2 j8 O. z4 J/ J, n. h
<lrm id="zabbix02">
/ e( s. _, d% T: S" g, o <lrm_resources> 6 _% t$ X1 [8 C8 @5 d3 |4 B
<lrm_resource id="zabbix-server"type="zabbix_server" class="lsb"> : ^" y7 \; K3 I" R
<lrm_rsc_op id="zabbix-server_last_0"operation_key="zabbix-server_stop_0" operation="stop"crm-debug-origin="do_update_resource"crm_feature_set="3.0.10"transition-key="9:56:0:76f2a258-6d10-4819-8063-54875d8f896a"transition-magic="0:0;9:56:0:76f2a258-6d10-4819-8063-54875d8f896a"on_node="zabbix02" call-id="39" rc-code="0"op-status="0" interval="0" last-run="1505940302"last-rc-change="1505940302" exec-time="50"queue-time="0"op-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/>
# h6 v! B- v; d: b <lrm_rsc_op id="zabbix-server_monitor_5000"operation_key="zabbix-server_monitor_5000"operation="monitor" crm-debug-origin="build_active_RAs"crm_feature_set="3.0.10"transition-key="9:53:0:76f2a258-6d10-4819-8063-54875d8f896a" transition-magic="0:0;9:53:0:76f2a258-6d10-4819-8063-54875d8f896a"on_node="zabbix02" call-id="37" rc-code="0"op-status="0" interval="5000"last-rc-change="1505940185" exec-time="17"queue-time="0" op-digest="873ed4f07792aa8ff18f3254244675ea"/>
* Z( q- ], s! P. s$ P </lrm_resource> * ?% r3 U7 @# {1 G
<lrm_resource id="ClusterIP" type="IPaddr2"class="ocf" provider="heartbeat"> : ^; g# w9 o% |4 D
<lrm_rsc_op id="ClusterIP_last_0"operation_key="ClusterIP_stop_0" operation="stop"crm-debug-origin="do_update_resource"crm_feature_set="3.0.10" transition-key="6:56:0:76f2a258-6d10-4819-8063-54875d8f896a"transition-magic="0:0;6:56:0:76f2a258-6d10-4819-8063-54875d8f896a"on_node="zabbix02" call-id="41" rc-code="0"op-status="0" interval="0" last-run="1505940302"last-rc-change="1505940302" exec-time="93"queue-time="0" op-digest="31f0afcca8f5f3ffa441c421e2c65fe1"/>
# c$ e5 r" D2 I <lrm_rsc_opid="ClusterIP_monitor_2000"operation_key="ClusterIP_monitor_2000" operation="monitor"crm-debug-origin="build_active_RAs" crm_feature_set="3.0.10"transition-key="6:53:0:76f2a258-6d10-4819-8063-54875d8f896a"transition-magic="0:0;6:53:0:76f2a258-6d10-4819-8063-54875d8f896a"on_node="zabbix02" call-id="35" rc-code="0"op-status="0" interval="2000"last-rc-change="1505940185" exec-time="118" queue-time="0"op-digest="e05b48764ef09ebc8e21989b06afa924"/>
* [. w3 n: Y: C) R# Z </lrm_resource> * j# u, g/ m/ S% p; N
</lrm_resources> 3 ~# ]6 ^% [- u' Y( M! Q) F, {
</lrm>
3 S" b: t* w7 x( v </node_state>
0 p0 [/ v9 t% {: B+ Y$ G. r' Q3 ?- Y </status>
8 ]' ^! L% U7 K9 i- b+ Q4 \</cib> ; H2 t; q* |6 D7 \6 q5 f) B
3、查看资源
& ?' W' U3 T! F$ d `# G- C# pcs resource
; E, c( S' z3 [8 I& d! j- s5 {Resource Group:zabbix
! H, {! N0 x+ |) }& D7 _ClusterIP (ocf::heartbeat:IPaddr2):Started zabbix01
$ r# j( N; s3 P% yzabbix-server ( lsb:zabbix_server):Started zabbix01
e4 q U# |: t2 n4、查看资源组 ! f2 f5 u" C3 N% u( n9 H
#pcs resourcegroup list
4 v8 k, D5 `" szabbix:ClusterIP zabbix-server
6 a5 w' i* ]2 i/ D {8 F4 g8 n 0 `& c$ {6 _( G8 M' @) V
十三、zabbix启动脚本
4 [6 Q0 k& g7 |0 C# cat /etc/init.d/zabbix_server o* K2 k3 T* m! V% G$ u
#!/bin/bash3 C& {, _* G: U" R5 L& e6 O7 ~
#Location ofzabbix binary. Change path as neccessary
% k" f: S% F4 @ { d: u uDAEMON=/usr/local/zabbix/sbin/zabbix_server
) ^( J" H5 t$ S/ @; S4 Y. bNAME=`basename$DAEMON`
& S" d' F6 u1 H! p: c! j; @#Pid file ofzabbix, should be matched with pid directive in nginx config file.
9 }7 j" U, v( c, q8 EPIDFILE=/tmp/$NAME.pid
+ i0 l5 E0 _* A* N Z#this filelocation: M# w$ G6 K) V! @( t
SCRIPTNAME=/etc/init.d/$NAME
& L* T' P) W, v: l#only run ifbinary can be found
V3 m8 U( T0 K+ d, C% ktest -x $DAEMON|| exit 0; ~* L; l ]- o4 ]) h+ w+ O$ Y
RETVAL=0 3 Y: G/ }! B9 A- y: [
start() {: h3 S( i4 v8 E: l3 Q) z0 j6 H7 k
echo$"Starting $NAME"
% d& Q1 R$ d" P$ U; ? F/ B$DAEMON; {7 d- h: H% [/ \% a @% Z- Z
RETVAL=0
! R3 o. O5 y! P* M7 p' U} 5 W7 L# a3 }9 K, R6 Y
stop() {
/ ?) m/ P# X Y7 P% hecho$"Graceful stopping $NAME" r9 W' R3 x: x
[ -s"$PIDFILE" ] && kill -QUIT `cat $PIDFILE`8 I% X0 Z1 ^1 N4 w
RETVAL=0
; L; B+ p" i& `* x% a} , a/ b. \2 f! v+ s; c
forcestop() {* W a9 O; ]. e! h$ b5 ]
echo$"Quick stopping $NAME"
0 O- M% j: ^" q9 m0 e( v+ Z[ -s"$PIDFILE" ] && kill -TERM `cat $PIDFILE`& q7 V- R2 n. o, p6 M/ N
RETVAL=$?
+ n3 F7 s3 Z2 V! j3 `$ V} 8 d W2 Y, D: `- q. p1 r7 {
reload() {6 Z+ \/ `8 _: c" Y9 D* z$ q
echo$"Graceful reloading $NAME configuration"
" z2 e2 f. C$ c! j0 U) u[ -s"$PIDFILE" ] && kill -HUP `cat $PIDFILE`0 Z+ I; ^, v4 ~+ Y" \
RETVAL=$?
1 p) j8 l3 Q/ H$ N) A$ O} 1 G# g; f$ ?: P2 g- n: F% m
status() {3 m/ B0 `/ r# c! [1 u' T
if [ -s $PIDFILE]; then9 `, U6 K+ ^4 N$ V! U
echo $"$NAMEis running."6 m, r0 r+ j! S4 E6 e- |3 F6 l! U
RETVAL=03 _7 ^% |$ R/ \3 @% y# i6 o
else
. f5 x" X- u1 h8 P* v' Y# `) h# \' @echo$"$NAME stopped."! T; P$ f& M4 ?6 m9 x3 d
RETVAL=3- x7 r' M. _) u- m5 Q0 Q3 i
fi, e( v: o2 t7 f$ ~- i- ?
}
0 w4 f9 y9 G# V( a( F# See how wewere called.. ] m) p5 N- j [" A
case"$1" in. Z% g8 A/ `. N4 }% W
start)0 ~3 I. C5 u5 p( ~
start9 B* C3 C( y, g
;;
u1 R% B/ y* _1 D/ R7 C6 Vstop)" C4 o( Q% ^) m
stop
6 U: F) a. h& s% }4 v5 G;;
0 @6 e+ z& d6 k4 y0 c6 J* Uforce-stop)
0 J1 N# u5 B2 z0 `5 cforcestop5 Y9 U- j! F. W! B3 I. S
;;
6 L- \, l) v; ^0 r' wrestart)
7 S9 H& \! r' k/ Y$ q# Istop; u9 U: t5 U5 ~+ a
start- n C+ v6 y3 t6 W2 \0 R. T
;;0 M& O/ S* L1 W9 E7 u
reload)
1 J, v ]1 I4 R& K; y: E1 Sreload8 @# p1 R' q8 g9 ~8 K- L# M
;;/ d# D" z+ F- k6 H" n9 n( ]
status)
% f5 x+ G$ |7 s6 e3 a4 {status& k( R* G0 x; e1 L( |, g, f
;;
0 `- D7 s0 P- ~* `" d*)
# _" Q6 n [6 s' _4 t0 gecho$"Usage: $0 {start|stop|force-stop|restart|reload|status}"
- R3 y( w5 [6 A! o( Bexit 1
1 d1 X# L0 b3 B; z& @, r" E& [7 Sesac
% B z" C# z8 H1 A5 H5 Jexit $RETVAL
4 |4 o& u# r9 p</pre>
: z, G2 m) }1 @( @5 @( R, V A- e , \' T- U" j! s( |" ^; A
|