|
edis和docker这两个词语会自动被修改为首字母大写并链接到知识库，所以在这里先写一遍，后面就不会被改写了。 0、具体操作见 （vmware中搭建k8s），virtulbox也是相同的流程。! a+ {" y8 z. v7 g6 a
1、学习k8s，读的是这本书《KUBERNETES权威指南 从DOCKET到KURBERNETES实践全接触.pdf》 2、这边书刚开始讲的是在单机上搭建一个k8s的hello world，用的是centos7. 于是我也在virtulbox中安装了centos7，并成功运行hello world。 3、然后，开始尝试集群了。在网上找了个教程，比较简单，很快就成功了，kubectl get nodes能看到各node了。 但是，应用跑的不正常，有的节点可以访问，有的节点不可以。而且从不同节点访问，查到的数据不相同，似乎是多个独立的系统。怀疑是iptables中cluster ip的规则有问题。
5 `: { A6 }5 [7 U- N/ l' e7 Z 接着，集群坏了，k8s的基础服务都启动不了。怀疑是不是因为我创建这些虚机时，用的是链接式拷贝，是不是原始的虚机安装了其它软件，导致k8s集群启动不了。 4、删掉重来，用完全拷贝的方式建立虚机。 一切正常，但是在启动redis-master-controller.yaml时，docker中无法建立容器， 用kubectl describe pod redis-master命令排查，发现下拉不了镜像。其实这些镜像已经存在于docker中了（搭建单机k8s时，自动pull的）。只是镜像名字前面多加了docker.io/ 我修改了redis-master-controller.yaml中镜像的名字，仍然pull失败。奇怪。 5、今天定位了网络问题（见virtualbox虚拟机无法上网），网络搞通后，hello world终于正常运行了。 之前的一些疑问，有答案了 1、随便访问哪个node的ip（比如http://192.168.56.251:30001/,http://192.168.56.252:30001/）(http://192.168.56.250:30001/是不行的)，都可以访问到服务的（proxy自动转的） 2、在内部时，访问真实端口也是可以的。 [root@centm ~]# kubectl get svc
& u. s, z! T% S5 h% KNAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE0 S/ s F( B' m
frontend 10.254.218.57 <nodes> 80/TCP 5m
+ Z b7 G) m) {: j3 D! ]5 f* ]kubernetes 10.254.0.1 <none> 443/TCP 15d
! S+ u2 H( W2 Q) S6 i0 n+ kredis-master 10.254.142.174 <none> 6379/TCP 8m! h- p" F# C- O$ R" P4 [; k7 x
redis-slave 10.254.201.123 <none> 6379/TCP 6m% l8 \! }1 a& z9 a- [
curl 10.254.218.57:80 可以通。 ping 10.254.218.57 不通 3、node中多了一个127.0.0.1 ，不知道为什么$ o& n, i8 ^4 J8 M# p
[root@centm ~]# kubectl get nodes; [9 ~- g6 h( N: c. L" @( `0 o
NAME STATUS AGE
5 N7 n/ G5 q) I4 h3 J127.0.0.1 NotReady 15d
7 q6 ]+ U. k1 `( y5 a k- hcents1 Ready 1d* F0 C* ?* Y7 R% }
cents2 Ready 1d- L7 n2 p; ?8 F$ h, s& p
[root@centm ~]# ps -ef|grep kube
8 c( _: r k) h1 S3 Y0 G1 Qkube 578 1 0 Jan20 ? 00:15:55 /usr/bin/kube-controller-manager --logtostderr=true --v=0 --master=http://127.0.0.1:8080
1 U3 n7 R& O; {* pkube 588 1 0 Jan20 ? 00:01:09 /usr/bin/kube-scheduler --logtostderr=true --v=0 --master=http://127.0.0.1:8080
4 r5 r. i( C2 M# Ykube 2079 1 0 Jan20 ? 00:08:11 /usr/bin/kube-apiserver --logtostderr=true --v=0 --etcd-servers=http://127.0.0.1:2379 --insecure-bind-address=0.0.0.0 --allow-privileged=false --service-cluster-ip-range=10.254.0.0/16 --admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota% S$ m4 i- E+ N: `1 I J
4、clusterip是一个虚ip，实际是iptables中的几个转发规则。 [root@cents2 ~]# ip a
1 r2 a) f; I0 ]% H" G; X1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1( y3 U+ ^& h1 }
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
/ o% z# u! F4 M5 y8 c2 ]# c inet 127.0.0.1/8 scope host lo4 r8 g% u. G* j8 v1 d; z6 Y" O$ {
valid_lft forever preferred_lft forever2 v8 b0 a, X) a
inet6 ::1/128 scope host+ F& Z: ]- i' `) v& N6 R+ N A
valid_lft forever preferred_lft forever
) ~, D7 f- i3 y8 p9 M2 z3 S+ z* l0 |2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
+ N- }' z' g9 a# M- K5 }& |/ B. Y link/ether 08:00:27:58:5d:6e brd ff:ff:ff:ff:ff:ff
+ w' D- S0 }9 U inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic enp0s3& R7 P: {! Z5 n6 m: z6 H5 `
valid_lft 82058sec preferred_lft 82058sec
) V3 i; M+ G+ V4 }. H inet6 fe80::b171:84d0:5173:de63/64 scope link, ?9 o6 O; b) u7 y7 ~
valid_lft forever preferred_lft forever
. B; r5 k# P" ]2 z e/ M3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
' r a) _% ]! C8 f: j+ }# d& I link/ether 08:00:27:7a:24:14 brd ff:ff:ff:ff:ff:ff" Y! \7 L8 D! S" T$ }
inet 192.168.56.252/24 brd 192.168.56.255 scope global enp0s8
9 f4 @/ \' p0 g% C) X$ ~ valid_lft forever preferred_lft forever' `& k1 E; c8 m K
inet6 fe80::a00:27ff:fe7a:2414/64 scope link4 G1 \- _0 l6 t9 t6 E. m/ _
valid_lft forever preferred_lft forever& e; E1 X4 ]7 r- Z
4: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN
; n" m8 h4 ]0 E2 B link/ether fa:5a:c7:c5:aa:e5 brd ff:ff:ff:ff:ff:ff8 e r3 n5 m2 Z+ X( C! e
inet 172.16.80.0/16 scope global flannel.1# s, _4 X. S3 r9 o
valid_lft forever preferred_lft forever4 t/ C) n( g- a7 c( C
5: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP
$ L* T H& F2 f3 L link/ether 02:42:72:2f:1f:ae brd ff:ff:ff:ff:ff:ff+ P$ ^: Y" l v$ S# s
inet 172.16.80.1/24 scope global docker06 z7 Y" I: U0 c
valid_lft forever preferred_lft forever; C0 S2 h$ x! ?4 ^; d0 L$ A, S# }
inet6 fe80::42:72ff:fe2f:1fae/64 scope link
6 ]* j" c, U" s valid_lft forever preferred_lft forever5 Z J0 t5 ^# B0 n+ U. i" ^
7: vethc56c1d4@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master docker0 state UP9 `8 e7 r; D+ Q1 t# _
link/ether 92:c8:3d:3f:b9:49 brd ff:ff:ff:ff:ff:ff link-netnsid 0
2 v! `8 R5 u& }* r2 n. C) B inet6 fe80::90c8:3dff:fe3f:b949/64 scope link3 F8 b, A% v- i
valid_lft forever preferred_lft forever
& z9 f+ v$ a! A" A0 I" H4 r% `9: vethf961994@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master docker0 state UP
x* Y$ j& ^' P+ G5 D4 e0 U link/ether d6:be:4b:6e:26:81 brd ff:ff:ff:ff:ff:ff link-netnsid 10 G. U% N1 v+ ]+ q, e2 F
inet6 fe80::d4be:4bff:fe6e:2681/64 scope link
( d- x& @5 Y" l% Q& ?5 p9 g valid_lft forever preferred_lft forever# z3 n) K% W) I* e8 P* i& G3 a! |5 @
11: vethe4cd28e@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master docker0 state UP
, N2 e0 p# m4 A( {4 w link/ether ee:55:55:df:4e:50 brd ff:ff:ff:ff:ff:ff link-netnsid 2* j) w U8 R( G/ n* X) [
inet6 fe80::ec55:55ff:fedf:4e50/64 scope link Q& }! h0 h' J" h+ F
valid_lft forever preferred_lft forever' C1 x! c7 \( g! {6 _& v
/ ?& q' }) y9 z3 R7 t
[root@cents2 ~]# iptables-save7 Q$ R! i" b! u* I* [) s% O
# Generated by iptables-save v1.4.21 on Sun Jan 22 00:41:01 2017. H& D+ M" x' H8 M6 u) t4 m; P
*filter$ g. q9 K" D' ]4 J7 n6 `, ?
:INPUT ACCEPT [27:4324]
& L j* Q" w. a+ h3 [' A:FORWARD ACCEPT [0:0]
! @4 V2 a6 H+ f:OUTPUT ACCEPT [25:2821]
- |7 }) b7 F' Z: T1 N:DOCKER - [0:0]
# D6 K x! ~) g' K:DOCKER-ISOLATION - [0:0]
/ h$ H+ P3 U/ p: V:KUBE-SERVICES - [0:0]
1 y( y8 H U H0 a0 u" h& a3 H* g-A FORWARD -j DOCKER-ISOLATION
7 t) B5 W7 O: R6 f/ m4 F( \& _-A FORWARD -o docker0 -j DOCKER
, l5 Y7 R1 e) i+ A6 e" g/ `+ m- I, S-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
! p* ]0 S6 `; F6 a! U7 ^-A FORWARD -i docker0 ! -o docker0 -j ACCEPT4 x2 X; u& z9 L8 u4 S$ f X! R
-A FORWARD -i docker0 -o docker0 -j ACCEPT
6 T2 p! ~1 p! ?# t6 a4 n0 c( H" q2 d7 J-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
( Q% g! k* V5 u2 ^1 y) j( Q) v' I-A DOCKER-ISOLATION -j RETURN
8 |; H0 C: l0 d* Z% x+ TCOMMIT" F3 D/ w& U( v* i$ c
# Completed on Sun Jan 22 00:41:01 2017- f+ K- f, {2 z s* f
# Generated by iptables-save v1.4.21 on Sun Jan 22 00:41:01 2017
: ]- q: E& R" m- P*nat6 u: K1 K) \& s3 s- T, s% P, ]
:PREROUTING ACCEPT [0:0]
7 \; s$ v* @8 K( `3 L* r5 y:INPUT ACCEPT [0:0]
3 Y8 n8 m5 T0 K2 {9 L:OUTPUT ACCEPT [2:119]
4 C3 U0 O" C# }7 t:POSTROUTING ACCEPT [2:119]
6 R0 `# Z" g( _; M; m:DOCKER - [0:0]2 ?1 I4 I# o) D" ~- I1 {" W
:KUBE-MARK-MASQ - [0:0] d7 F6 f9 l. A. R" I- P
:KUBE-NODEPORTS - [0:0]* T2 R& Z; \0 V8 d* p) ?
:KUBE-POSTROUTING - [0:0]
5 Y3 n9 D/ [, S3 t6 C:KUBE-SEP-63GTHXGNEQIFF6GY - [0:0]
4 X; ] `. S, e7 B- ]:KUBE-SEP-77PLGVXVTAKNHL2K - [0:0]
# n* p m5 U3 t5 y. A/ I:KUBE-SEP-7R2ESD4YYXMXFEFZ - [0:0] L3 ~. J& F, f7 r% c
:KUBE-SEP-GIMIRAR4ZAKGMA2Q - [0:0]
1 @8 `/ n* O/ o h2 v. K:KUBE-SEP-LYGBYJFMWSAWPLXU - [0:0]
: X. C8 |" P3 E; I' L* h; G:KUBE-SEP-Y7WMR7EBCL4N3QJX - [0:0]
+ V- o! h+ A T g8 ~! w7 n:KUBE-SEP-ZDWRYP3AMCRYOGNR - [0:0]
% r% Y; a% L1 | P; A4 I:KUBE-SERVICES - [0:0]) v/ V1 t2 j& q5 T0 N* y
:KUBE-SVC-7GF4BJM3Z6CMNVML - [0:0]
1 ?2 L0 N9 O- l' K- c. V2 S0 R:KUBE-SVC-AGR3D4D4FQNH4O33 - [0:0]! E# l3 K5 e/ E
:KUBE-SVC-GYQQTB6TY565JPRW - [0:0]' ~9 H% d/ O" d4 g# S q
:KUBE-SVC-NPX46M4PTMTKRN6Y - [0:0]. M/ a0 ?7 D1 i
-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
) y. ?' U" L7 P+ [9 A-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER# x. C1 k# i$ Y7 F" a. J8 u( ]4 w
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES# g+ Y3 R( k- J9 D
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
# T6 O- C: G* q2 j-A POSTROUTING -s 172.16.80.0/24 ! -o docker0 -j MASQUERADE1 k* C# C6 _4 r( h8 m
-A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING, n! L" \, b: f0 t2 y
-A DOCKER -i docker0 -j RETURN
; L( I: W( U2 z' e2 K# {-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
: Y3 R, b. O5 v. }9 o-A KUBE-NODEPORTS -p tcp -m comment --comment "default/frontend:" -m tcp --dport 30001 -j KUBE-MARK-MASQ$ ~: J' s. s: X7 R
-A KUBE-NODEPORTS -p tcp -m comment --comment "default/frontend:" -m tcp --dport 30001 -j KUBE-SVC-GYQQTB6TY565JPRW9 Y+ ?3 G9 s A' W2 O
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -m mark --mark 0x4000/0x4000 -j MASQUERADE6 }2 ?( F# p) D/ ~% i. m
-A KUBE-SEP-63GTHXGNEQIFF6GY -s 172.16.62.4/32 -m comment --comment "default/frontend:" -j KUBE-MARK-MASQ8 ]: ]# i$ `2 d) o }
-A KUBE-SEP-63GTHXGNEQIFF6GY -p tcp -m comment --comment "default/frontend:" -m tcp -j DNAT --to-destination 172.16.62.4:800 D; J6 H! d: j# w" i% c# d
-A KUBE-SEP-77PLGVXVTAKNHL2K -s 172.16.80.3/32 -m comment --comment "default/frontend:" -j KUBE-MARK-MASQ
2 {: n' d8 J+ S; f6 u-A KUBE-SEP-77PLGVXVTAKNHL2K -p tcp -m comment --comment "default/frontend:" -m tcp -j DNAT --to-destination 172.16.80.3:80; b# r$ l7 N( ~4 _
-A KUBE-SEP-7R2ESD4YYXMXFEFZ -s 172.16.80.2/32 -m comment --comment "default/redis-slave:" -j KUBE-MARK-MASQ
" Q: p7 o! z, C7 x ~9 W5 I0 r, p-A KUBE-SEP-7R2ESD4YYXMXFEFZ -p tcp -m comment --comment "default/redis-slave:" -m tcp -j DNAT --to-destination 172.16.80.2:6379
1 Y% L2 F: \5 W d% N; Z- s-A KUBE-SEP-GIMIRAR4ZAKGMA2Q -s 192.168.56.250/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ
9 o+ b. s6 ?+ r+ F0 }! E* [" _-A KUBE-SEP-GIMIRAR4ZAKGMA2Q -p tcp -m comment --comment "default/kubernetes:https" -m recent --set --name KUBE-SEP-GIMIRAR4ZAKGMA2Q --mask 255.255.255.255 --rsource -m tcp -j DNAT --to-destination 192.168.56.250:6443
$ y" w0 S/ @' n& c! b% ~-A KUBE-SEP-LYGBYJFMWSAWPLXU -s 172.16.62.3/32 -m comment --comment "default/redis-slave:" -j KUBE-MARK-MASQ
# y6 V( T; a! z& z+ E' @-A KUBE-SEP-LYGBYJFMWSAWPLXU -p tcp -m comment --comment "default/redis-slave:" -m tcp -j DNAT --to-destination 172.16.62.3:6379
) `7 E' e6 O1 M, D$ ^: H7 ?-A KUBE-SEP-Y7WMR7EBCL4N3QJX -s 172.16.62.2/32 -m comment --comment "default/redis-master:" -j KUBE-MARK-MASQ
4 s9 }7 E& ~6 X" E7 U-A KUBE-SEP-Y7WMR7EBCL4N3QJX -p tcp -m comment --comment "default/redis-master:" -m tcp -j DNAT --to-destination 172.16.62.2:63799 Y: b5 E; h! u1 P; j% c8 y
-A KUBE-SEP-ZDWRYP3AMCRYOGNR -s 172.16.80.4/32 -m comment --comment "default/frontend:" -j KUBE-MARK-MASQ1 t# J0 D( w; j2 z+ }2 ?
-A KUBE-SEP-ZDWRYP3AMCRYOGNR -p tcp -m comment --comment "default/frontend:" -m tcp -j DNAT --to-destination 172.16.80.4:80
$ E8 x _7 x1 N/ f& B. p- L-A KUBE-SERVICES -d 10.254.218.57/32 -p tcp -m comment --comment "default/frontend: cluster IP" -m tcp --dport 80 -j KUBE-SVC-GYQQTB6TY565JPRW
+ a, F- \5 M( {+ y+ f2 I2 Q-A KUBE-SERVICES -d 10.254.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
' |# D/ q9 _4 k-A KUBE-SERVICES -d 10.254.142.174/32 -p tcp -m comment --comment "default/redis-master: cluster IP" -m tcp --dport 6379 -j KUBE-SVC-7GF4BJM3Z6CMNVML
. p( |9 U. P& J. e& @-A KUBE-SERVICES -d 10.254.201.123/32 -p tcp -m comment --comment "default/redis-slave: cluster IP" -m tcp --dport 6379 -j KUBE-SVC-AGR3D4D4FQNH4O33* x# M/ q' V3 S" u6 Z! U
-A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS
% d# v0 |& l) y( s-A KUBE-SVC-7GF4BJM3Z6CMNVML -m comment --comment "default/redis-master:" -j KUBE-SEP-Y7WMR7EBCL4N3QJX' M* C% U3 `5 X/ P+ t3 n1 a* O
-A KUBE-SVC-AGR3D4D4FQNH4O33 -m comment --comment "default/redis-slave:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-LYGBYJFMWSAWPLXU0 Z+ d% f; w9 Q
-A KUBE-SVC-AGR3D4D4FQNH4O33 -m comment --comment "default/redis-slave:" -j KUBE-SEP-7R2ESD4YYXMXFEFZ
" [. D0 |4 G6 `-A KUBE-SVC-GYQQTB6TY565JPRW -m comment --comment "default/frontend:" -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-63GTHXGNEQIFF6GY
- B3 H d9 {" S" c2 j. a6 p$ e-A KUBE-SVC-GYQQTB6TY565JPRW -m comment --comment "default/frontend:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-77PLGVXVTAKNHL2K
9 q* P9 z6 Y) Y1 ~4 C-A KUBE-SVC-GYQQTB6TY565JPRW -m comment --comment "default/frontend:" -j KUBE-SEP-ZDWRYP3AMCRYOGNR
" C. s5 W; X" n* Z-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -m recent --rcheck --seconds 180 --reap --name KUBE-SEP-GIMIRAR4ZAKGMA2Q --mask 255.255.255.255 --rsource -j KUBE-SEP-GIMIRAR4ZAKGMA2Q' d$ `6 |; m t: d
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -j KUBE-SEP-GIMIRAR4ZAKGMA2Q
3 b3 C' K( A# [2 a( N# JCOMMIT
/ g2 y+ ?1 Z8 w2 V# Completed on Sun Jan 22 00:41:01 20175 ?: b3 `0 C# s: ^0 s& ^ \
- N4 E2 _8 ^7 W! }) G
---------------------------------------------------- 尝试了本地卷 [root@centm ~]# cat redis-master-controller_with_volume.yaml' [3 K, \6 c0 v* Q6 `4 O
apiVersion: v1
9 b7 |* ~# m8 f7 G% e4 e' @kind: ReplicationController
- n; ?6 a4 D2 f0 ^# vmetadata:
5 p. [* t$ Z: m1 T" k3 u/ U labels:edis-master
' K2 J: {- y0 N% Y1 L" b" o, @ name: redis-master ' k% }$ r+ ^6 G; }. w0 A
spec:
5 T; u7 O3 \% k. N replicas: 1
5 X0 } }( v+ r selector:0 U5 d) l- t6 p$ k1 E* _1 D
name: redis-master5 X: K9 M b! i# _. {9 @$ f& s0 g
template: $ z! N. D$ n/ b7 z7 c
metadata:$ l1 a# X/ D: T1 g) D
labels:
) ?) r# \* `1 T: G name: redis-master+ }$ H4 |/ y4 {5 B
spec: n8 y5 r# u, L2 H7 ?
volumes:
. m% ~' @) {' j7 T- N$ ~ - name: "gf-dir1"
( |# I9 @8 X0 d$ c, a5 ?2 H, Z hostPath:
" u7 X5 |7 C: l" { M4 B0 _0 e path: "/tmp"
& T$ z7 U! Z k# T" h
9 w1 s* P' g1 I1 ]' c containers:
& J& c1 f- A0 H J1 M9 d7 P6 n7 ^ - name: master 7 _8 a3 Q5 J+ D. B3 c8 S1 e5 G
ports:: docker.io/kubeguide/redis-master:latest + B! f+ T" U0 X. l6 G: V& {
- containerPort : 6379
/ \$ n+ ]& F. X volumeMounts:
5 i7 C* u" z5 O/ C. R" q - name: "gf-dir1"/ T' P; M/ C0 h& D( _4 E, G* P2 u
mountPath: "/gf1" |
发表于 2018-9-19 17:08:18