马上注册,结交更多好友,享用更多功能,让你轻松玩转社区。
您需要 登录 才可以下载或查看,没有账号?开始注册
x
edis和docker这两个词语会自动被修改为首字母大写并链接到知识库,所以在这里先写一遍,后面就不会被改写了。 0、具体操作见 (vmware中搭建k8s),virtulbox也是相同的流程。
- k5 ^! @3 f$ w7 T8 }& H% F 1、学习k8s,读的是这本书《KUBERNETES权威指南 从DOCKET到KURBERNETES实践全接触.pdf》 2、这边书刚开始讲的是在单机上搭建一个k8s的hello world,用的是centos7. 于是我也在virtulbox中安装了centos7,并成功运行hello world。 3、然后,开始尝试集群了。在网上找了个教程,比较简单,很快就成功了,kubectl get nodes能看到各node了。 但是,应用跑的不正常,有的节点可以访问,有的节点不可以。而且从不同节点访问,查到的数据不相同,似乎是多个独立的系统。怀疑是iptables中cluster ip的规则有问题。
& y7 i8 |2 ` }( h( X$ K' @ 接着,集群坏了,k8s的基础服务都启动不了。怀疑是不是因为我创建这些虚机时,用的是链接式拷贝,是不是原始的虚机安装了其它软件,导致k8s集群启动不了。 4、删掉重来,用完全拷贝的方式建立虚机。 一切正常,但是在启动redis-master-controller.yaml时,docker中无法建立容器, 用kubectl describe pod redis-master命令排查,发现下拉不了镜像。其实这些镜像已经存在于docker中了(搭建单机k8s时,自动pull的)。只是镜像名字前面多加了docker.io/ 我修改了redis-master-controller.yaml中镜像的名字,仍然pull失败。奇怪。 5、今天定位了网络问题(见virtualbox虚拟机无法上网),网络搞通后,hello world终于正常运行了。 之前的一些疑问,有答案了 1、随便访问哪个node的ip(比如http://192.168.56.251:30001/,http://192.168.56.252:30001/)(http://192.168.56.250:30001/是不行的),都可以访问到服务的(proxy自动转的) 2、在内部时,访问真实端口也是可以的。 [root@centm ~]# kubectl get svc) j+ ^4 Y2 _8 Z4 l3 d% M
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
$ W0 F' e6 {6 M \( {9 C4 }, mfrontend 10.254.218.57 <nodes> 80/TCP 5m+ B; p9 g2 [* [! d
kubernetes 10.254.0.1 <none> 443/TCP 15d
; }; }. Y) g- c# `3 O9 predis-master 10.254.142.174 <none> 6379/TCP 8m
* f U2 e* A" ?redis-slave 10.254.201.123 <none> 6379/TCP 6m {2 K2 d3 }, d0 @3 q" Y
curl 10.254.218.57:80 可以通。 ping 10.254.218.57 不通 3、node中多了一个127.0.0.1 ,不知道为什么
$ @2 @$ x' ]6 `9 z! V4 v [root@centm ~]# kubectl get nodes. o0 v7 ?) x, ^6 d a. T- B! q6 A) i4 f
NAME STATUS AGE, z* V: f4 a# V/ @0 U* y$ [
127.0.0.1 NotReady 15d* b7 g; k' b% ^+ q( ], c( ^+ F1 Y
cents1 Ready 1d
8 P: F, \& r0 G; X1 z5 Ocents2 Ready 1d
, [( v! ?$ h# q; R0 c9 v- L[root@centm ~]# ps -ef|grep kube
9 k d7 a: {" @1 b+ Ukube 578 1 0 Jan20 ? 00:15:55 /usr/bin/kube-controller-manager --logtostderr=true --v=0 --master=http://127.0.0.1:8080* N, x4 I$ q: A% b R
kube 588 1 0 Jan20 ? 00:01:09 /usr/bin/kube-scheduler --logtostderr=true --v=0 --master=http://127.0.0.1:8080
& Y5 \: \) F z! B- C; Wkube 2079 1 0 Jan20 ? 00:08:11 /usr/bin/kube-apiserver --logtostderr=true --v=0 --etcd-servers=http://127.0.0.1:2379 --insecure-bind-address=0.0.0.0 --allow-privileged=false --service-cluster-ip-range=10.254.0.0/16 --admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota0 V. R4 l( M0 Y; s) h m& o
4、clusterip是一个虚ip,实际是iptables中的几个转发规则。 [root@cents2 ~]# ip a. u( I, R7 E2 ]4 _
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1; H! v: N% y; j n1 O _7 D
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:002 B1 J$ m: }- E2 |
inet 127.0.0.1/8 scope host lo
( J m# q* {$ [3 a& O! ] valid_lft forever preferred_lft forever. M8 h; h1 | z5 u
inet6 ::1/128 scope host" J( |7 G- h" g- i+ l/ s+ f
valid_lft forever preferred_lft forever
4 N; n" B$ `8 I I1 x- `: A2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
9 ~" {( N2 m, U( B5 j) q link/ether 08:00:27:58:5d:6e brd ff:ff:ff:ff:ff:ff
N. |/ m3 x1 S, g6 l$ ~0 X: e inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic enp0s3
4 ]+ m* c O; @ valid_lft 82058sec preferred_lft 82058sec- i$ g1 ?9 ^' A- M/ A6 x
inet6 fe80::b171:84d0:5173:de63/64 scope link3 c: V1 E5 q) J- u: j3 ?" [$ [
valid_lft forever preferred_lft forever
2 ~7 E7 _$ S8 Q0 W( j% f3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000- N' H; A6 l5 @
link/ether 08:00:27:7a:24:14 brd ff:ff:ff:ff:ff:ff
2 `0 g0 \) P" D9 O, J; g p inet 192.168.56.252/24 brd 192.168.56.255 scope global enp0s8
3 q1 c1 m; x' s# K& z# K6 H) U; G valid_lft forever preferred_lft forever
7 z* x0 K+ U, { N inet6 fe80::a00:27ff:fe7a:2414/64 scope link6 p4 p& k$ z9 B, I8 ^/ u4 V# h
valid_lft forever preferred_lft forever3 s( y3 z0 \0 r. S% N& ]
4: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN
; N/ W8 g$ u1 G$ G% R link/ether fa:5a:c7:c5:aa:e5 brd ff:ff:ff:ff:ff:ff
9 w0 ~( g3 e- L# D7 _ A inet 172.16.80.0/16 scope global flannel.1% U8 g& h C2 j- ]
valid_lft forever preferred_lft forever3 M* }+ E' e# h' c: x# m
5: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP
4 ^7 N Q9 N. z link/ether 02:42:72:2f:1f:ae brd ff:ff:ff:ff:ff:ff/ K* W% q% m" C% O7 W! ~
inet 172.16.80.1/24 scope global docker0
1 Z! f+ z. K* ` valid_lft forever preferred_lft forever
! \$ W0 Y5 G& L4 ~ inet6 fe80::42:72ff:fe2f:1fae/64 scope link
0 I: m! R0 q/ z$ ~0 r* a' x valid_lft forever preferred_lft forever( Y2 m- k+ p W7 h# h; ^4 g
7: vethc56c1d4@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master docker0 state UP* X3 p0 J3 X) M, ?4 n m. ]: t
link/ether 92:c8:3d:3f:b9:49 brd ff:ff:ff:ff:ff:ff link-netnsid 04 F8 ?5 X- V2 Q; Y' X G( G
inet6 fe80::90c8:3dff:fe3f:b949/64 scope link# _2 v0 C5 u3 a. i$ r
valid_lft forever preferred_lft forever
8 B9 i1 q$ H* P0 U% R5 J0 Z9: vethf961994@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master docker0 state UP
! Y( Z8 U; m$ i1 C" B link/ether d6:be:4b:6e:26:81 brd ff:ff:ff:ff:ff:ff link-netnsid 1
% a" b2 w, j7 W; K% | inet6 fe80::d4be:4bff:fe6e:2681/64 scope link
$ q. K- Q- `! S9 m valid_lft forever preferred_lft forever' v$ h* q) `; }5 _( | I' W f, B
11: vethe4cd28e@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master docker0 state UP4 W! q- {; C3 K) \& \+ r
link/ether ee:55:55:df:4e:50 brd ff:ff:ff:ff:ff:ff link-netnsid 2
9 l5 ?/ _/ }+ W, \2 e inet6 fe80::ec55:55ff:fedf:4e50/64 scope link
4 R$ [6 M; n. Q. G! S& o2 ^3 r valid_lft forever preferred_lft forever
4 V+ o4 W4 R8 m
0 G' W, R7 K- h; Y[root@cents2 ~]# iptables-save
3 \: I- `+ k; m- K5 f, _2 D& b8 B# Generated by iptables-save v1.4.21 on Sun Jan 22 00:41:01 2017
% h; ^+ `- e9 R) |1 L*filter: |2 o1 L n- o
:INPUT ACCEPT [27:4324]
* F, H" h: Z+ G% U:FORWARD ACCEPT [0:0]3 l% }9 y. ~2 D" V/ _, x
:OUTPUT ACCEPT [25:2821]
7 G, x5 j3 w. w0 b$ C$ T6 x! u& L:DOCKER - [0:0]7 f# F- X& t1 {1 p, x2 q
:DOCKER-ISOLATION - [0:0]
$ y- V, p: ~- ~! q5 Y' G( B; [:KUBE-SERVICES - [0:0]
$ o5 z) \$ O& P-A FORWARD -j DOCKER-ISOLATION* z6 J$ Z: h2 l. W7 V
-A FORWARD -o docker0 -j DOCKER
# s/ `/ K/ V! P/ D-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT" E* h0 v. C- H, T% ~
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT' W% B' v( N9 G
-A FORWARD -i docker0 -o docker0 -j ACCEPT9 m6 S6 }9 [: z
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES, V+ {. f+ }& t' S& ~, F& x) d
-A DOCKER-ISOLATION -j RETURN
' W& X' D O9 T$ ECOMMIT
# o0 C& {3 f2 z: I6 H' s4 f# Completed on Sun Jan 22 00:41:01 2017( n! D4 _- Q! I5 a' W
# Generated by iptables-save v1.4.21 on Sun Jan 22 00:41:01 2017 }3 K1 Q# U/ b7 H! Q
*nat
" Y* `2 t% e% z8 m; a3 [' n. q:PREROUTING ACCEPT [0:0]
" l' n W1 ]3 h/ ^:INPUT ACCEPT [0:0]
' D4 c. z( n4 E& G:OUTPUT ACCEPT [2:119]
4 J2 p/ @- c! F3 b8 i1 D:POSTROUTING ACCEPT [2:119]
6 o% K1 Y5 t3 Y, p% R8 ~:DOCKER - [0:0]/ t( x& W2 N- n E( F5 S
:KUBE-MARK-MASQ - [0:0]
! \& v) k' {! [0 J9 I R:KUBE-NODEPORTS - [0:0]
- J% ]+ \7 X9 n9 O* s:KUBE-POSTROUTING - [0:0]
3 [1 U, v7 k8 y6 d9 H2 D, ]:KUBE-SEP-63GTHXGNEQIFF6GY - [0:0]' @3 I* J* W2 w
:KUBE-SEP-77PLGVXVTAKNHL2K - [0:0]
$ q) V0 k% |7 e0 p' I:KUBE-SEP-7R2ESD4YYXMXFEFZ - [0:0]
8 Z* {3 P+ e9 v:KUBE-SEP-GIMIRAR4ZAKGMA2Q - [0:0]# Z' _; O7 B3 X& l4 L
:KUBE-SEP-LYGBYJFMWSAWPLXU - [0:0]2 K, }! I2 x' w) y, f$ Y
:KUBE-SEP-Y7WMR7EBCL4N3QJX - [0:0]7 q u+ Y$ W: S* l1 G7 d1 Q
:KUBE-SEP-ZDWRYP3AMCRYOGNR - [0:0]
/ A# k% Q3 E5 A1 S:KUBE-SERVICES - [0:0]
; u( S h: U" d$ X {:KUBE-SVC-7GF4BJM3Z6CMNVML - [0:0]
" ~+ X! q3 k/ ~5 K2 r4 k:KUBE-SVC-AGR3D4D4FQNH4O33 - [0:0]4 q( A! p G. ^" d
:KUBE-SVC-GYQQTB6TY565JPRW - [0:0]6 r5 F* ~1 S" q( ^6 @6 R# ]( b1 H
:KUBE-SVC-NPX46M4PTMTKRN6Y - [0:0]2 O0 O# Z! j% K) W, q
-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES0 P+ T! z- I# P# m5 n5 P$ }
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER2 @4 O0 Z- v- z- E/ _- e
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
( h8 J6 [4 d s, j4 f2 y4 s- L-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER$ A% s2 p6 }( C1 ]. O
-A POSTROUTING -s 172.16.80.0/24 ! -o docker0 -j MASQUERADE
# g" g& k0 @: z( \-A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING
9 r( b, W2 K u. f; v-A DOCKER -i docker0 -j RETURN! z, T" [+ M2 G" k1 c
-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
0 F" p2 A0 H* R% h) [) g-A KUBE-NODEPORTS -p tcp -m comment --comment "default/frontend:" -m tcp --dport 30001 -j KUBE-MARK-MASQ& e% Q/ R W9 T4 K1 b0 N
-A KUBE-NODEPORTS -p tcp -m comment --comment "default/frontend:" -m tcp --dport 30001 -j KUBE-SVC-GYQQTB6TY565JPRW* {6 ~# o+ q- I% ^; `& ?
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -m mark --mark 0x4000/0x4000 -j MASQUERADE8 H% o9 P* V& ~% G. D# }
-A KUBE-SEP-63GTHXGNEQIFF6GY -s 172.16.62.4/32 -m comment --comment "default/frontend:" -j KUBE-MARK-MASQ
J; }7 [1 U; {5 E4 V r" B, x-A KUBE-SEP-63GTHXGNEQIFF6GY -p tcp -m comment --comment "default/frontend:" -m tcp -j DNAT --to-destination 172.16.62.4:80' {' \3 `" l; A1 `$ i2 K# Y% [
-A KUBE-SEP-77PLGVXVTAKNHL2K -s 172.16.80.3/32 -m comment --comment "default/frontend:" -j KUBE-MARK-MASQ5 \: Q+ Z. N; g' G. z) {
-A KUBE-SEP-77PLGVXVTAKNHL2K -p tcp -m comment --comment "default/frontend:" -m tcp -j DNAT --to-destination 172.16.80.3:80+ u5 j7 q. G/ k: u
-A KUBE-SEP-7R2ESD4YYXMXFEFZ -s 172.16.80.2/32 -m comment --comment "default/redis-slave:" -j KUBE-MARK-MASQ
; d0 v6 H. m& L$ r-A KUBE-SEP-7R2ESD4YYXMXFEFZ -p tcp -m comment --comment "default/redis-slave:" -m tcp -j DNAT --to-destination 172.16.80.2:6379( a# A, J8 B) h. Z3 ~' Y
-A KUBE-SEP-GIMIRAR4ZAKGMA2Q -s 192.168.56.250/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ* p3 Z1 [/ r" P$ i
-A KUBE-SEP-GIMIRAR4ZAKGMA2Q -p tcp -m comment --comment "default/kubernetes:https" -m recent --set --name KUBE-SEP-GIMIRAR4ZAKGMA2Q --mask 255.255.255.255 --rsource -m tcp -j DNAT --to-destination 192.168.56.250:6443
. w9 @5 P2 J0 R3 Y7 j-A KUBE-SEP-LYGBYJFMWSAWPLXU -s 172.16.62.3/32 -m comment --comment "default/redis-slave:" -j KUBE-MARK-MASQ
) H; U) m/ P: w-A KUBE-SEP-LYGBYJFMWSAWPLXU -p tcp -m comment --comment "default/redis-slave:" -m tcp -j DNAT --to-destination 172.16.62.3:6379
* ?* `$ X' H1 z; U( ]4 K-A KUBE-SEP-Y7WMR7EBCL4N3QJX -s 172.16.62.2/32 -m comment --comment "default/redis-master:" -j KUBE-MARK-MASQ
( z: ?* m1 k3 Q- ^8 e2 a/ \-A KUBE-SEP-Y7WMR7EBCL4N3QJX -p tcp -m comment --comment "default/redis-master:" -m tcp -j DNAT --to-destination 172.16.62.2:6379
9 V! W; `3 N# I" M) J, h-A KUBE-SEP-ZDWRYP3AMCRYOGNR -s 172.16.80.4/32 -m comment --comment "default/frontend:" -j KUBE-MARK-MASQ
) k& u; j" `1 u! O# O5 H" E-A KUBE-SEP-ZDWRYP3AMCRYOGNR -p tcp -m comment --comment "default/frontend:" -m tcp -j DNAT --to-destination 172.16.80.4:80
* A& T' C$ C1 N* }7 _& P-A KUBE-SERVICES -d 10.254.218.57/32 -p tcp -m comment --comment "default/frontend: cluster IP" -m tcp --dport 80 -j KUBE-SVC-GYQQTB6TY565JPRW
4 O. m4 R7 X+ [$ ]9 Z5 O- g-A KUBE-SERVICES -d 10.254.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y1 d. h5 u( N$ _" |* p1 i* x
-A KUBE-SERVICES -d 10.254.142.174/32 -p tcp -m comment --comment "default/redis-master: cluster IP" -m tcp --dport 6379 -j KUBE-SVC-7GF4BJM3Z6CMNVML& s' d, [3 y) ~; a0 k# s# F
-A KUBE-SERVICES -d 10.254.201.123/32 -p tcp -m comment --comment "default/redis-slave: cluster IP" -m tcp --dport 6379 -j KUBE-SVC-AGR3D4D4FQNH4O33
) ^$ K) w$ H8 p% |/ T! v4 K-A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS5 u1 Q# U) P7 ~8 O$ @: x
-A KUBE-SVC-7GF4BJM3Z6CMNVML -m comment --comment "default/redis-master:" -j KUBE-SEP-Y7WMR7EBCL4N3QJX
$ S4 P L% j, g; o9 l; K, W-A KUBE-SVC-AGR3D4D4FQNH4O33 -m comment --comment "default/redis-slave:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-LYGBYJFMWSAWPLXU5 r" f3 _0 C$ w" i- x2 u
-A KUBE-SVC-AGR3D4D4FQNH4O33 -m comment --comment "default/redis-slave:" -j KUBE-SEP-7R2ESD4YYXMXFEFZ
' ]0 @" _0 z- c* {7 y-A KUBE-SVC-GYQQTB6TY565JPRW -m comment --comment "default/frontend:" -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-63GTHXGNEQIFF6GY/ e; k8 Y Q" X8 l6 k
-A KUBE-SVC-GYQQTB6TY565JPRW -m comment --comment "default/frontend:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-77PLGVXVTAKNHL2K
* h2 C$ ]: k" ~) X! R2 e-A KUBE-SVC-GYQQTB6TY565JPRW -m comment --comment "default/frontend:" -j KUBE-SEP-ZDWRYP3AMCRYOGNR0 J: c0 g8 z' a/ w0 b
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -m recent --rcheck --seconds 180 --reap --name KUBE-SEP-GIMIRAR4ZAKGMA2Q --mask 255.255.255.255 --rsource -j KUBE-SEP-GIMIRAR4ZAKGMA2Q
( {! Z! A& a2 Z: _' k-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -j KUBE-SEP-GIMIRAR4ZAKGMA2Q' h9 E7 o; p; p- t3 y
COMMIT
( D2 k$ O( I+ y+ A: m! b; \# Completed on Sun Jan 22 00:41:01 2017
4 W/ K7 X( {! c# ?& u& \- L1 P' E5 `$ S& U( ^" g: G
---------------------------------------------------- 尝试了本地卷 [root@centm ~]# cat redis-master-controller_with_volume.yaml9 A" x' W- t' R& Y
apiVersion: v1
~9 y8 m& D9 x2 zkind: ReplicationController 2 e9 x: @2 x* ^0 a
metadata: ! v/ d9 x6 r3 N' J- X$ U
labels:edis-master " \' @; f8 J2 K5 N- W
name: redis-master ) h/ [$ y$ J, D* X4 U: q$ t1 H
spec:
# K8 h2 Q, m! x3 _ replicas: 1/ N/ S' a/ c3 s& f; O0 f i+ K
selector:' l( w6 z7 }9 {* G9 Z6 Z3 X9 I
name: redis-master$ ^/ f& ?/ Z' c" Z, F
template:
' h2 ]. o$ v+ \) M7 H" ]+ d2 Y/ C metadata:& Y- L, v7 }4 L! G. j, S
labels:
; ^ {9 P1 h4 J! ~0 F. K# W name: redis-master0 T4 x. w3 t* Z9 q$ q* i! ^8 e! |2 k
spec:) U: I/ p1 Z8 Z4 p% B
volumes:
3 Y" A+ B' \; y* [5 f! _# [7 @ - name: "gf-dir1"
$ k4 [0 E+ L& T4 j$ g9 q4 k7 g, t hostPath:
/ \: R( D/ E$ E0 A- P$ q path: "/tmp"
7 [( V; C( `1 G$ L7 ?2 n# I+ n; Y7 o% s0 X; o
containers:
8 @( d0 _6 m. B8 j4 |: d - name: master
5 N7 F7 _* y- o, B: j4 I- v ports:: docker.io/kubeguide/redis-master:latest
5 z$ h4 \5 J& O( R( J6 a - containerPort : 6379
2 h! Q( P+ y y a8 G volumeMounts:
" x: e1 n6 f ^1 i+ Q% a# r6 e - name: "gf-dir1"% [5 j, E3 U+ F6 s- a
mountPath: "/gf1" |