k8s学习二：k8s编译安装集群搭建——单master多node简易部署

admin · 发表于 2018-9-20 11:08:15

服务器环境

centos7.5
mac装的pd虚拟机
作用 IP 部署服务配置
master 10.211.55.10 etcd、kube-apiserver、kube-controller-manager、kube-scheduler 2C、2G
node1 10.211.55.11 docker 、kubelet、kube-proxy 2C、2G
node2 10.211.55.12 docker 、kubelet、kube-proxy 2C、2G
- 计划采用二进制包进行部署：

所需二进制包下载地址：
1.https://dl.k8s.io/v1.10.4/kubernetes-server-linux-amd64.tar.gz
2.https://dl.k8s.io/v1.10.4/kubernetes-node-linux-amd64.tar.gz
3.https://github.com/coreos/etcd/r ... -linux-amd64.tar.gz
注意所有服务器都需要关闭防火墙
Master部署

二进制安装基本都是以下几个步骤：
1、复制对应的二进制文件到/usr/bin目录下
2、创建systemd service启动服务文件
3、创建service中对应的配置参数文件
4、将该应用加入到开机自启
5、启动服务并查看服务状态
etcd部署

下载二进制安装包并安装：
wget https://github.com/coreos/etcd/r ... -linux-amd64.tar.gz
cd etcd-v3.2.22-linux-amd64/
cp etcd /usr/bin/
cp etcdctl /usr/bin/
mkdir /var/lib/etcd
mkdir /etc/etcd

编辑systemd管理文件
vim /usr/lib/systemd/system/etcd.service

[Unit]
Description=Etcd Server
After=network.target

[Service]
Type=simple
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
ExecStart=/usr/bin/etcd

[Install]
WantedBy=multi-user.target

启动服务,并设置开机启动
systemctl daemon-reload
systemctl start etcd
systemctl enable etcd

查看服务状态的三种命令
systemctl status etcd.service

curl -L http://127.0.0.1:2379/version

etcdctl cluster-health

这个安装的还挺顺利，很快就ok了。继续。。。。
kube-apiserver

下载并安装
wget https://dl.k8s.io/v1.10.4/kubernetes-server-linux-amd64.tar.gz
tar -xzvf kubernetes-server-linux-amd64.tar.gz
cd kubernetes/server/bin
cp kube-apiserver /usr/bin/

# 一起拷贝吧，后面就直接配置了
cp kube-controller-manager /usr/bin/
cp kube-scheduler /usr/bin/

编辑systemd的启动文件
vim /usr/lib/systemd/system/kube-apiserver.service

[Unit]
Description=Kubernetes API Server
Documentation=https://kubernetes.io/docs/concepts/overview
After=network.target
After=etcd.service

[Service]
EnvironmentFile=/etc/kubernetes/apiserver
ExecStart=/usr/bin/kube-apiserver $KUBE_API_ARGS
Restart=on-failure
Type=notify
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

配置参数文件
mkdir /etc/kubernetes/
vim /etc/kubernetes/apiserver

KUBE_API_ARGS="--storage-backend=etcd3 \
            --etcd-servers=http://127.0.0.1:2379 \
            --bind-address=0.0.0.0 \
            --secure-port=6443  \
            --service-cluster-ip-range=192.168.2.0/16  \
            --service-node-port-range=1-65535 \
            --client-ca-file=/etc/kubernetes/ssl/ca.crt \
            --tls-private-key-file=/etc/kubernetes/ssl/server.key  \
            --tls-cert-file=/etc/kubernetes/ssl/server.crt  \
            --enable-admission-plugins=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,DefaultStorageClass,ResourceQuota \
            --logtostderr=false \
            --log-dir=/var/log/kubernetes \
            --v=2"

service-cluster-ip-range是servcies的虚拟IP的IP范围，这里可以自己定义，不能当前的宿主机网段重叠。
bind-addres 指定的apiserver监听地址，对应的监听端口是6443，使用的https的方式。(0.0.0.0 表示绑定所有地址)
client-ca-file 这是认证的相关文件，这预先定义，后面会创建证书文件，并放置到对应的路径。
创建日志目录和证书目录
mkdir -p /etc/kubernetes/ssl
mkdir -p /var/log/kubernete

kube-controller-manager

kube-controller-manager 依赖 kube-apiserver服务
编辑systemd启动文件
vim /usr/lib/systemd/system/kube-controller-manager.service

[Unit]
Description=Kubernetes Controller Manager
Documentation=https://kubernetes.io/docs/setup
After=kube-apiserver.service
Requires=kube-apiserver.service

[Service]
EnvironmentFile=/etc/kubernetes/controller-manager
ExecStart=/usr/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

配置启动参数
vim /etc/kubernetes/controller-manager

KUBE_CONTROLLER_MANAGER_ARGS="--master=https://10.211.55.10:6443 \
            --service-account-private-key-file=/etc/kubernetes/ssl/server.key  \
            --root-ca-file=/etc/kubernetes/ssl/ca.crt \
            --kubeconfig=/etc/kubernetes/kubeconfig \
            --logtostderr=false \
            --log-dir=/var/log/kubernetes \
            --v=2"

kube-scheduler

kube-scheduler也依赖kubu-apiserver
- 编辑systemd启动文件
vim /usr/lib/systemd/system/kube-scheduler.service

[Unit]
Description=Kubernetes Controller Manager
Documentation=https://kubernetes.io/docs/setup
After=kube-apiserver.service
Requires=kube-apiserver.service

[Service]
EnvironmentFile=/etc/kubernetes/scheduler
ExecStart=/usr/bin/kube-scheduler $KUBE_SCHEDULER_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
配置参数文件
vim /etc/kubernetes/scheduler

KUBE_SCHEDULER_ARGS="--master=https://10.211.55.10:6443 --kubeconfig=/etc/kubernetes/kubeconfig \
            --logtostderr=false \
            --log-dir=/var/log/kubernetes \
            --v=2"

创建CA证书

注意生成证书前先同步一下服务器时间：ntpdate s2m.time.edu.cn
创建kube-apiserver的CA证书和私钥文件
cd  /etc/kubernetes/ssl/
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -subj "/CN=10.211.55.10" -days 5000 -out ca.crt
openssl genrsa -out server.key 2048

创建master_ssl.cnf文件
vim master_ssl.cnf

[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster.local
DNS.5 = k8s_master
IP.1 = 192.168.2.1    # ClusterIP 地址
IP.2 = 10.211.55.10 # master IP地址

生成apiserver证书
openssl req -new -key server.key -subj "/CN=10.211.55.10" -config master_ssl.cnf -out server.csr

openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 5000 -extensions v3_req -extfile master_ssl.cnf -out server.crt

设置kube-controller-manager相关证书
openssl genrsa -out cs_client.key 2048
openssl req -new -key cs_client.key -subj "/CN=10.211.55.10" -out cs_client.csr
openssl x509 -req -in cs_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out cs_client.crt -days 5000

创建kubeconfig文件,kube-controller-manager和kube-scheduler公用的配置文件
vim /etc/kubernetes/kubeconfig

apiVersion: v1
kind: Config
users:
- name: controllermanager
  user:
client-certificate: /etc/kubernetes/ssl/cs_client.crt
client-key: /etc/kubernetes/ssl/cs_client.key
clusters:
- name: local
  cluster:
certificate-authority: /etc/kubernetes/ssl/ca.crt
contexts:
- context:
cluster: local
user: controllermanager
  name: my-context
current-context: my-context

启动服务

启动kube-apiserver
systemctl daemon-reload
systemctl enable kube-apiserver
systemctl start kube-apiserver

启动kube-controller-manager
systemctl enable kube-controller-manager
systemctl start kube-controller-manager

启动kube-scheduler
systemctl enable kube-scheduler
systemctl start kube-scheduler

Node

安装docker

使用aliyun的yum源
curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
curl -o /etc/yum.repos.d/docker-ce.repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum makecache

yum安装docker工具
yum install docker-ce
systemctl start docker
systemctl enable docker

docker -v

安装kubelet服务

安装包下载，整理
wget https://dl.k8s.io/v1.10.4/kubernetes-node-linux-amd64.tar.gz
tar -xzvf kubernetes-node-linux-amd64.tar.gz
cd kubernetes/node/bin
cp * /usr/bin

添加systemctl启动配置
vim /usr/lib/systemd/system/kubelet.service
mkdir -p /var/lib/kubelet
mkdir -p /etc/kubernetes/
mkdir -p /var/log/kubernetes

[Unit]
Description=Kubelet Service
After=docker.service
Requires=docker.service
[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet $KUBELET_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

kuberlet运行参数配置
安装kube-proxy服务

添加systemctl启动配置
vim /usr/lib/systemd/system/kube-proxy.service

[Unit]
Description=K8s kube-proxy Service
After=network.target
After=docker.service
After=network.target
After=network.service

[Service]
EnvironmentFile=/etc/kubernetes/kube-proxy
ExecStart=/usr/bin/kube-proxy $KUBE_PROXY_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

生成CA证书

将master节点上的kube-apiserver证书ca.crt和ca.key拷贝到Node上
使用ca.crt和ca.key生成node证书
openssl genrsa -out kubelet_client.key 2048
openssl req -new -key kubelet_client.key -subj "/CN=10.211.55.11" -out kubelet_client.csr
openssl x509 -req -in kubelet_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out kubelet_client.crt -days 5000

mkdir /etc/kubernetes/ssl
mv kubelet_client.* /etc/kubernetes/ssl/
mv ca.crt /etc/kubernetes/ssl/

配置kubeconfig
vim /etc/kubernetes/kubeconfig

apiVersion: v1
kind: Config
users:
- name: kubelet
  user:
   client-certificate: /etc/kubernetes/ssl/kubelet_client.crt
   client-key: /etc/kubernetes/ssl/kubelet_client.key
clusters:
- name: local
  cluster:
   certificate-authority: /etc/kubernetes/ssl/ca.crt
   server: https://10.211.55.10:6443
contexts:
- context:
   cluster: local
   user: kubelet
  name: my-context
current-context: my-context

kubelet启动参数配置
vim /etc/kubernetes/kubelet

KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --hostname-override=10.211.55.11 --logtostderr=false --log-dir=/var/log/kubernetes --v=2 --fail-swap-on=false"
这里要注意–fail-swap-on=false或者禁用swap，我这里选择配置–fail-swap-on=false
设置kube-proxy启动参数
vim /etc/kubernetes/kube-proxy

KUBE_PROXY_ARGS="--master=https://10.211.55.10:6443 --kubeconfig=/etc/kubernetes/kubeconfig --logtostderr=false --log-dir=/var/log/kubernetes --v=2"
启动服务

systemctl daemon-reload
systemctl start kubelet.service
systemctl status kubelet.service

systemctl start kube-proxy
systemctl status kube-proxy
node 2就按照上面的步骤进行安装即可

admin · 发表于 2018-9-20 11:11:21

搭建私有库

私有库用于系统内部存储成品镜像，能够快速进行下载及被k8s调度。

1.下载并启动私有库

[centos-master]:docker run --name registry -v /etc/localtime:/etc/localtime -v /opt/registry:/var/lib/registry -p 5000:5000 -itd docker.io/registry

#--name 表示启动的容器后名称，此处为registry
#-v 表示挂载路径  格式为宿主机路径:容器内路径
#-p 表示映射端口  格式为宿主机端口:容器内端口
#-itd docker的内部参数，此处声明后台运行容器并分配一个伪终端并绑定到容器的标准输入上，后跟镜像名称此处为docker.io/registry

2.创建一个secret服务，用于k8s调度私有库容器时的“令牌”。简单来说，secret服务就是一个存储密码的服务

[centos-master]:kubectl create secret docker-registry registrykey --docker-server=registry.evehicle.cn --docker-username=docker --docker-password=docker --docker-email=lienhua@zhongchuangsanyou.com

[centos-master]:kubectl get secret
NAME       TYPE                   DATA    AGE
registrykey kubernetes.io/dockercfg 1       6s

此时登录时会提示认证错误

[centos-master]:docker login -u docker -p docker -e lienhua@zhongchuangsanyou.com registry.evehicle.cn
Flag --email has been deprecated, will be removed in 1.13.
Error response from daemon: login attempt to https://registry.evehicle.cn/v2/ failed with status: 401 Unauthorized

这是因为Docker官方是推荐采用Secure Registry的工作模式的，即transport采用tls。这样我们就需要为Registry配置tls所需的key和crt文件了

3.配置nginx反向代理
[centos-master]: cat registry.evehicle.cn.conf

# For versions of nginx > 1.3.9 that include chunked transfer encoding support
# Replace with appropriate values where necessary

upstream docker-registry {
  server 192.168.121.9:5000;
  #server 10.44.170.95:5000;
}

# uncomment if you want a 301 redirect for users attempting to connect
# on port 80
# NOTE: docker client will still fail. This is just for convenience
# server {
# listen *:80;
# server_name my.docker.registry.com;
# return 301 https://$server_name$request_uri;
# }

server {
listen 443;
server_name registry.evehicle.cn;

ssl on;
ssl_certificate ssl/registry.evehicle.cn.crt;
ssl_certificate_key ssl/registry.evehicle.cn.key;

client_max_body_size 0; # disable any limits to avoid HTTP 413 for large image uploads

# required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
chunked_transfer_encoding on;

location / {
      auth_basic  "Restricted";
      auth_basic_user_file  passwd;
      add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always;

      proxy_pass                         http://docker-registry;
      proxy_set_header  Host             $http_host; # required for docker client's sake
      proxy_set_header  X-Real-IP       $remote_addr; # pass on real client's IP
      proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header  X-Forwarded-Proto $scheme;
      proxy_read_timeout                900;
      }

location /_ping {
      auth_basic off;
      include             docker-registry.conf;
}

location /v1/_ping {
      auth_basic off;
      include             docker-registry.conf;
}

location /v2/_ping {
      auth_basic off;
      include             docker-registry.conf;
}
}

将key及crt证书文件放到../ssl目录下。使用htpasswd生成密码放于./上一级目录

htpasswd -bcm passwd docker docker
#-c：创建一个加密文件
#-m：md5加密，默认可不填写
#-b：表示用户名密码在命令行中一并输入，不用分别填写

4.再次登录

[centos-master]:docker login -u docker -p docker -e lienhua@zhongchuangsanyou.com registry.evehicle.cn

Login Succeeded
表示成功，此时再pull\push既在私有库中进行

构建服务

docker的本意是将代码包含在容器内制作成镜像形成“产品”。但出于公司的（频繁修改代码及服务器资源受限）的特殊性，我们将代码以“外挂”的形式运行在宿主机上。下面以部署官网（apache）服务为例：
1.从docker的公有库里下载centos7的原生镜像

[centos-master]:docker pull centos

Using default tag: latest
Trying to pull repository docker.io/library/centos ...
latest: Pulling from docker.io/library/centos
d9aaf4d82f24: Downloading [>             ] 540 kB/73.39 MB
d9aaf4d82f24: Pulling fs layer
Digest: sha256:eba772bac22c86d7d6e72421b4700c3f894ab6e35475a34014ff8de74c10872e
Status: Downloaded newer image for centos:latest

2.编写Dockerfile制造apache基础镜像

######httpd####
FROM centos
MAINTAINER lienhua lienhua@zhongchuangsanyou.com
RUN yum -y install epel-release
RUN yum -y install httpd  php php-mysql php-memcache* php-mbstring
ADD httpd.conf /etc/httpd/conf/httpd.conf

EXPOSE 80

CMD ["/usr/sbin/apachectl", "-D", "FOREGROUND"]

其中httpd.conf文件需要在当前目录下真实存在，此处其内容为

ServerRoot "/etc/httpd"
Listen 80
Listen 8080
Include conf.modules.d/*.conf
Include zcsy/*.conf
User apache
Group apache
ServerAdmin root@localhost
<Directory />
AllowOverride none
Require all denied
</Directory>
DocumentRoot "/var/www/html"
<Directory "/var/www">
AllowOverride None
Require all granted
</Directory>
<Directory "/var/www/html">
Options Indexes FollowSymLinks
AllowOverride None
Require all granted
</Directory>
<IfModule dir_module>
DirectoryIndex index.html
</IfModule>
<Files ".ht*">
Require all denied
</Files>
ErrorLog "logs/error_log"
LogLevel warn
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
<IfModule logio_module>
   LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>
CustomLog "logs/access_log" combined
</IfModule>
<IfModule alias_module>
ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"
</IfModule>
<Directory "/var/www/cgi-bin">
AllowOverride None
Options None
Require all granted
</Directory>
<IfModule mime_module>
TypesConfig /etc/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
AddType application/x-httpd-php .php
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
</IfModule>
AddDefaultCharset UTF-8
<IfModule mime_magic_module>
MIMEMagicFile conf/magic
</IfModule>
EnableSendfile off
EnableMMAP off
IncludeOptional conf.d/*.conf

执行[centos-master]:docker build -t registry.evehicle.cn/httpd . 命令制作名为”registry.evehicle.cn/httpd”的镜像（注意此处的点必须要有，并且其意义代表当前目录下的Dockerfile文件）

3.将制作好的镜像上传到私有库

docker push registry.evehicle.cn/httpd

4.编写启动apache服务的yaml文件

[centos-master]:cat 13-rc-httpd.yaml

apiVersion: v1
kind: ReplicationController
metadata:
  name: 13-rc-httpd
  labels:
name: 13-rc-httpd
spec:
  replicas: 2
  selector:
name: 13-rc-httpd
  template:
metadata:
   labels:
      name: 13-rc-httpd
spec:
   containers:
   - name: 13-rc-httpd
      image: registry.evehicle.cn/httpd
      env:
      - name: LANG
      value: en_US.UTF-8
      ports:
      - containerPort: 80
      hostPort: 80
      volumeMounts:
      - name: time
      mountPath: /etc/localtime
      - name: zcsy
      mountPath: /etc/httpd/zcsy
      - name: deploy
      mountPath: /docker/httpd/deploy
      - name: log
      mountPath: /var/log/httpd
   volumes:
      - name: time
      hostPath:
         path: /etc/localtime
      - name: zcsy
      hostPath:
         path: /docker/httpd/zcsy
      - name: deploy
      hostPath:
         path: /docker/httpd/deploy
      - name: log
      hostPath:
         path: /docker/httpd/log
   nodeSelector:
      slave: "13"
   imagePullSecrets:
   - name: registrykey

5.给其中一个node加上标签为“13”

kubectl label nodes centos-minion-1 slave=13

6.此时拥有标签“13”的nodes应具备的条件

/docker/httpd/zcsy下需要有官网的配置文件

<VirtualHost *:80>
ServerName www.evehicle.cn
  DocumentRoot /var/deploy/wordpress/
      RewriteEngine on
      RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-d
      RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
      RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !^.*\.(ico|pdf|flv|jpe?g|js|gif|png|html|shtml|zip|xml|gz|rar|swf|txt|apk|bmp|css|m4a|ogg|mp3|ipa|plist)$
      RewriteCond %{REQUEST_URI} !^/server-status$
      RewriteRule . /index.php [QSA,PT,L]

</VirtualHost>
<Directory /var/deploy/wordpress/>
Options FollowSymLinks
AllowOverride All
Require all granted
</Directory>

以及/docker/httpd/deploy下需要有官网的代码

7.运行yaml文件启动容器

[centos-master]: kuberctl create -f 13-rc-httpd.yaml

8.查看服务

[centos-master]: kuberctl get rc

NAME                DESIRED CURRENT AGE
13-rc-httpd       2       2       168d

9.程序中涉及的mysql\redis\memcache等服务也需使用容器运行起来

[centos-master]: docker pull redis
[centos-master]: docker tag registry.evehicle.cn/redis redis
[centos-master]: docker push registry.evehicle.cn/redis
[centos-master]: kubectl create -f rc-redis.yaml
[centos-master]: cat rc-redis.yaml

apiVersion: v1
kind: ReplicationController
metadata:
  name: redis
  labels:
name: redis
spec:
  replicas: 2
  selector:
name: redis
  template:
metadata:
   labels:
      name: redis
spec:
   containers:
   - name: redis
      image: registry.evehicle.cn/redis
      ports:
      - containerPort: 6379
      hostPort: 6379
      volumeMounts:
      - name: data
      mountPath: /data
      - name: time
      mountPath: /etc/localtime
   volumes:
      - name: data
      hostPath:
         path: /docker/redis/6379
      - name: time
      hostPath:
         path: /etc/localtime
   nodeSelector:
      slave: "13"
   imagePullSecrets:
   - name: registrykey

启动memcache
[centos-master]: docker pull memcache
[centos-master]: docker tag registry.evehicle.cn/memcached memcache
[centos-master]: docker push registry.evehicle.cn/memcached
[centos-master]: kubectl create -f rc-memcached.yaml
[centos-master]: cat rc-memcached.yaml

apiVersion: v1
kind: ReplicationController
metadata:
  name: memcached
  labels:
name: memcached
spec:
  replicas: 3
  selector:
name: memcached
  template:
metadata:
   labels:
      name: memcached
spec:
   containers:
   - name: memcached
      image: registry.evehicle.cn/memcached
      ports:
      - containerPort: 11211
      hostPort: 11211
   #nodeSelector:
   #  slave: "13"
   imagePullSecrets:
   - name: registrykey

制造mysql镜像
[centos-master]: cat Dockerfile

FROM alpine

COPY startup.sh /startup.sh
RUN addgroup mysql && \
adduser -H -D -s /bin/false -G mysql mysql && \
apk add --update mysql mysql-client && rm -f /var/cache/apk/* && \
mkdir /data && \
chown -R mysql:mysql /data /etc/mysql && \
chmod 755 /startup.sh \
;

WORKDIR /data
VOLUME /data
VOLUME /etc/mysql

EXPOSE 3306
CMD ["/startup.sh"]

启动mysql（建议mysql在宿主机启动）
[centos-master]: docker build -t registry.evehicle.cn/mysql
[centos-master]: docker push registry.evehicle.cn/mysql
[centos-master]: kubectl create -f rc-mysql.yaml
[centos-master]: cat rc-mysql.yaml

apiVersion: v1
kind: ReplicationController
metadata:
  name: 13-rc-mysql
  labels:
name: 13-rc-mysql
spec:
  replicas: 2
  selector:
name: 13-rc-mysql
  template:
metadata:
   labels:
      name: 13-rc-mysql
spec:
   containers:
   - name: 13-rc-mysql
      image: registry.evehicle.cn/mysql
      env:
      - name: MYSQL_DATABASE
      value: admin
      - name: MYSQL_USER
      value: tony
      - name: MYSQL_PASSWORD
      value: 456
      - name: MYSQL_ROOT_PASSWORD
      value: 123
      ports:
      - containerPort: 3306
      hostPort: 3306
      volumeMounts:
      - name: time
      mountPath: /etc/localtime
      - name: data
      mountPath: /data
      - name: etc
      mountPath: /etc/mysql
      - name: run
      mountPath: /run/mysqld
   volumes:
      - name: time
      hostPath:
         path: /etc/localtime
      - name: data
      hostPath:
         path: /docker/mysql/data
      - name: etc
      hostPath:
         path: /docker/mysql/etc
      - name: run
      hostPath:
         path: /docker/mysql/run
   nodeSelector:
      slave: "13"
   imagePullSecrets:
   - name: registrykey

为方便代码编写及统一管理，应提前做好内部DNS解析。将所负责的应用规整到对应的机器上。

admin · 发表于 2018-9-20 16:11:19

kubectl config set-cluster default-cluster --server=http://192.168.121.9:8080
kubectl config set-context default-context --cluster=default-cluster --user=default-admin
kubectl config use-context default-context

admin · 发表于 2018-9-20 21:31:29

搭建私有库

私有库用于系统内部存储成品镜像，能够快速进行下载及被k8s调度。

1.下载并启动私有库

[centos-master]:docker run --name registry -v /etc/localtime:/etc/localtime -v /opt/registry:/var/lib/registry -p 5000:5000 -itd docker.io/registry

#--name 表示启动的容器后名称，此处为registry
#-v 表示挂载路径  格式为宿主机路径:容器内路径
#-p 表示映射端口  格式为宿主机端口:容器内端口
#-itd docker的内部参数，此处声明后台运行容器并分配一个伪终端并绑定到容器的标准输入上，后跟镜像名称此处为docker.io/registry

2.创建一个secret服务，用于k8s调度私有库容器时的“令牌”。简单来说，secret服务就是一个存储密码的服务

[centos-master]:kubectl create secret docker-registry registrykey --docker-server=registry.evehicle.cn --docker-username=docker --docker-password=docker --docker-email=lienhua@zhongchuangsanyou.com

[centos-master]:kubectl get secret
NAME       TYPE                   DATA    AGE
registrykey kubernetes.io/dockercfg 1       6s

此时登录时会提示认证错误

[centos-master]:docker login -u docker -p docker -e lienhua@zhongchuangsanyou.com registry.evehicle.cn
Flag --email has been deprecated, will be removed in 1.13.
Error response from daemon: login attempt to https://registry.evehicle.cn/v2/ failed with status: 401 Unauthorized

这是因为Docker官方是推荐采用Secure Registry的工作模式的，即transport采用tls。这样我们就需要为Registry配置tls所需的key和crt文件了

3.配置nginx反向代理
[centos-master]: cat registry.evehicle.cn.conf

# For versions of nginx > 1.3.9 that include chunked transfer encoding support
# Replace with appropriate values where necessary

upstream docker-registry {
  server 192.168.121.9:5000;
  #server 10.44.170.95:5000;
}

# uncomment if you want a 301 redirect for users attempting to connect
# on port 80
# NOTE: docker client will still fail. This is just for convenience
# server {
# listen *:80;
# server_name my.docker.registry.com;
# return 301 https://$server_name$request_uri;
# }

server {
listen 443;
server_name registry.evehicle.cn;

ssl on;
ssl_certificate ssl/registry.evehicle.cn.crt;
ssl_certificate_key ssl/registry.evehicle.cn.key;

client_max_body_size 0; # disable any limits to avoid HTTP 413 for large image uploads

# required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
chunked_transfer_encoding on;

location / {
      auth_basic  "Restricted";
      auth_basic_user_file  passwd;
      add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always;

      proxy_pass                         http://docker-registry;
      proxy_set_header  Host             $http_host; # required for docker client's sake
      proxy_set_header  X-Real-IP       $remote_addr; # pass on real client's IP
      proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header  X-Forwarded-Proto $scheme;
      proxy_read_timeout                900;
      }

location /_ping {
      auth_basic off;
      include             docker-registry.conf;
}

location /v1/_ping {
      auth_basic off;
      include             docker-registry.conf;
}

location /v2/_ping {
      auth_basic off;
      include             docker-registry.conf;
}
}

将key及crt证书文件放到../ssl目录下。使用htpasswd生成密码放于./上一级目录

htpasswd -bcm passwd docker docker
#-c：创建一个加密文件
#-m：md5加密，默认可不填写
#-b：表示用户名密码在命令行中一并输入，不用分别填写

4.再次登录

[centos-master]:docker login -u docker -p docker -e lienhua@zhongchuangsanyou.com registry.evehicle.cn

Login Succeeded
表示成功，此时再pull\push既在私有库中进行

		自动登录	找回密码
密码			注册