Kubernetes 容器集群Pod滚动更新特性

 云计算, 容器化  No Responses »
5 月 062020
 

滚动更新（Rolling Update）通过策略控制每次更新副本的数量来保障业务连续性。

准备使用httpd:2.4.41版本镜像的配置文件

[root@k8s-01 ~]# vi httpd-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpd
spec:
  replicas: 3
  selector:
    matchLabels:
      run: httpd
  template:
    metadata:
      labels:
        run: httpd
    spec:
      containers:
      - name: httpd
        image: httpd:2.4.41
        ports:
        - containerPort: 80

应用配置文件并获取deployment和replicaset及pod列表信息

[root@k8s-01 ~]# kubectl apply -f httpd-deployment.yaml
deployment.apps/httpd created
[root@k8s-01 ~]#
[root@k8s-01 ~]# kubectl get deployments.apps httpd -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES         SELECTOR
httpd   3/3     3            3           20s   httpd        httpd:2.4.41   run=httpd
[root@k8s-01 ~]#
[root@k8s-01 ~]# kubectl get replicasets.apps -o wide
NAME               DESIRED   CURRENT   READY   AGE   CONTAINERS   IMAGES         SELECTOR
httpd-5bb8cdb99c   3         3         3       36s   httpd        httpd:2.4.41   pod-template-hash=5bb8cdb99c,run=httpd
[root@k8s-01 ~]#
[root@k8s-01 ~]# kubectl get pods -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP           NODE     NOMINATED NODE   READINESS GATES
httpd-5bb8cdb99c-454mz   1/1     Running   0          51s   10.244.2.4   k8s-03   <none>           <none>
httpd-5bb8cdb99c-qlzbh   1/1     Running   0          51s   10.244.1.5   k8s-02   <none>           <none>
httpd-5bb8cdb99c-rpt59   1/1     Running   0          51s   10.244.1.6   k8s-02   <none>           <none>
[root@k8s-01 ~]#

修改配置文件为使用httpd:2.4.43版本镜像

apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpd
spec:
  replicas: 3
  selector:
    matchLabels:
      run: httpd
  template:
    metadata:
      labels:
        run: httpd
    spec:
      containers:
      - name: httpd
        image: httpd:2.4.43
        ports:
        - containerPort: 80

应用配置文件并获取deployment和replicaset列表信息

[root@k8s-01 ~]# kubectl apply -f httpd-deployment.yaml
deployment.apps/httpd configured
[root@k8s-01 ~]# kubectl get deployments.apps httpd -o wide
NAME    READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES         SELECTOR
httpd   3/3     3            3           3m2s   httpd        httpd:2.4.43   run=httpd
[root@k8s-01 ~]# kubectl get replicasets.apps -o wide
NAME               DESIRED   CURRENT   READY   AGE     CONTAINERS   IMAGES         SELECTOR
httpd-5bb8cdb99c   0         0         0       3m11s   httpd        httpd:2.4.41   pod-template-hash=5bb8cdb99c,run=httpd
httpd-7c68f97dc5   3         3         3       24s     httpd        httpd:2.4.43   pod-template-hash=7c68f97dc5,run=httpd
[root@k8s-01 ~]#

查看滚动更新详情（每次只更新替换一个低版本镜像Pod）

[root@k8s-01 ~]# kubectl describe deployments.apps httpd
Name:                   httpd
Namespace:              default
CreationTimestamp:      Wed, 06 May 2020 09:20:14 +0000
Labels:                 <none>
Annotations:            deployment.kubernetes.io/revision: 2
                        kubectl.kubernetes.io/last-applied-configuration:
                          {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"name":"httpd","namespace":"default"},"spec":{"replicas":3,"selec...
Selector:               run=httpd
Replicas:               3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  run=httpd
  Containers:
   httpd:
    Image:        httpd:2.4.43
    Port:         80/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   httpd-7c68f97dc5 (3/3 replicas created)
Events:
  Type    Reason             Age    From                   Message
  ----    ------             ----   ----                   -------
  Normal  ScalingReplicaSet  4m27s  deployment-controller  Scaled up replica set httpd-5bb8cdb99c to 3
  Normal  ScalingReplicaSet  100s   deployment-controller  Scaled up replica set httpd-7c68f97dc5 to 1
  Normal  ScalingReplicaSet  93s    deployment-controller  Scaled down replica set httpd-5bb8cdb99c to 2
  Normal  ScalingReplicaSet  93s    deployment-controller  Scaled up replica set httpd-7c68f97dc5 to 2
  Normal  ScalingReplicaSet  85s    deployment-controller  Scaled down replica set httpd-5bb8cdb99c to 1
  Normal  ScalingReplicaSet  85s    deployment-controller  Scaled up replica set httpd-7c68f97dc5 to 3
  Normal  ScalingReplicaSet  84s    deployment-controller  Scaled down replica set httpd-5bb8cdb99c to 0
[root@k8s-01 ~]#

K8S集群内Service的NodePort访问方式

 云计算, 容器化  No Responses »
5 月 062020
 

获取集群内的服务列表（类型为ClusterIP）

[root@k8s-01 ~]# kubectl get service -o wide
NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE   SELECTOR
httpd-service   ClusterIP   10.109.145.140   <none>        8080/TCP   78m   run=httpd
kubernetes      ClusterIP   10.96.0.1        <none>        443/TCP    85m   <none>
[root@k8s-01 ~]#

修改服务配置文件以添加NodePort配置并应用

[root@k8s-01 ~]# vi httpd-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: httpd-service
spec:
  type: NodePort
  selector:
    run: httpd
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 80
[root@k8s-01 ~]# kubectl apply -f httpd-service.yaml
service/httpd-service configured
[root@k8s-01 ~]#

获取集群内的服务列表（类型为NodePort）

[root@k8s-01 ~]# kubectl get service -o wide
NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE   SELECTOR
httpd-service   NodePort    10.109.145.140   <none>        8080:30093/TCP   81m   run=httpd
kubernetes      ClusterIP   10.96.0.1        <none>        443/TCP          88m   <none>
[root@k8s-01 ~]#

使用节点的IP+Port方式访问集群内的服务（借助iptbales实现负载均衡的包转发）

[root@k8s-01 ~]# curl 167.99.108.90:30093
<html><body><h1>It works!</h1></body></html>
[root@k8s-01 ~]# curl 206.189.165.254:30093
<html><body><h1>It works!</h1></body></html>
[root@k8s-01 ~]# curl 167.99.108.90:30093
<html><body><h1>It works!</h1></body></html>
[root@k8s-01 ~]#

为NodePort指定固定端口号（默认为30000-32767的随机端口号）

[root@k8s-01 ~]# vi httpd-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: httpd-service
spec:
  type: NodePort
  selector:
    run: httpd
  ports:
  - protocol: TCP
    nodePort: 31234
    port: 8080
    targetPort: 80
[root@k8s-01 ~]# kubectl apply -f httpd-service.yaml
service/httpd-service configured
[root@k8s-01 ~]#

获取集群内的服务列表

[root@k8s-01 ~]# kubectl  get services -o wide
NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE    SELECTOR
httpd-service   NodePort    10.109.145.140   <none>        8080:31234/TCP   93m    run=httpd
kubernetes      ClusterIP   10.96.0.1        <none>        443/TCP          100m   <none>
[root@k8s-01 ~]#

端口类型说明

nodePort：节点监听端口
port：ClusterIP监听端口
targetPort：Pod监听端口

Kubernetes集群指定组件版本安装脚本

 未分类  No Responses »
5 月 062020
  
#!/bin/bash
#

# 禁用SELINUX设置
setenforce 0;
sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config;
# 加载内核模块及修改内核参数
cat > /etc/modules-load.d/containerd.conf <<EOF
overlay
br_netfilter
EOF
modprobe overlay;
modprobe br_netfilter;
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system;
# 准备容器运行环境
yum makecache;
yum install -y yum-utils;
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo;
yum install -y docker-ce docker-ce-cli containerd.io;
containerd config default > /etc/containerd/config.toml;
#
mkdir /etc/docker;
cat <<EOF > /etc/docker/daemon.json
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF
#
systemctl daemon-reload;
systemctl enable docker;
systemctl restart docker;
# 准备K8S运行环境（指定组件版本）
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
#
yum install -y kubectl-1.16.9 kubelet-1.16.9 kubeadm-1.16.9;
systemctl enable kubelet;

Kubernetes 1.16.9 当前支持的经验证的Docker CE版本为18.09

[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.8. Latest validated version: 18.09

查看Docker CE YUM仓库当前可用版本（18.09.9）

[root@k8s-01 ~]# yum list docker-ce --showduplicates | sort -r
 * updates: mirrors.sonic.net
Loading mirror speeds from cached hostfile
Loaded plugins: fastestmirror
Installed Packages
 * extras: mirror.keystealth.org
docker-ce.x86_64            3:19.03.8-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:19.03.8-3.el7                    @docker-ce-stable
docker-ce.x86_64            3:19.03.7-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:19.03.6-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:19.03.5-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:19.03.4-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:19.03.3-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:19.03.2-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:19.03.1-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:19.03.0-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:18.09.9-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:18.09.8-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:18.09.7-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:18.09.6-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:18.09.5-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:18.09.4-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:18.09.3-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:18.09.2-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:18.09.1-3.el7                    docker-ce-stable 
docker-ce.x86_64            3:18.09.0-3.el7                    docker-ce-stable 
docker-ce.x86_64            18.06.3.ce-3.el7                   docker-ce-stable 
docker-ce.x86_64            18.06.2.ce-3.el7                   docker-ce-stable 
docker-ce.x86_64            18.06.1.ce-3.el7                   docker-ce-stable 
docker-ce.x86_64            18.06.0.ce-3.el7                   docker-ce-stable 
docker-ce.x86_64            18.03.1.ce-1.el7.centos            docker-ce-stable 
docker-ce.x86_64            18.03.0.ce-1.el7.centos            docker-ce-stable 
docker-ce.x86_64            17.12.1.ce-1.el7.centos            docker-ce-stable 
docker-ce.x86_64            17.12.0.ce-1.el7.centos            docker-ce-stable 
docker-ce.x86_64            17.09.1.ce-1.el7.centos            docker-ce-stable 
docker-ce.x86_64            17.09.0.ce-1.el7.centos            docker-ce-stable 
docker-ce.x86_64            17.06.2.ce-1.el7.centos            docker-ce-stable 
docker-ce.x86_64            17.06.1.ce-1.el7.centos            docker-ce-stable 
docker-ce.x86_64            17.06.0.ce-1.el7.centos            docker-ce-stable 
docker-ce.x86_64            17.03.3.ce-1.el7                   docker-ce-stable 
docker-ce.x86_64            17.03.2.ce-1.el7.centos            docker-ce-stable 
docker-ce.x86_64            17.03.1.ce-1.el7.centos            docker-ce-stable 
docker-ce.x86_64            17.03.0.ce-1.el7.centos            docker-ce-stable 
 * base: sjc.edge.kernel.org
Available Packages
[root@k8s-01 ~]#

Kuberntes集群中pod中容器服务的访问

 云计算, 容器化  No Responses »
5 月 062020
 

Kubernetes集群中的Service从逻辑上代表了一组Pod，并通过label建立与pod的关联

准备Deployment配置文件

[root@k8s-01 ~]# vi httpd-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpd
spec:
  replicas: 3
  selector:
    matchLabels:
      run: httpd
  template:
    metadata:
      labels:
        run: httpd
    spec:
      containers:
      - name: httpd
        image: httpd:2.4.41
        ports:
        - containerPort: 80
[root@k8s-01 ~]# kubectl apply -f httpd-deployment.yaml 
deployment.apps/httpd created
[root@k8s-01 ~]#

获取集群pod列表详情

[root@k8s-01 ~]# kubectl get pods -o wide
NAME                     READY   STATUS    RESTARTS   AGE     IP           NODE     NOMINATED NODE   READINESS GATES
httpd-5bb8cdb99c-g5m95   1/1     Running   0          4m29s   10.244.2.3   k8s-03   <none>           <none>
httpd-5bb8cdb99c-hzjqd   1/1     Running   0          4m29s   10.244.1.3   k8s-02   <none>           <none>
httpd-5bb8cdb99c-s4q25   1/1     Running   0          4m29s   10.244.1.4   k8s-02   <none>           <none>
[root@k8s-01 ~]#

使用CURL模拟浏览器请求pod的IP地址（Pod的IP地址只能被集群中的容器和节点访问到）

[root@k8s-01 ~]# curl 10.244.2.3
<html><body><h1>It works!</h1></body></html>
[root@k8s-01 ~]# curl 10.244.1.3
<html><body><h1>It works!</h1></body></html>
[root@k8s-01 ~]# curl 10.244.1.4
<html><body><h1>It works!</h1></body></html>
[root@k8s-01 ~]#

[root@k8s-02 ~]# curl 10.244.2.3
<html><body><h1>It works!</h1></body></html>
[root@k8s-02 ~]# curl 10.244.1.3
<html><body><h1>It works!</h1></body></html>
[root@k8s-02 ~]# curl 10.244.1.4
<html><body><h1>It works!</h1></body></html>
[root@k8s-02 ~]#

[root@k8s-03 ~]# curl 10.244.2.3
<html><body><h1>It works!</h1></body></html>
[root@k8s-03 ~]# curl 10.244.1.3
<html><body><h1>It works!</h1></body></html>
[root@k8s-03 ~]# curl 10.244.1.4
<html><body><h1>It works!</h1></body></html>
[root@k8s-03 ~]#

对Pod IP进行PING测试

[root@k8s-01 ~]# ping -c 2 10.244.2.3
PING 10.244.2.3 (10.244.2.3) 56(84) bytes of data.
64 bytes from 10.244.2.3: icmp_seq=1 ttl=63 time=2.03 ms
64 bytes from 10.244.2.3: icmp_seq=2 ttl=63 time=0.660 ms

--- 10.244.2.3 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.660/1.348/2.036/0.688 ms
[root@k8s-01 ~]# ping -c 2 10.244.1.3
PING 10.244.1.3 (10.244.1.3) 56(84) bytes of data.
64 bytes from 10.244.1.3: icmp_seq=1 ttl=63 time=1.58 ms
64 bytes from 10.244.1.3: icmp_seq=2 ttl=63 time=0.641 ms

--- 10.244.1.3 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.641/1.115/1.589/0.474 ms
[root@k8s-01 ~]# ping -c 2 10.244.1.4
PING 10.244.1.4 (10.244.1.4) 56(84) bytes of data.
64 bytes from 10.244.1.4: icmp_seq=1 ttl=63 time=0.658 ms
64 bytes from 10.244.1.4: icmp_seq=2 ttl=63 time=0.483 ms

--- 10.244.1.4 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.483/0.570/0.658/0.090 ms
[root@k8s-01 ~]#

创建服务Service配置文件

[root@k8s-01 ~]# vi httpd-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: httpd-service
spec:
  selector:
    run: httpd
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 80
[root@k8s-01 ~]# kubectl apply -f httpd-service.yaml
service/httpd-service created
[root@k8s-01 ~]#

获取集群Service列表详情

[root@k8s-01 ~]# kubectl get services -o wide
NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE    SELECTOR
httpd-service   ClusterIP   10.109.145.140   <none>        8080/TCP   4m9s   run=httpd
kubernetes      ClusterIP   10.96.0.1        <none>        443/TCP    10m    <none>
[root@k8s-01 ~]#

尝试ping集群IP地址（默认无法ping通）

[root@k8s-01 ~]# ping 10.109.145.140
PING 10.109.145.140 (10.109.145.140) 56(84) bytes of data.
^C
--- 10.109.145.140 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 1999ms

[root@k8s-01 ~]#

使用Service获得的集群IP访问具有run=httpd标签的后端Pod及容器

[root@k8s-01 ~]# curl 10.109.145.140:8080
<html><body><h1>It works!</h1></body></html>
[root@k8s-01 ~]# curl 10.109.145.140:8080
<html><body><h1>It works!</h1></body></html>
[root@k8s-01 ~]# curl 10.109.145.140:8080
<html><body><h1>It works!</h1></body></html>
[root@k8s-01 ~]# curl -I 10.109.145.140:8080
HTTP/1.1 200 OK
Date: Wed, 06 May 2020 07:24:57 GMT
Server: Apache/2.4.41 (Unix)
Last-Modified: Mon, 11 Jun 2007 18:53:14 GMT
ETag: "2d-432a5e4a73a80"
Accept-Ranges: bytes
Content-Length: 45
Content-Type: text/html

[root@k8s-01 ~]#

获取服务详情以确认Cluster IP指向的后端Pod IP信息

[root@k8s-01 ~]# kubectl describe services httpd-service
Name:              httpd-service
Namespace:         default
Labels:            <none>
Annotations:       kubectl.kubernetes.io/last-applied-configuration:
                     {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"httpd-service","namespace":"default"},"spec":{"ports":[{"port":80...
Selector:          run=httpd
Type:              ClusterIP
IP:                10.109.145.140
Port:              <unset>  8080/TCP
TargetPort:        80/TCP
Endpoints:         10.244.1.3:80,10.244.1.4:80,10.244.2.3:80
Session Affinity:  None
Events:            <none>
[root@k8s-01 ~]#
[root@k8s-01 ~]# kubectl get endpoints httpd-service
NAME            ENDPOINTS                                   AGE
httpd-service   10.244.1.3:80,10.244.1.4:80,10.244.2.3:80   5m23s
[root@k8s-01 ~]#