kubeadm部署Kubernetes集群高可用

FeiYi
Kubernetes

发布于：2021年2月15日
次浏览

字数：4k字

时长：20分钟

kubernetes官网

安装环境

ip Docker Version Kubernetes Version 硬件要求
192.168.1.11（master1） 19.03.8 v1.18.1 内存4G，双核CPU
192.168.1.12（master2） 19.03.8 v1.18.1 内存4G，双核CPU
192.168.1.13（master3） 19.03.8 v1.18.1 内存4G，双核CPU
192.168.1.100（VIP）

这里因为我使用的VMware虚拟机，数量有限，只做master，实际可以在多做很多node

安装步骤

环境准备

为了修改文件传输文件的方便，将主机名更改为安装环境所述的master，并写入hosts文件

本地hosts解析

master1

[root@localhost ~]# sed -i '$a\192.168.1.11 master1\n192.168.1.12 master2\n192.168.1.13 master3' /etc/hosts

ssh免密

方便传输文件

master1

[root@localhost ~]# ssh-keygen
[root@localhost ~]# ssh-copy-id -i root@master2
[root@localhost ~]# ssh-copy-id -i root@master3

传输hosts文件

[root@localhost ~]# scp /etc/hosts root@master2:/etc
[root@localhost ~]# scp /etc/hosts root@master3:/etc

更改主机名

master1

[root@localhost ~]# hostnamectl set-hostname master1
[root@localhost ~]# bash
[root@master1 ~]#

master2

[root@localhost ~]# hostnamectl set-hostname master2
[root@localhost ~]# bash
[root@master2 ~]#

master3

[root@localhost ~]# hostnamectl set-hostname master3
[root@localhost ~]# bash
[root@master3 ~]#

关闭防火墙沙盒

master1/2/3

systemctl stop firewalld && systemctl disable firewalld
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/g' /etc/selinux/config

kubernetes安装环境要求

官方文档

防火墙端口

如果不关闭防火墙，请放下以下端口

Master节点

协议 方向 端口范围 目的 使用者
TCP协议 入站 64430-64439 Kubernetes API server 所有
TCP协议 入站 2379-2380 etcd server client API kube-apiserver, etcd
TCP协议 入站 10250 Kubelet API Self, Control plane
TCP协议 入站 10251 kube-scheduler Self
TCP协议 入站 10252 kube-controller-manager Self

Node节点

协议 方向 端口范围 目的 使用者
TCP协议 入站 10250 Kubelet API Self, Control plane
TCP协议 入站 30000-32767 NodePort Services† 所有

验证每个节点的mac地址和product_uuid是唯一的

kubernetes通过这两个值来确定集群中的节点

node1

[root@master1 ~]# cat /sys/class/dmi/id/product_uuid
E2B74D56-23A9-4E8B-620C-555387355616

node2

[root@master2 ~]# cat /sys/class/dmi/id/product_uuid
371E4D56-E8D8-294C-DA42-D06C62FD9B62

node3

[root@master3 ~]# cat /sys/class/dmi/id/product_uuid
96E94D56-D4D8-18DC-FDAE-89060C320C7A

iptables桥接流量

因为增加内存的原因，需要关机，所以服务需要重启

systemctl start docker && systemctl enable docker

master1

关于iptables的两项配置依赖于docker，docker服务必须启动

[root@master1 ~]# vim /etc/sysctl.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
vm.swappiness = 0   # 当内存oom时不使用交换分区

如果以上两条iptables的参数不生效，再执行modprobe ip_vs_rr和
modprobe br_netfilter

在执行sysctl -p之前要确认docker服务已经启动了

sysctl -p
modprobe  ip_vs_rr
modprobe br_netfilter
scp /etc/sysctl.conf root@master2:/etc
scp /etc/sysctl.conf root@master3:/etc

master2

sysctl -p
modprobe  ip_vs_rr
modprobe br_netfilter

master3

sysctl -p
modprobe  ip_vs_rr
modprobe br_netfilter

关闭swap分区

三台操作同样，这一步不要scp，因为fstab不一样

swapoff -a
sed -i 's/^[^#].*swap*/#&/g' /etc/fstab
# 将分区类型为swap的一行注释掉

查看是否关闭

[root@master1 ~]# free -m
           total        used        free      shared  buff/cache   available
Mem:        3770         793        2151          21         825        2732
Swap:          0           0           0

Haproxy搭建

这里使用yum安装，方便实验，生产环境可根据实际情况使用源码安装

三台作为master服务器都需要安装

yum -y install haproxy

三台的配置文件一致即可

$ vim /etc/haproxy/haproxy.cfg
# 需要修改的地方使用中文注释已经标识
global

    log         127.0.0.1 local2

    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

    stats socket /var/lib/haproxy/stats

defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000

frontend  kubernetes-apiserver  # 与下方backend的名字一致
    mode  tcp   # 使用tcp协议
    bind  *:9443  # kube默认端口是6443，这里使用9443进行负载6443
    acl url_static       path_beg       -i /static /images /javascript /stylesheets
    acl url_static       path_end       -i .jpg .gif .png .css .js

    use_backend static          if url_static
    default_backend             kubernetes-apiserver  # 这里的值修改为下方定义的backend

backend static
    balance     roundrobin
    server      static 127.0.0.1:4331 check

backend kubernetes-apiserver  # 自定义名字为kubernetes-apiserver
    mode    tcp   # 使用tcp协议
    balance     roundrobin   # 轮询方式为RR轮询
    # 下方将所有master的api地址设置好，也可以配置更多的haproxy的健康检查方式
    server  master1 192.168.1.11:6443 check
    server  master2 192.168.1.12:6443 check
    server  master3 192.168.1.13:6443 check

启动服务

systemctl enable haproxy && systemctl start haproxy

Keepalived搭建

这里使用yum安装，方便实验，生产环境可根据实际情况使用源码安装

三台作为master服务器都需要安装

yum clean all && yum -y install keepalived

配置文件修改（master1）

[root@master1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id master1  # 改为hostname即可，不改也行
   vrrp_skip_check_adv_addr
 # vrrp_strict    # 将该行注释，否则会严格执行vrrp协议，导致VIP无法通信
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state BACKUP  # 一共三台，都为BACKUP，会根据priority选举master
    interface ens33  # 修改为本机使用的网络设备名
    virtual_router_id 51  # 三台必须一致，可以不修改
    priority 100  # 优先级，其他两台都比100低即可
    advert_int 1
    nopreempt   # 添加，故障修复后不抢占ip
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.100   # VIP
    }
}
# 以下都删除
[root@master1 ~]# scp /etc/keepalived/keepalived.conf master2:/etc/keepalived/ 
[root@master1 ~]# scp /etc/keepalived/keepalived.conf master3:/etc/keepalived/

配置文件修改（master2）

[root@master2 ~]# vim /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id master2   # 修改为master2的hostname
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 51
    priority 90  #优先级为90
    advert_int 1
    nopreempt
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.100
    }
}

配置文件修改（master3）

[root@master3 ~]# vim /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id master3  # 修改为master2的hostname
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 51
    priority 80  # 优先级为80
    advert_int 1
    nopreempt
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.100
    }
}

三台同时启动Keepalived

systemctl start keepalived && systemctl enable keepalived

通过ip addr查看到VIP在master1上

[root@master1 ~]# ip addr | grep 192.168.1.100
    inet 192.168.1.100/32 scope global ens33

修改Docker默认Cgroup Driver

Docker默认的Cgroup Driver是cgroupfs，而Kubernetes推荐使用systemd

三台全部修改

vim /etc/docker/daemon.json
{
  "registry-mirrors": ["https://519ref89.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"]
}

加载配置文件并重启docker

systemctl daemon-reload && systemctl restart docker

这里我们设置的优先级为：master1>master2>master3，所以VIP所在的位置是master1

[root@master1 ~]# ip a | grep 192.168.1.100 
    inet 192.168.1.100/32 scope global ens33

kubernetes安装

这里使用阿里云镜像站安装

kubernetes阿里云镜像站

master1

进入上面这个页面，使用centos的yum源

[root@master1 ~]# cat << EOF >> /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

将yum源传给每台主机

scp /etc/yum.repos.d/kubernetes.repo root@master2:/etc/yum.repos.d/
scp /etc/yum.repos.d/kubernetes.repo root@master3:/etc/yum.repos.d/

master1/2/3

yum install -y kubelet-1.18.1 kubeadm-1.18.1 kubectl-1.18.1

安装kubernetes的tab命令补齐

yum -y install bash-completion
vim .bashrc
# 末尾添加
source <(kubeadm completion bash)
source <(kubectl completion bash)
source .bashrc

三台启动服务

systemctl enable kubelet.service && systemctl start kubelet.service

此时会启动失败，因为此时的配置还没初始化完成，所以此时不能启动kubelet,等后续kubeadm启动成功后再查看

kubernetes创建集群

master1

执行这条后会等待一段时间，需要进行下载镜像

提前下载所需镜像(三台都要执行)

kubeadm config images pull --kubernetes-version=v1.18.1 --image-repository=registry.aliyuncs.com/google_containers

使用配置文件初始化集群

cat <<EOF > ./kubeadm-init.yaml
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.18.1
# 修改镜像仓库为阿里云，可以访问国外网也可以不修改
imageRepository: registry.aliyuncs.com/google_containers  
apiServer:
  certSANs:  # apiserver下添加certSANs字段，并填写集群中所有节点的hostname、ip以及VIP
  - master1
  - master2
  - master3
  - 192.168.1.11
  - 192.168.1.12
  - 192.168.1.13
  - 192.168.1.100
  timeoutForControlPlane: 4m0s
controlPlaneEndpoint: "192.168.1.100:9443"  # 此处的ip为VIP，端口号为Haproxy使用的9443
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
  podSubnet: "10.16.0.0/16" 
EOF

初始化集群

[root@master1 ~]# kubeadm init --config=kubeadm-init.yaml  --upload-certs

完成后会有以下输出信息

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:
# 要控制集群，先执行这三条命令，方便调用api
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
# 部署集群网络插件的方法
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:
# 想要添加新的master节点，需要复制证书文件到目标主机，在目标主机执行此命令
kubeadm join 192.168.1.100:6443 --token tdgw3z.yalniqaw3vasy0oo \
    --discovery-token-ca-cert-hash sha256:23b97ec2e162b7777bfa8c93c4e37046c43d284e612c63d68e7eb412c4051b59\
    --control-plane --certificate-key 10ff7b70185b3e71172fab68f31ce4d7957b92682868f1d3345effeb177aecaa

Then you can join any number of worker nodes by running the following on each as root:
# 想要添加新的worker节点，直接在其他主机执行此命令
kubeadm join 192.168.1.100:6443 --token tdgw3z.yalniqaw3vasy0oo \
    --discovery-token-ca-cert-hash sha256:23b97ec2e162b7777bfa8c93c4e37046c43d284e612c63d68e7eb412c4051b59

按照以上，先设置控制集群的api

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

验证命令的使用

[root@master1 ~]# kubectl get nodes
NAME      STATUS     ROLES    AGE     VERSION
master1   NotReady   master   5m40s   v1.18.1

master2/3加入集群

master2/3执行加入集群命令

kubeadm join 192.168.1.100:6443 --token tdgw3z.yalniqaw3vasy0oo \
    --discovery-token-ca-cert-hash sha256:23b97ec2e162b7777bfa8c93c4e37046c43d284e612c63d68e7eb412c4051b59\
    --control-plane --certificate-key 10ff7b70185b3e71172fab68f31ce4d7957b92682868f1d3345effeb177aecaa

完成后输出一下信息

This node has joined the cluster and a new control plane instance was created:

* Certificate signing request was sent to apiserver and approval was received.
* The Kubelet was informed of the new secure connection details.
* Control plane (master) label and taint were applied to the new node.
* The Kubernetes control plane instances scaled up.
* A new etcd member was added to the local/stacked etcd cluster.

To start administering your cluster from this node, you need to run the following as a regular user:

    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config

Run 'kubectl get nodes' to see this node join the cluster.

作为master节点，也需要调用api来控制集群

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

验证查看

[root@master2 ~]# kubectl get nodes
NAME    STATUS     ROLES    AGE    VERSION
node1   NotReady   master   18m    v1.18.1
node2   NotReady   master   2m7s   v1.18.1
node3   NotReady   master   60s    v1.18.1

部署网络插件

这里使用kubeovn，pod网段为10.16.0.0/16

kubeovn插件的部署，有独特的node label，由于是多master，所以所有master节点都要添加lable

[root@master1 ~]# kubectl label nodes master1 master2 master3 kube-ovn/role=master
node/master1 labeled
node/master2 labeled
node/master3 labeled

提前下载镜像，减少脚本等待时间，使用的kubeovn版本为v1.6.0

# 三台master都要拉取镜像
docker pull kubeovn/kube-ovn:v1.6.0

使用了代理的网络可直接执行脚本

wget https://raw.githubusercontent.com/alauda/kube-ovn/v1.6.0/dist/images/install.sh
chmod +x install.sh
./install.sh

国内网络的用户可以访问github项目并复制脚本内容，粘贴到服务器，也可以使用我站点的文件

github->install.sh

wget https://www.feiyiblog.com/files/kubeovn/install.sh

脚本执行完成后，查看节点是否全部Ready

[root@master1 ~]# kubectl get nodes
NAME    STATUS   ROLES    AGE   VERSION
master1   Ready    master   30m   v1.18.1
master2   Ready    master   13m   v1.18.1
master3   Ready    master   12m   v1.18.1

如果想使用其他类似跳板机来管理集群，在能够与集群ip互通和Docker&Kubernetes的版本一致的前提下，可以将master节点的/etc/kubernetes/admin.conf文件复制到跳板机的$HOME/.kube/config

验证Keepalived+Haproxy

查看Haproxy的9443端口

master1

[root@master1 ~]# netstat -anput | grep 9443
tcp        0      0 0.0.0.0:9443            0.0.0.0:*               LISTEN      10578/haproxy       
tcp        0      0 192.168.1.100:40970     192.168.1.100:9443      ESTABLISHED 19345/kube-proxy    
tcp        0      0 192.168.1.100:9443      192.168.1.13:41112      ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.13:41196      ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.100:40874     ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.100:42592     ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.12:48618      ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.13:58758      ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.100:40970     ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:32789     192.168.1.100:9443      ESTABLISHED 20329/kube-schedule 
tcp        0      0 192.168.1.100:42592     192.168.1.100:9443      ESTABLISHED 20322/kube-controll 
tcp        0      0 192.168.1.100:42604     192.168.1.100:9443      ESTABLISHED 20329/kube-schedule 
tcp        0      0 192.168.1.100:9443      192.168.1.14:56264      ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.12:58698      ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.100:32789     ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.13:40896      ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.100:42604     ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:40874     192.168.1.100:9443      ESTABLISHED 18963/kubelet       
tcp        0      0 192.168.1.100:9443      192.168.1.13:41304      ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.12:58632      ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.13:40848      ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.14:56242      ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.12:58364      ESTABLISHED 10578/haproxy       
tcp        0      0 192.168.1.100:9443      192.168.1.12:58756      ESTABLISHED 10578/haproxy

master2

[root@master2 ~]# netstat -anput | grep 9443
tcp        0      0 0.0.0.0:9443            0.0.0.0:*               LISTEN      9341/haproxy        
tcp        0      0 192.168.1.12:58632      192.168.1.100:9443      ESTABLISHED 11534/kubelet       
tcp        0      0 192.168.1.12:46332      192.168.1.100:9443      ESTABLISHED 12066/kube-schedule 
tcp        0      0 192.168.1.12:58364      192.168.1.100:9443      ESTABLISHED 11808/kube-proxy    
tcp        0      0 192.168.1.12:58756      192.168.1.100:9443      ESTABLISHED 12053/kube-controll 
tcp        0      0 192.168.1.12:58698      192.168.1.100:9443      ESTABLISHED 12066/kube-schedule

master3

[root@master3 ~]# netstat -anput | grep 9443
tcp        0      0 0.0.0.0:9443            0.0.0.0:*               LISTEN      10784/haproxy       
tcp        0      0 192.168.1.13:57234      192.168.1.100:9443      ESTABLISHED 14262/kube-schedule 
tcp        0      0 192.168.1.13:40848      192.168.1.100:9443      ESTABLISHED 14265/kube-controll 
tcp        0      0 192.168.1.13:41112      192.168.1.100:9443      ESTABLISHED 14265/kube-controll 
tcp        0      0 192.168.1.13:41196      192.168.1.100:9443      ESTABLISHED 14786/kubelet       
tcp        0      0 192.168.1.13:40896      192.168.1.100:9443      ESTABLISHED 14262/kube-schedule 
tcp        0      0 192.168.1.13:41304      192.168.1.100:9443      ESTABLISHED 15917/kube-proxy

模拟master1故障，停掉Keepalived或者关机都可以，这里选择关机来更加真实的模拟

[root@master1 ~]# poweroff

按照Keepalived设置的优先级，VIP应该漂移到master2上

[root@master2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:57:1a:8f brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.14/24 brd 192.168.1.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.1.100/32 scope global ens33  # VIP
       valid_lft forever preferred_lft forever
    inet6 fe80::e898:9670:7d6d:6182/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

查看node状态

[root@master2 ~]# kubectl get nodes
NAME      STATUS     ROLES    AGE   VERSION
master1   NotReady   master   19m   v1.18.1
master2   Ready      master   16m   v1.18.1
master3   Ready      master   16m   v1.18.1

master1已经宕机了

重新开机后，不抢占master2机器的VIP，并且集群可以正常使用，如果master1彻底损坏，可以执行kubectl delete nodes master1，将master1踢出集群，重新加入一台master即可

关于Keepalived三台全都是BACKUP的说明

以上的Keepalived配置过程中，三台的state的配置都是BACKUP，这里为什么不用master，是因为nopreempt参数，这个参数的作用是当一台宕机后，重新启动不抢占VIP，但是只能搭配BACKUP来实现，且一开始的master选举是由priority的值的高低来竞选的，不需要人工参与。

博客内容遵循 署名-非商业性使用-相同方式共享 4.0 国际 (CC BY-NC-SA 4.0) 协议

本文永久链接是：https://www.feiyiblog.com/2021/02/15/kubeadm%E9%83%A8%E7%BD%B2Kubernetes%E9%9B%86%E7%BE%A4%E9%AB%98%E5%8F%AF%E7%94%A8/

更新于：2021年5月2日

Kubernetes

字数：4k字

时长：20分钟

评论




正在载入...
京ICP备-20009585号
PoweredHexo
HostedAliyun
DNSAliyun
ThemeVolantis
UV
PV
BY-NC-SA 4.0