KubernetesClusterHA如何配置
本篇内容介绍了“Kubernetes Cluster HA如何配置”的有关知识，在实际案例的操作过程中，不少人都会遇到这样的困境，接下来就让小编带领大家学习一下如何处理这些情况吧！希望大家仔细阅读，能够学有所成！
创新互联建站坚持“要么做到，要么别承诺”的工作理念，服务领域包括：成都网站建设、成都网站制作、企业官网、英文网站、手机端网站、网站推广等服务，满足客户于互联网时代的东川网站设计、移动媒体设计的需求，帮助企业找到有效的互联网解决方案。努力成为您成熟可靠的网络建设合作伙伴！
Kubernetes Master HA架构图
配置与说明
所有组件可以通过kubelet static pod的方式启动和管理，由kubelet static pod机制保证宿主机上各个组件的高可用, 注意kubelet要添加配置
--allow-privileged=true;管理static pod的kubelet的高可用通过systemd来负责；
当然，你也可以直接通过进程来部署这些组件，systemd来直接管理这些进程；（我们选择的是这种方式，降低复杂度。）
上图中，etcd和Master部署在一起，三个Master节点分别部署了三个etcd，这三个etcd组成一个集群；（当然，如果条件允许，建议将etcd集群和Master节点分开部署。）
每个Master中的apiserver、controller-manager、scheduler都使用hostNetwork, controller-manager和scheduler通过localhost连接到本节点的apiserver，而不会和其他两个Master节点的apiserver连接；
外部的rest-client、kubectl、kubelet、kube-proxy等都通过TLS证书，在LB节点做TLS Termination，LB出来就是http请求发到经过LB策略（RR）到对应的apiserver instance；
apiserver到kubelet server和kube-proxy server的访问也类似，Https到LB这里做TLS Termination，然后http请求出来到对应node的kubelet/kube-proxy server；
apiserver的HA通过经典的haproxy + keepalived来保证，集群对外暴露VIP；
controller-manager和scheduler的HA通过自身提供的leader选举功能（--leader-elect=true），使得3个controller-manager和scheduler都分别只有一个是leader，leader处于正常工作状态，当leader失败，会重新选举新leader来顶替继续工作；
因此，该HA方案中，通过haproxy+keepalived来做apiserver的LB和HA，controller-manager和scheduler通过自身的leader选举来达到HA，etcd通过raft协议保证etcd cluster数据的一致性，达到HA；
keepalived的配置可参考如下：
vrrp_script check_script { script "/etc/keepalived/check_haproxy.py http://caicloud:caicloud@127.0.0.1/haproxy?stats" interval 5 # check every 5 seconds weight 5 fall 2 # require 2 fail for KO rise 1 # require 1 successes for OK } vrrp_instance VI_01 { state MASTER (BACKUP) interface eth2 track_interface { eth2 } vrrp_garp_master_repeat 5 vrrp_garp_master_refresh 10 virtual_router_id 51 priority 100 (97) advert_int 1 authentication { auth_type PASS auth_pass username } virtual_ipaddress { 192.168.205.254 dev eth2 label eth2:vip } track_script { check_script } notify "etc/keepalived/notify_state.sh" }haproxy的配置可参考如下：
global log 127.0.0.1 local0 maxconn 32768 pidfile /run/haproxy.pid # turn on stats unix socket stats socket /run/haproxy.stats tune.ssl.default-dh-param 2048 default log global mode http option httplog option dontlognull retries 3 timeout connect 5000ms timeout client 50000ms timeout server 50000ms timeout check 50000ms timeout queue 50000ms frontend frontend-apisver-http bind *:8080 option forwardfor acl local_net src 192.168.205.0/24 http-request allow if local_net http-request deny default_backend backend-apiserver-http frontedn frontend-apiserver-https # haproxy enable ssl bind *:443 ssl crt /etc/kubernetes/master-lb.pem option forwardfor default_backend backend-apiserver-http backend backend-apiserver-http balance roundrobin option forward-for server master-1 192.168.205.11:8080 check server master-2 192.168.205.12:8080 check server master-3 192.168.205.13:8080 check listen admin_stats bind 0.0.0.0:80 log global mode http maxconn 10 stats enable #Hide HAPRoxy version, a necessity for any public-facing site stats hide-version stats refresh 30s stats show-node stats realm Haproxy\ Statistics stats auth caicloud:caicloud stats uri /haproxy?stats
LB所在的节点，注意确保ip_vs model已加载、ip_forward和ip_nonlocal_bind已开启；
# make sure ip_vs kernel model is loaded modprobe ip_vs modprobe ip_vs_rr modprobe ip_vs_wrr # enable ip_forward and ip_nonlocal_bind echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf echo "net.ipv4.ip_nonlocal_bind = 1" >> /etc/sysctl.conf
如果你通过pod来部署K8S的组件，可参考官方给出的Yaml：
apiserver
apiVersion: v1 kind: Pod metadata: name: kube-apiserver spec: hostNetwork: true containers: - name: kube-apiserver image: gcr.io/google_containers/kube-apiserver:9680e782e08a1a1c94c656190011bd02 command: - /bin/sh - -c - /usr/local/bin/kube-apiserver --address=127.0.0.1 --etcd-servers=http://127.0.0.1:4001 --cloud-provider=gce --admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota --service-cluster-ip-range=10.0.0.0/16 --client-ca-file=/srv/kubernetes/ca.crt --basic-auth-file=/srv/kubernetes/basic_auth.csv --cluster-name=e2e-test-bburns --tls-cert-file=/srv/kubernetes/server.cert --tls-private-key-file=/srv/kubernetes/server.key --secure-port=443 --token-auth-file=/srv/kubernetes/known_tokens.csv --v=2 --allow-privileged=False 1>>/var/log/kube-apiserver.log 2>&1 ports: - containerPort: 443 hostPort: 443 name: https - containerPort: 7080 hostPort: 7080 name: http - containerPort: 8080 hostPort: 8080 name: local volumeMounts: - mountPath: /srv/kubernetes name: srvkube readOnly: true - mountPath: /var/log/kube-apiserver.log name: logfile - mountPath: /etc/ssl name: etcssl readOnly: true - mountPath: /usr/share/ssl name: usrsharessl readOnly: true - mountPath: /var/ssl name: varssl readOnly: true - mountPath: /usr/ssl name: usrssl readOnly: true - mountPath: /usr/lib/ssl name: usrlibssl readOnly: true - mountPath: /usr/local/openssl name: usrlocalopenssl readOnly: true - mountPath: /etc/openssl name: etcopenssl readOnly: true - mountPath: /etc/pki/tls name: etcpkitls readOnly: true volumes: - hostPath: path: /srv/kubernetes name: srvkube - hostPath: path: /var/log/kube-apiserver.log name: logfile - hostPath: path: /etc/ssl name: etcssl - hostPath: path: /usr/share/ssl name: usrsharessl - hostPath: path: /var/ssl name: varssl - hostPath: path: /usr/ssl name: usrssl - hostPath: path: /usr/lib/ssl name: usrlibssl - hostPath: path: /usr/local/openssl name: usrlocalopenssl - hostPath: path: /etc/openssl name: etcopenssl - hostPath: path: /etc/pki/tls name: etcpkitls
apiVersion: v1 kind: Pod metadata: name: kube-controller-manager spec: containers: - command: - /bin/sh - -c - /usr/local/bin/kube-controller-manager --master=127.0.0.1:8080 --cluster-name=e2e-test-bburns --cluster-cidr=10.245.0.0/16 --allocate-node-cidrs=true --cloud-provider=gce --service-account-private-key-file=/srv/kubernetes/server.key --v=2 --leader-elect=true 1>>/var/log/kube-controller-manager.log 2>&1 image: gcr.io/google_containers/kube-controller-manager:fda24638d51a48baa13c35337fcd4793 livenessProbe: httpGet: path: /healthz port: 10252 initialDelaySeconds: 15 timeoutSeconds: 1 name: kube-controller-manager volumeMounts: - mountPath: /srv/kubernetes name: srvkube readOnly: true - mountPath: /var/log/kube-controller-manager.log name: logfile - mountPath: /etc/ssl name: etcssl readOnly: true - mountPath: /usr/share/ssl name: usrsharessl readOnly: true - mountPath: /var/ssl name: varssl readOnly: true - mountPath: /usr/ssl name: usrssl readOnly: true - mountPath: /usr/lib/ssl name: usrlibssl readOnly: true - mountPath: /usr/local/openssl name: usrlocalopenssl readOnly: true - mountPath: /etc/openssl name: etcopenssl readOnly: true - mountPath: /etc/pki/tls name: etcpkitls readOnly: true hostNetwork: true volumes: - hostPath: path: /srv/kubernetes name: srvkube - hostPath: path: /var/log/kube-controller-manager.log name: logfile - hostPath: path: /etc/ssl name: etcssl - hostPath: path: /usr/share/ssl name: usrsharessl - hostPath: path: /var/ssl name: varssl - hostPath: path: /usr/ssl name: usrssl - hostPath: path: /usr/lib/ssl name: usrlibssl - hostPath: path: /usr/local/openssl name: usrlocalopenssl - hostPath: path: /etc/openssl name: etcopenssl - hostPath: path: /etc/pki/tls name: etcpkitls
apiVersion: v1 kind: Pod metadata: name: kube-scheduler spec: hostNetwork: true containers: - name: kube-scheduler image: gcr.io/google_containers/kube-scheduler:34d0b8f8b31e27937327961528739bc9 command: - /bin/sh - -c - /usr/local/bin/kube-scheduler --master=127.0.0.1:8080 --v=2 --leader-elect=true 1>>/var/log/kube-scheduler.log 2>&1 livenessProbe: httpGet: path: /healthz port: 10251 initialDelaySeconds: 15 timeoutSeconds: 1 volumeMounts: - mountPath: /var/log/kube-scheduler.log name: logfile - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: default-token-s8ejd readOnly: true volumes: - hostPath: path: /var/log/kube-scheduler.log name: logfile
apiVersion: v1 kind: Pod metadata: name: etcd-server spec: hostNetwork: true containers: - image: gcr.io/google_containers/etcd:2.0.9 name: etcd-container command: - /usr/local/bin/etcd - --name - ${NODE_NAME} - --initial-advertise-peer-urls - http://${NODE_IP}:2380 - --listen-peer-urls - http://${NODE_IP}:2380 - --advertise-client-urls - http://${NODE_IP}:4001 - --listen-client-urls - http://127.0.0.1:4001 - --data-dir - /var/etcd/data - --discovery - ${DISCOVERY_TOKEN} ports: - containerPort: 2380 hostPort: 2380 name: serverport - containerPort: 4001 hostPort: 4001 name: clientport volumeMounts: - mountPath: /var/etcd name: varetcd - mountPath: /etc/ssl name: etcssl readOnly: true - mountPath: /usr/share/ssl name: usrsharessl readOnly: true - mountPath: /var/ssl name: varssl readOnly: true - mountPath: /usr/ssl name: usrssl readOnly: true - mountPath: /usr/lib/ssl name: usrlibssl readOnly: true - mountPath: /usr/local/openssl name: usrlocalopenssl readOnly: true - mountPath: /etc/openssl name: etcopenssl readOnly: true - mountPath: /etc/pki/tls name: etcpkitls readOnly: true volumes: - hostPath: path: /var/etcd/data name: varetcd - hostPath: path: /etc/ssl name: etcssl - hostPath: path: /usr/share/ssl name: usrsharessl - hostPath: path: /var/ssl name: varssl - hostPath: path: /usr/ssl name: usrssl - hostPath: path: /usr/lib/ssl name: usrlibssl - hostPath: path: /usr/local/openssl name: usrlocalopenssl - hostPath: path: /etc/openssl name: etcopenssl - hostPath: path: /etc/pki/tls name: etcpkitls
“Kubernetes Cluster HA如何配置”的内容就介绍到这里了，感谢大家的阅读。如果想了解更多行业相关的知识可以关注创新互联网站，小编将为大家输出更多高质量的实用文章！
文章题目：KubernetesClusterHA如何配置
网站路径：http://scyanting.com/article/giieop.html
