In chapter 02 we set up a cluster with a single master. It works for learning, but it is a single point of failure: if that node goes down, kubectl stops, the scheduler stops, and new pods are not created.

Control plane high availability is not "having three machines". It is having three etcds with quorum, a stable endpoint, and a join process that can withstand the failure of one member.

Course seal: tested on Kubernetes 1.35 and Ubuntu 24.04. Fictional IPs (192.168.56.x).

Topology

Name	IP	Role
k8s-master-01	192.168.56.10	control plane + etcd
k8s-master-02	192.168.56.11	control plane + etcd
k8s-master-03	192.168.56.12	control plane + etcd
VIP	192.168.56.100	API server endpoint
workers	192.168.56.21–23	workloads

The endpoint 192.168.56.100:6443 needs to point to the three masters. Who can do this can be:

• kube-vip (VIP with ARP/BGP, runs as a static pod).
• HAProxy/nginx stream + keepalived (VIP on layer 4).
• A cloud load balancer.

In this course we use the idea of kube-vip/HAProxy: a virtual IP that survives the failure of a master.

Why the number of masters must be odd

etcd uses Raft consensus. The majority needs to be available:

Masters	Quorum	Tolerates failures
1	1	0
2	2	0 (worse than 1)
3	2	1
5	3	2

With 2 masters, you tolerate no failures. That's why the serious minimum is 3. Never set up HA with 2.

1. Prepare the stable endpoint

You have two practical options.

Option A — kube-vip

kube-vip runs as a static pod and announces the VIP on the network. The manifest is at /etc/kubernetes/manifests/kube-vip.yaml, with vip, interface, and the kube-vip image. It is created before kubeadm init and survives the control plane.

Option B — HAProxy + keepalived

Example of a TCP frontend:

frontend k8s-api
    bind *:6443
    mode tcp
    default_backend k8s-api

backend k8s-api
    mode tcp
    balance roundrobin
    option tcp-check
    server master1 192.168.56.10:6443 check
    server master2 192.168.56.11:6443 check
    server master3 192.168.56.12:6443 check

keepalived maintains the VIP 192.168.56.100 on one of the proxies. If the active proxy fails, the VIP migrates.

Attention: the endpoint must exist before kubeadm init. Changing the endpoint later is surgery. Plan from the beginning.

2. Initialize the first master

On master-01:

sudo kubeadm init \
  --control-plane-endpoint=192.168.56.100:6443 \
  --pod-network-cidr=10.244.0.0/16 \
  --upload-certs

--upload-certs publishes the certificates needed for the other masters in the cluster (encrypted with a temporary certificate-key).

Save the output: it contains the join command for workers and control planes.

3. Join the other masters

On master-02 and master-03:

sudo kubeadm join 192.168.56.100:6443 \
  --token <token> \
  --discovery-token-ca-cert-hash sha256:<hash> \
  --control-plane \
  --certificate-key <certificate-key>

If the certificate-key expires, generate another one on master-01:

sudo kubeadm init phase upload-certs --upload-certs

After the join, validate:

kubectl get nodes
kubectl -n kube-system get pods -o wide | grep -E 'apiserver|etcd|scheduler|controller'

You should see three apiservers, three etcds, three schedulers, and three controllers.

4. Join the workers

sudo kubeadm join 192.168.56.100:6443 \
  --token <token> \
  --discovery-token-ca-cert-hash sha256:<hash>

5. Verify etcd

etcd is the heart of HA. List the members:

kubectl -n kube-system exec etcd-k8s-master-01 -- \
  etcdctl --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  member list

Check:

• 3 members, all started.
• A leader elected.
• etcdctl endpoint health healthy on all three.

If etcd loses quorum (2 out of 3), the control plane stops accepting writes. Recovering quorum is an emergency procedure: restore snapshot or remove the dead member with member remove.

6. Failure test

Do the test before you need it in production:

# em um master, pare o kubelet (simula falha do nó)
sudo systemctl stop kubelet

# em outro terminal
kubectl get nodes
kubectl get pods -A | head

kubectl should continue working via VIP, and etcd should maintain quorum with 2/3. Then:

sudo systemctl start kubelet

If you shut down the entire master, the behavior is the same — as long as the VIP migrates and etcd maintains 2/3.

7. Common errors

Symptom	Probable cause
connection refused on VIP	kube-vip/HAProxy did not start or health check failed
master join fails	certificate-key expired
etcdserver: no leader	quorum lost (2 masters down)
apiserver restarts in a loop	wrong certificate or endpoint
workers do not join	expired token or wrong hash
HA "works" but fails with 1 master	you set up with 2 masters

Checklist

• [ ] Stable VIP/endpoint defined before init.
• [ ] 3 masters with stacked etcd.
• [ ] --control-plane-endpoint the same on all.
• [ ] kubectl get nodes shows 3 control planes.
• [ ] etcdctl member list with 3 members and a leader.
• [ ] Failure test of 1 master documented.
• [ ] etcd snapshot configured.

Glossary

• quorum: majority needed for etcd to decide.
• VIP: virtual IP that migrates between nodes.
• stacked etcd: etcd running on the same node as the control plane.
• certificate-key: key to distribute certificates to masters.
• fencing: ensure that the old node does not continue writing.

Next

With the control plane in HA, we will talk about pod networking: flannel, Cilium, and why NetworkPolicy can be a no-op if you choose the wrong CNI.

Next: Kubernetes Course 06 — CNI: flannel, Cilium, and NetworkPolicy