$ emrebener
home blogs infrastructure & operations kubernetes etcd topologies in kubernetes: stacked vs. external

etcd topologies in Kubernetes: stacked vs. external

author: emre bener read time: 5 min about: etcd, kubernetes, high availability
published: updated: mentions: raft, kubeadm, consensus algorithm, load balancing, transport layer security

1. etcd and the control plane

A Kubernetes cluster keeps all of its state in etcd. Every object you’ve ever created (Pods, Deployments, Secrets, all of it) lives there as key-value data, and it’s the one thing in the cluster you genuinely can’t afford to lose. Lose your control plane and you can rebuild it; lose etcd without a backup and the cluster is gone.

etcd is a distributed key-value store built on Raft, a consensus protocol that commits a write only once a majority of members agree on it. That majority requirement is why etcd runs with an odd number of members: a three-member cluster keeps its majority through one failure, a five-member cluster through two. The same arithmetic drives the control plane counts below.

The control plane is the set of components that make cluster-wide decisions: the API server (kube-apiserver), the scheduler (kube-scheduler), and the controller manager (kube-controller-manager). One detail matters for everything below: only the API server talks to etcd. Every other component reads and writes state through the API server. So “where does etcd live” really means “how do the API servers reach their data store.”

That’s the whole decision. etcd can run on the same machines as your control plane components, which is the stacked topology, or on a separate set of machines that Kubernetes doesn’t manage, which is the external topology. kubeadm, the standard tool for bootstrapping a cluster, supports both. The choice comes down to how many machines you’re willing to run and how much you want etcd’s failures kept apart from the control plane’s. For most clusters, default to stacked; reach for external only when you specifically need to give etcd its own failure domain or scale it independently.

2. Stacked etcd

In a stacked topology, every control plane node runs its own local etcd member, and that member talks only to the API server on the same node. The members still form one etcd cluster across the nodes; each one just happens to sit next to a control plane instance. This is kubeadm’s default. Run kubeadm init, then kubeadm join --control-plane for the rest, and kubeadm creates a local etcd member on each control plane node automatically, with no extra configuration.

The appeal is operational simplicity. There are fewer machines to provision and you manage one set of nodes instead of two. etcd replication rides along on the control plane nodes you were going to run anyway.

The cost is coupling. A stacked node carries both an etcd member and a control plane instance, so a node failure takes out both at once. The kubeadm docs put it plainly: “If one node goes down, both an etcd member and a control plane instance are lost, and redundancy is compromised.” Drop from three nodes to two and etcd drops from three members to two. That still has a majority, but it no longer tolerates a second failure. The fix is to add more control plane nodes, which widens the etcd cluster in the same move.

For high availability, run a minimum of three stacked control plane nodes, enough for etcd to keep its majority through a single node loss.

Stacked topologycontrol plane nodecontrol plane nodecontrol plane nodeetcdetcdetcdkube-apiserverkube-apiserverkube-apiserverload balancerworkerworkerworkerStacked topologycontrol plane nodecontrol plane nodecontrol plane nodeetcdetcdetcdkube-apiserverkube-apiserverkube-apiserverload balancerworkerworkerworker

3. External etcd

In an external topology, etcd runs on its own hosts, separate from the control plane nodes, and Kubernetes doesn’t manage it. Each etcd host serves every API server rather than a single co-located one, and standing up, securing, and maintaining that etcd cluster is your job, not kubeadm’s.

What you buy is decoupling. The control plane and the data store become separate failure domains. Losing a control plane node takes out only a control plane instance; the etcd cluster is untouched. Lose an etcd host and only an etcd member goes with it, while the API servers keep serving against the rest. In the kubeadm docs’ words, losing either one “has less impact and does not affect the cluster redundancy as much as the stacked HA topology.”

The price is machines and operational surface. External etcd needs roughly twice the hosts: a minimum of three control plane nodes plus three etcd hosts, so six machines for a highly available setup where stacked needed three. And since Kubernetes isn’t managing etcd, its upgrades, backups, certificate rotation and monitoring all fall to you.

You point kubeadm at the existing etcd cluster through the cluster configuration:

apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
etcd:
  external:
    endpoints:
      - https://10.0.0.1:2379
      - https://10.0.0.2:2379
      - https://10.0.0.3:2379
    caFile: /etc/kubernetes/pki/etcd/ca.crt
    certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt
    keyFile: /etc/kubernetes/pki/apiserver-etcd-client.key

With etcd.external set, kubeadm skips creating any local member and wires every API server to those endpoints over mutual TLS, using the client certificate and key to authenticate against etcd.

External topologyetcd hostetcd hostetcd hostetcdetcdetcdcontrol plane nodecontrol plane nodecontrol plane nodekube-apiserverkube-apiserverkube-apiserverload balancerworkerworkerworkerExternal topologyetcd hostetcd hostetcd hostetcdetcdetcdcontrol plane nodecontrol plane nodecontrol plane nodekube-apiserverkube-apiserverkube-apiserverload balancerworkerworkerworker

4. When to choose external over stacked

Default to stacked, and move to external only when you have a specific reason to separate etcd from the control plane. Stacked is kubeadm’s default for a reason: for most clusters it delivers real high availability on half the machines with far less to run.

Reach for external etcd when one of these holds:

  • You want etcd failures isolated from control plane failures, so a bad control plane node can never drag a data-store member down with it.
  • You need to scale or tune etcd on its own: faster disks, a dedicated network, or heavier hardware for a write-heavy cluster.
  • You already operate etcd as shared infrastructure, or you have the operational maturity (backups, upgrades, monitoring) to run it well.
  • Blast-radius or compliance requirements call for keeping the data store on separate, separately-secured hosts.

If none of those are pulling on you, stacked is the simpler and cheaper answer.

StackedExternal
etcd locationon each control plane nodeon separate hosts
Managed by Kubernetesyes (kubeadm auto-creates it)no (you run it)
Minimum hosts for HA36 (3 + 3)
Failure couplingnode loss takes etcd and control plane togetherfailure domains stay separate
Setup and operationssimplermore hosts, more to maintain
kubeadm defaultyesno (needs etcd.external)

The choice isn’t permanent, but switching later isn’t free. Moving from stacked to external means standing up a new etcd cluster and migrating the data into it, so it’s worth settling when you first build the cluster rather than after it’s carrying load.