$ emrebener

08. Concurrency

author: emre bener read time: 8 min about: twelve-factor app, scalability, concurrency (computing)
published: updated: mentions: kubernetes, heroku, systemd, node.js, go, horizontal pod autoscaler

1. The rule: scale out via the process model

The factor is one sentence: scale out via the process model. When an app needs to handle more work, you add concurrency by running more processes, not by growing one process bigger. The process is the unit of scale.

That commits the app to horizontal scaling. As the series introduction put it, a twelve-factor app scales by adding more copies of itself. Concurrency is the factor where that becomes a rule: when load climbs, you run more identical processes and let them share it, rather than moving the app onto a bigger machine. The rest of this post is the vocabulary for doing that, why running more processes is a reliable operation rather than a fragile one, and why the model turned out to describe exactly what a container orchestrator does.

2. Process types and the process formation

Two words carry this factor: process type and process formation. A process type is a kind of work the app does. The common split is a web process type that handles HTTP requests and a worker process type that handles long-running background jobs: sending email, encoding video, draining a queue. Same codebase, same deploy, different entry point and different job. The process formation is the array of process types and the count of each: three web processes and two worker processes, say. Scaling the app means changing those numbers.

It is the one-app-many-process-types idea in concrete form: one codebase and one deploy, several kinds of process pulled from it, each type doing one job. The formation only works because the processes underneath are share-nothing. A twelve-factor process holds no state another process depends on, which is the whole point of Processes: every web process is interchangeable with every other web process, so a request can land on any of them. Statelessness is the precondition; the formation is what you build on top of it. Take it away and the count stops being a dial you can turn freely, because the processes would no longer be interchangeable.

3. Horizontal scale-out and the vertical ceiling

Running more processes is a reliable way to add capacity because the processes are share-nothing: adding one takes no coordination with the others. That is the property that makes the whole factor practical, and it’s worth being precise about what it rules out and what it doesn’t.

It doesn’t rule out concurrency inside a process. A single process can and should do plenty of its own: several threads inside the runtime VM, or the async, evented model where one thread juggles many in-flight requests by never blocking on I/O. Node.js is the everyday example of the evented style; EventMachine and Twisted were the same idea years earlier. In-process concurrency is genuinely useful and this factor is not an argument against it. Each process should still make good use of the machine it sits on.

What the factor argues is that in-process concurrency alone isn’t enough, for two reasons. A single process can only grow so large: one machine has a fixed ceiling of CPU and memory, and vertical scaling runs into it. And however large you grow that one process, it’s still one process, so a crash, a deploy, or a lost machine takes all of its capacity at once. Growing up gives you a bigger single point of failure; growing out gives you many small interchangeable ones. So the app has to span multiple processes across multiple machines, and because the processes are share-nothing and horizontally partitionable, that spread is cheap: you start another process, it binds its own port, and a load balancer starts sending it traffic, with nothing to re-coordinate. This is where Concurrency meets Port binding: every web process exports its service on a port of its own, and the fleet behind a load balancer is just many of those ports.

Scale up (vertical)Scale out (horizontal)fixed ceiling: one machine's CPU + RAM1 machine1 processgrows bigger(more CPU/RAM)load balancermachine 1processprocessprocessprocessmachine 2Scale up (vertical)Scale out (horizontal)fixed ceiling: one machine's CPU + RAM1 machine1 processgrows bigger(more CPU/RAM)load balancermachine 1processprocessprocessprocessmachine 2

4. Don’t daemonize; let a process manager run the formation

Twelve-factor processes should never daemonize themselves or write their own PID files. A twelve-factor process runs in the foreground, as an ordinary child of whatever started it, and writes its output to stdout. Keeping the process alive, capturing its output, and stopping it cleanly is not the app’s job; it belongs to a process manager outside the app.

Which process manager depends on where the app runs. On a plain server in production, that is systemd, capturing the process’s output and restarting it if it exits. On a cloud platform it is the platform’s own distributed process manager, spreading the formation across a fleet of machines. In development it can be a tool like Foreman, which reads the formation and starts one of each process type locally, so your laptop runs the same shape as production. The division is the same in every case: the app just runs and writes to stdout, and the manager supervises output streams, restarts crashed processes and handles shutdown.

Restarting crashed processes is the part that matters most, and it’s where this factor hands off to Disposability. Because the processes are disposable, the manager’s recovery strategy can be blunt: if a process dies, start another one. There is no state to recover and no cleanup to coordinate, so “just restart it” is a complete answer. The app stays simple by not managing its own lifecycle, and the manager stays simple because the processes it supervises are disposable.

5. The process formation is a Kubernetes Deployment

Line the process model up against a container orchestrator and it maps almost exactly. The process formation becomes a set of Kubernetes Deployments, one per process type: a web Deployment and a worker Deployment, each with a replica count that is the number of that process type to run. Scaling out is changing the replica count, the same dial as before under a different name, and the Horizontal Pod Autoscaler turns that dial for you, adding replicas as load rises and removing them as it falls. What the factor called scaling the process formation is, in a modern cluster, an autoscaler editing a number.

The formation shows up even more literally on Heroku, where this factor was written. A Procfile with web: and worker: lines is a process formation on disk: one line per process type, naming the command that starts it, and you scale each type on its own. “Scale out, not up” as the reflexive cloud-native default, more small processes instead of one bigger server, is this factor restated as a platform’s opinion.

The don’t-daemonize rule maps just as cleanly, and containers are what made it unavoidable. A container is built to run a single foreground process, and that process is your app, writing to stdout, exactly as the factor asks. It doesn’t daemonize, because the orchestrator is now the process manager. Kubernetes watches the container, restarts it when it exits, and reschedules it onto another node when a machine dies, the same supervising job systemd did a section ago, lifted up to the cluster. You don’t run a process manager inside the container to keep your app alive; the container holds one foreground process, and the orchestrator keeps it alive from outside.

Horizontal PodAutoscaleradjusts the webreplica count (N)web Deployment · replicas = Nweb1 foregroundprocessweb1 foregroundprocessweb1 foregroundprocessworker Deployment · replicas = Mworker1 foregroundprocessworker1 foregroundprocessHorizontal PodAutoscaleradjusts the webreplica count (N)web Deployment · replicas = Nweb1 foregroundprocessweb1 foregroundprocessweb1 foregroundprocessworker Deployment · replicas = Mworker1 foregroundprocessworker1 foregroundprocess

6. “Scale out” doesn’t mean “never use threads”

Scaling out and in-process concurrency aren’t a choice between two options; you use both, and they multiply. Within each process you still run threads, async I/O, or goroutines; across the fleet you run many such processes; the total concurrency is the product of the two, the work one process handles at once times the number of processes you run. Horizontal scale-out sits on top of in-process concurrency rather than replacing it.

The async ceiling is much higher now than when this factor was written. A single Node or Go process can hold tens of thousands of open connections without breaking a sweat, which raises a fair question: if one process can absorb that much, why spread out at all? Because of the same single-point-of-failure problem from earlier: one process is one thing to lose, however capable it is. The factor’s bias toward horizontal scale is about elasticity and fault tolerance, not a claim that squeezing concurrency out of each process is wrong. You want both: each process busy, and enough of them that losing one doesn’t matter.

One wrinkle comes with “just run a foreground process” inside a container. The process running as PID 1 is treated specially by the kernel: by default it doesn’t reap zombie child processes, and doesn’t act on signals like SIGTERM the way a normal init would, so an app running as PID 1 that spawns children or expects a clean shutdown signal can misbehave. The common fix is a tiny init such as tini as PID 1, doing the small reaping-and-signals job the OS process manager used to do for free. It’s a detail to get right, not a hole in the model.

Put it together and the durable idea is small. Make the process the unit of scale. Keep it share-nothing, so any copy is as good as any other. Group your processes into types, and let the platform run the formation: more of a type when you need it, fewer when you don’t. In-process concurrency makes each process do more; the process model makes the app do more by running more of them. Small interchangeable processes that a platform can multiply on demand are what turn scaling from a re-architecture into a number you change.