Stop calling a service that isn't answering

Picture the ordinary version of a bad afternoon. A downstream service, a payments API, a recommendations backend, some internal thing you call on every request, gets slow. Not down, just slow: responses that used to take 40 milliseconds now take twelve seconds before they time out. Nothing in your code is wrong. And yet within a few minutes your own service, which has nothing to do with the outage, is also on fire.

The reason is worth sitting with, because it’s the whole point. Every request that needs that slow dependency now holds one of your threads, or one of your connections, hostage for twelve seconds instead of forty milliseconds. Those are finite. While they’re all stuck waiting on a service that’s going to fail anyway, requests that don’t touch the sick dependency can’t get a worker either. One service’s problem has become your problem, and your problem is about to become the problem of everyone who calls you. That’s a cascading failure, and it’s how a small, contained outage turns into a wall of red across a whole system.

A circuit breaker is the piece that refuses to participate in this. The name is borrowed exactly from the thing in your wall: a component whose only job is to stop the flow when continuing would do damage. You wrap your calls to a dependency in it, and it watches how those calls go.

It lives in three states. Closed is normal: calls pass straight through, and the breaker just keeps a tally of how they’re going. When failures cross a threshold you set, too many errors, too many timeouts, in some window, it flips to open. Open is the important state: for a cooldown period, the breaker doesn’t even attempt the call. It fails instantly, right there, without tying up a thread waiting on a service you already have strong evidence is down. Then, after the cooldown, it moves to half-open: it lets a single trial request (or a trickle) through to see if the dependency has recovered. If that probe succeeds, the breaker closes and normal traffic resumes. If it fails, back to open, and the cooldown timer starts again.

Notice what the breaker is actually protecting. It is not protecting the dependency, the dependency is already having its own bad day, and your restraint mostly just spares it a pile of futile retries. The breaker protects you: your thread pool, your latency, your ability to keep serving the requests that have nothing to do with the sick dependency. It draws a blast radius around the failure so it can’t spread up the call chain. (Keeping your traffic from overwhelming them is a different tool, that’s rate limiting, sitting on the other side of the wire.)

The mistake teams make is thinking the breaker is the whole answer. It isn’t. “Fail fast” is only good news if you’ve decided what failing looks like from the user’s seat. An open breaker gives you a clean, instant “this isn’t available right now,” and now you get to choose what to put in that space. A cached value that’s a little stale. A degraded response that hides the section entirely. A write parked on a queue to replay when things recover. Or, honestly, a clear error, which is still vastly better than a spinner that hangs for twelve seconds and then errors. The breaker’s job is to hand you that decision quickly and cheaply. Designing the fallback behind it is still your job, and it’s the part users actually feel.

Two tuning notes that matter more than they look. The failure threshold and the cooldown are the whole personality of the breaker: too twitchy and it trips on a normal blip and needlessly cuts you off from a healthy service; too sluggish and it lets the cascade get going before it acts. And the half-open probe should be one request, deliberately, the failure mode to avoid is a breaker that, the instant its cooldown ends, slams the recovering service with the full backlog and knocks it straight back over. Recovery is a moment to test gently, not a gate to fling open.

The deep idea is almost philosophical: in a distributed system, the willingness to keep trying is not a virtue. Past a certain point, retrying a service you have good evidence is down is just a way of donating your own health to someone else’s outage. The circuit breaker is where a system finally learns the thing that’s hard for eager code and eager people alike, that the right move, sometimes, is to stop knocking on a door and give it a minute to open on its own.

The science, to look up: the circuit breaker pattern from Michael Nygard’s “Release It!”, Martin Fowler’s writeup of the three-state machine, and implementations like Netflix Hystrix and resilience4j.