Why the bakery can switch itself off before the bill does
The Workers Paid plan has no spending cap, so a bug can run up a bill that nobody sees for a day. This is how the bakery reads the platform’s own meter every 15 minutes, prices what it finds, and turns itself down in two steps. It has met only a fake so far.
A bill that arrives after the money is gone
Two five-figure Cloudflare bills of 2026 are cited in docs/cost.md. In one, a Durable Object alarm re-armed itself for the current time and ran about 600 times a second for about two weeks, a $10,811.41 invoice. In the other, a queue consumer re-enqueued its own writes, and the bill came to about $36,000. Both rest on their developers’ own posts. The bakery holds both shapes, because its aggregates run on alarms and a Queue joins them.
What a simpler design does, and where it breaks
The simple design sets budget alerts. The bakery has them, at $10, $25, $50 and $100, and treats a ring as a guardrail failing, but an alert gives notice within a day. At the rate of the smallest loop docs/cost.md prices, $4.65 an hour, a day of it is $111.60, over the $100 ceiling. The second is a cap on what the code admits, which the bakery also has: Control refuses sandboxes past 300 a day. That cap counts what passes through the door. A looping alarm never uses the door, so it spends without being counted. The lab’s index is where the lab covers the family of techniques, and this principle has no lab page yet.
The breaker is the third layer, and it measures what neither of the others can: what the platform itself says was used. Nygard set the pattern down in Release It!, and Martin Fowler’s page describes its shape: it trips when failures pass a threshold, answers at once while open, and is reset by people. Here the count is dollars, and what trips is the whole demo.
Five meters, two lines, one decision
Every 15 minutes the API Worker sends four GraphQL requests to Cloudflare’s Analytics API, one per dataset, with a token of its own. They return five meters: Worker requests, Durable Object requests, rows written, Durable Object duration in gigabyte-seconds, and Queue operations. The window is the last complete 15-minute slot, and each count is priced in micro-dollars at the Paid plan’s overage rates, as if every unit were past the included amount. That is deliberately cautious.
Two lines come from numbers the bakery already has. The dollar line is the $100 ceiling divided by the 2,880 windows in a month, $0.0347, the rate that would spend the ceiling in a month. Each meter’s line is four times its 15-minute share of the day’s budget, which is Control’s cap of 300 sandboxes times the per-sandbox meter in docs/cost.md: 1,875 Worker requests and 62,500 rows written, for example. The Worker does not judge. It sends the reads to Control, which calls a pure function in crates/app/src/breaker.rs and, on a rise, raises the admission flag in the same transaction, so the verdict and the flag cannot disagree.
The steps are two. A window over a meter’s line, or priced over the dollar line, sets read-only. A window priced over four times the dollar line, $0.1389 and about a week of the ceiling, or still over a line after a trip, sets off, and the Worker then enables the pre-built WAF rule that blocks the API hostname, through the Rulesets API. The state only goes up. A person resets it, in Data Studio and the dashboard, once the cause is fixed. An automatic reset after quiet windows would let a loop start again every few hours.
The nuances of the tech
- The analytics lag, so the slot before. Cloudflare’s Workers page, read on 2026-11-06, warns of a slight delay in aggregation and gives no number. The current slot would look empty, so the breaker reads the one that has ended. The price is time: a loop runs for up to 30 minutes before the first run that can see it. A zero for Worker or Durable Object requests is logged as empty, because the cron’s own run leaves at least one of each, so a zero there means late data, not a quiet account.
- Two field names nobody at Cloudflare documents. The page for the periodic Durable Object dataset documents only
cpuTimeand points to introspection. The names for rows written and duration,rowsWrittenandduration, come from a third-party dump of the schema. If they are wrong, Cloudflare answers with a validation error, and the two meters are marked unread and shown on the status page. The price is then a floor, and the other three meters still count. - One request per dataset. A GraphQL validation error fails the whole request. Asking for all five meters at once would let one wrong name hide the other four, so each dataset has its own request and its own five-second timeout.
- A rule update wants every field. The update page, read on 2026-11-06, says to include all the rule fields wanted in the new definition, so the breaker reads the rule first and sends its action, expression and description back with
enabled: true. Whetherenabledalone would do is not tested. - A zero ceiling goes straight to off. Any priced usage is past a line of zero, so the launch checklist’s forced trip, with
BREAKER_CEILING_USDset to “0”, skips read-only and enables the real kill rule. Run it only when someone is ready to reset it. - The whole account, every count billable. The ceiling belongs to the account, so the breaker prices the site’s lab Worker as well, and counts every unit as past the free allowance. It trips earlier than the invoice would, which is the safe side: a false trip is reset by hand, and a late one is not undone.
What it costs
Each run is 4 requests to Cloudflare, 384 a day, and one request to Control and about one row, which docs/cost.md counts in its idle month: 2,910 Worker requests with the wipes, 0.03% of those included. The API Worker’s wasm was 4,400,280 bytes on 2026-11-06, against a budget of 5,317,776 (docs/roadmap.md, PR 17). The larger cost is the reach of a false trip. Read-only turns away new work at the door, and off blocks the API hostname, so the public pages stay but the signed-in screens go offline. There is no email yet, so the signals are the status page’s Breaker line and a morning issue.
What the breaker cannot do is stop an alarm: neither the flag nor the rule reaches a loop inside an object, so the loop guard bounds each object to 100 re-arms a day. And it has met no real usage yet. The field names, the token’s permission names and the update body are listed in docs/cost.md as open until the first real run.
Try it
Read the status page’s Breaker line, then the same state in GET /v1/status. To watch every path, run scripts/smoke-breaker.sh: it starts a fake of the two Cloudflare APIs and walks 114 checks through unconfigured, armed, read-only, off with the rule enabled, held while tripped, a rule enable that fails and is retried, an unread meter, a request that times out and a rejected token. Its last phase sets the ceiling to “0”. In the log, breaker_read lines give each window’s verdict and price, and breaker_meter_unread says which meter and why.
Sources
- Release It!, Michael T. Nygard, 2nd edition, 2018: the circuit breaker and the shedding of load, as the bakery’s
docs/architecture.mdcites them. The book was not read on 2026-11-06. - A paper on denial of wallet, Kelly, Glavin and Barrett, 2021: the attack, or the bug, that forces financial exhaustion on a serverless account, as
docs/cost.mddefines it. The paper was not read on 2026-11-06. - Metrics and analytics, Cloudflare’s Workers documentation, read on 2026-11-06: a slight delay in aggregation and metrics delivery for the most recent minutes, which is why the breaker reads the slot before the current one.
- Queues metrics, Cloudflare’s Queues documentation, read on 2026-11-06: the queueMessageOperationsAdaptiveGroups dataset and its billableOperations field, the sum of billable writes, reads and deletes.
- GraphQL analytics for Durable Objects, Cloudflare’s Durable Objects documentation, read on 2026-11-06: the request dataset, and for the periodic dataset only cpuTime documented, with introspection as the way to list the rest.
- Update a rule, Cloudflare’s Ruleset Engine documentation, read on 2026-11-06: a rule changed by a PATCH that should carry all the rule fields wanted in the new definition.
- API token permissions, Cloudflare’s API documentation, read on 2026-11-06: the names Account Analytics Read and Zone WAF Edit, with Zone WAF Write in a second table, so that the names are confirmed on the first real run.
- Cloudflare GraphQL schema docs, a third-party introspection dump, not Cloudflare’s own page, read on 2026-11-06: the field names rowsWritten and duration for the periodic dataset, which the breaker treats as unread until a real run confirms them.
- CircuitBreaker, Martin Fowler’s bliki, read on 2026-11-06: a protected call wrapped in an object that trips when failures pass a threshold, answers at once while open, and is watched and reset by people.
- The bakery’s cost.md, this repository, read on 2026-11-06: the price sheet, the lines, scenario D, the idle month and the items still open on the first real run.
Back to the breaker principle on How it’s built, to the decision that chose it, ADR 0010, or to the lab.