§ 80
Constant Work
A system whose work grows with the number of changes or failures does the most work exactly when it has the least headroom.
Keep in mind
In control loops, do the same work whether or not anything changed
What to do
For control loops that propagate configuration or health, send the whole state on a fixed schedule instead of deltas on change, and size the payload for the maximum, not the typical case. Keep the amount of work independent of how many things changed. Use it where the work is small; for request-serving capacity, scale with load and design the overload case explicitly.
Why it happens
Most systems do more work as conditions get worse: more retries, more changes to propagate, more health transitions to process. That extra work arrives when the system has the least headroom, and the code that handles it is the code that runs least often. A constant-work design does the same amount regardless: Route 53's health-check aggregators send DNS servers a fixed-size table of every status every few seconds, and Network Load Balancer nodes reload their full configuration from S3 every few seconds even when nothing has changed. A surge of failures produces no extra work, so there is no overload mode to discover in production.
When it stops applying
Elasticity: doing the full work every cycle wastes resources when little has changed; for configuration and health status the cost is negligible, but for a web fleet the savings from scaling with load usually win over running peak capacity all day.
Source
Colm MacCárthaigh, Reliability, constant work, and a good cup of coffee (Amazon Builders' Library, 2021) — the coffee-urn analogy, the Route 53 health-check and Network Load Balancer configuration examples, and the admission that physical constant work "usually come[s] at the cost of waste."
Read the sourceCite this
Hammurabi, § 80, “Constant Work.” https://kanywst.github.io/hammurabi/laws/constant-work/
BibTeX
@misc{hammurabi-constant-work,
title = {Constant Work},
author = {{Hammurabi contributors}},
note = {Hammurabi, § 80},
url = {https://kanywst.github.io/hammurabi/laws/constant-work/}
}Read in a column
- Making it faster Next in the column: Wirth's Law