§ 98
Genchi Genbutsu (Go and See)
Decisions made from reports drift away from what is actually happening, because each layer of summary drops the detail that did not fit its form.
Keep in mind
See the real thing yourself before you trust the summary
What to do
Before you decide on a fix, a priority or a postmortem conclusion, look at the primary evidence yourself: reproduce the bug, read the raw logs and the trace of one failing request, watch a real user hit the problem, or sit on the support queue for an hour. Use what you saw to question the dashboard, not to replace it.
Why it happens
A report is written by someone who chose what to include, in categories fixed before the problem appeared, so the anomaly that matters can be averaged away or left out. A dashboard shows only what someone thought to measure, and the failure nobody predicted has no panel. The further the person deciding sits from the work, the more layers of summary lie between them, and each layer is also shaped by what its writer expects the reader to want. Toyota made going to the source one of the five key concepts of the Toyota Way: find the facts where the work happens before deciding.
When it stops applying
Trust the Aggregate: a visit shows one shift, one user or one trace, and people judge from a small sample as if it stood for the whole (the belief in the law of small numbers), so for a question about the whole fleet the dashboard that covers every case is the better witness.
Source
Toyota Motor Corporation, The Toyota Way 2001, as recorded in Toyota's official history 75 Years of Toyota, ch. 4, sec. 7 — "Genchi Genbutsu (go to the source to find the facts)", one of five key concepts compiled in April 2001; Toyota, Sustainable Management Report 2016, p. 23 — "go to the source to find the facts to make correct decisions, build consensus and achieve goals at our best speed." The book's opening message credits the founder with applying it: "Kiichiro thoroughly applied the principle of genchi genbutsu (going and seeing for oneself)" (Fujio Cho, 75 Years of Toyota, Message, 2012). The bias behind the counter-force is from Amos Tversky & Daniel Kahneman, Belief in the Law of Small Numbers (Psychological Bulletin, 1971).
Read the sourceCite this
Hammurabi, § 98, “Genchi Genbutsu (Go and See).” https://kanywst.github.io/hammurabi/laws/genchi-genbutsu/
BibTeX
@misc{hammurabi-genchi-genbutsu,
title = {Genchi Genbutsu (Go and See)},
author = {{Hammurabi contributors}},
note = {Hammurabi, § 98},
url = {https://kanywst.github.io/hammurabi/laws/genchi-genbutsu/}
}Read in a column
- When the numbers run the work Next in the column: Error Budget
Related laws
- 83Five WhysThe first answer to "why did this fail" is usually a symptom, and fixing a symptom leaves the cause in place to fail again.
- 82Andon (Stop the Line)Letting anyone who sees an abnormality stop the line keeps a defect from traveling downstream, where it costs more to find and fix.
- 14Goodhart's LawA number stops measuring well once people are rewarded for moving it.
- 111Most Ideas FailIn online controlled experiments at large web companies, most ideas that survived the team's own pruning do nothing or make things worse once they are measured, and experts tend to be poor at telling in advance which ones.
- 114McNamara FallacyDecisions drift toward whatever can be counted, until what cannot be counted is treated as if it did not exist.