Where Everything Breaks

A radiologist’s screen shows a shadow on a lung. The report field offers a dropdown. There is no option for probably nothing, but watch it. There is only a list of conditions. She picks the closest one. By the time the chart reaches the next clinician, “suggestive of early pneumonia” has become “pneumonia.” Treatment begins. The shadow was an artefact of how the patient was positioned. The test was never wrong. The record was — not because anyone lied, but because the system had no way to hold a suspicion as a suspicion. A guess hardened into a fact the moment it was written down, and nothing downstream could tell the difference.

This is not a story about medicine. It is the same story everywhere.

Truth fails in two ways

Lost by accident

The diagnosis. A probabilistic inference — the image suggests X — is recorded as a committed fact — the patient has X. The suspicion and the fact were never distinguishable in the record, so the difference could not survive contact with the next reader.

The overwritten balance. A value lives in a database row. Two processes update it at nearly the same instant; one overwrites the other. Months later the figure is disputed and someone goes looking for the history. There is no history. The row only ever held now. What was true last Tuesday is not stale — it is gone, because the system was built to store state, not to remember what happened.

The decision no one made. An automated system blocks a customer, denies a claim, takes an action that causes harm. The post-mortem convenes. Who decided this? On what basis? The logs from three services disagree. The model version that scored it has been redeployed twice since. Eventually someone writes “the system decided.” No one decided. The moment a derivation became a binding action left no trace, so there is nothing to hold accountable.

The retraction. A celebrated finding is published. Hundreds of studies build on it. Products ship, policies are written, careers are staked. Then it fails to replicate, and is quietly retracted. But the retraction does not propagate. The downstream work still cites it; the decisions it justified are never revisited. The claim was demoted from established to refuted — and the demotion reached almost none of the things that depended on it. (Failure mode: non-propagating refutation.)

Refused on purpose

The innovation divide. A team faces a genuinely new method. The same evidence sits on the table for everyone. The innovators treat the unproven as nearly certain and push to build. The pragmatists withhold judgment until peers have validated it. The laggards dismiss real evidence as fashion. They are not reading different data. They are reading the same data through different appetites — building versus protecting, gain versus loss. The claim has one actual strength. Five people assign it five different strengths, and call the result an argument about facts.

The role labels echo adopter categories from Everett Rogers — but this vignette is not Rogers’s diffusion curve over a population. It is one room, one claim, strength confused with appetite. Nor is it Geoffrey Moore’s chasm between early and mainstream markets (see Innovation Divide for how this vignette differs from Moore and Rogers).

The promise. Someone makes confident commitments — to a partner, a team, an investor — that they have no record of keeping and no standing to make. The first few are believed. Trust does not collapse from a single broken promise; it erodes as commitments pile up unbacked by any track record. The mirror image is just as hollow: the seasoned expert who knows but will never commit to a call, never put their name on it. A commitment from someone with no relevant standing is worthless. So is deep expertise that refuses to commit. Trust lives only where the two meet.

The buried finding. A junior analyst brings the manager something true and unwelcome: the launch metric is inflated; the safety margin is gone. The manager, exposed by it, buries it. No record, no escalation. Truth does not change because it was buried — the reckoning simply arrives later, and worse. Here rank overruled evidence. Positional authority erased epistemic authority, and the only thing that could have stopped it was a record the manager was not able to delete. Homophily and heterophobia often sit underneath such failures — see homophilic dismissal.

Notice that the last two failures of the accidental kind and the last of the deliberate kind are the same organisational wound from different sides: the decision no one made is negligence — no one is accountable because nothing was recorded; the buried finding is its darker twin — someone with power made sure nothing was recorded.

The pre-AI rehearsal, now at industrial scale

One more, because it bridges to why this work exists at all.

The confident deck. A consultant delivers a beautiful report. Three interviews and an educated guess are rendered into crisp exhibits, confident headlines, a recommendation in bold. The polish is the argument. The client commits a large sum on the strength of how finished it looks. The underlying claims were weak and uncertain; the presentation said otherwise, and the presentation is what got acted on.

This is an old, human failure — it predates every computer in the building. But hold it next to what we have just built. We now have machines that produce infinite, fluent, confident-sounding prose on demand, at near-zero cost, with no native signal of how much is known and how much is invented. The confident deck used to require a consultant and a week. Now it is a paragraph and a second. The phenomenon has not changed. Its scale has changed by orders of magnitude. That is the problem this series is for.

The shared shape

Why this site exists

Genesis is the long theory. The rest of the site — terms, principles, patterns, frameworks, maps — is the apparatus: named failure modes from literature and experience, links between them, and the path toward a thinking exoskeleton governed by values with H0: Humane Core at the centre. For the full personal and strategic arc, and how to navigate the menus, see Why This Site on About.

Everything in this series follows from taking that wound seriously. We will need a record that cannot quietly lose the past, and a discipline that will not quietly bend or bury it. The first thing such a record requires is an idea so plain it sounds trivial until you watch how often systems are built to deny it.