ML reproducibility crisisreproducibility crisis in machine learningML-based science reproducibility

EpistemologyFormal foundations

ML Reproducibility Crisis

Large-scale failure to replicate ML research claims — cultural symptoms, architectural broken causal chain

MARKO ŽIVKOVIĆ In review

The ML reproducibility crisis is the sustained pattern in which machine learning research claims — especially in ML-based science — cannot be replicated by others or sometimes by the original authors: metrics differ, rankings reverse, or conclusions collapse when code, data, and environment are reconstructed honestly.

Concept maps: ML reproducibility crisis (concrete layering) · Provenance discipline stack (abstract, domain-transferable pattern).

It is usually discussed as culture — publication pressure, poor documentation (provenance deferred), withheld code — driven by misaligned incentives, short-termism, and local optimization. Those forces are real and rational under current reward geometry (publication over provenance). In this corpus culture and architecture co-produce the crisis: incentives make deferral attractive; mutable tooling makes deferral possible.

Architecture: the causal chain from raw capture through L1 inputs to L2 results is broken — ruptured provenance chain — mutable datasets, unpinned environments, unrecorded configuration, metrics without provenance — so reproduction is heroic discipline rather than a query over committed history. When replication or audit does disprove a claim, non-propagating refutation often compounds the damage. Canonical treatment: Event Sourced Science.

What goes wrong

Failure mode	Corpus read
Ruptured provenance chain	Chain from capture → commit → derive severed; cannot replay runs or trace metrics to inputs
Data leakage	Train/test contamination, temporal leakage, illegitimate features — inflated performance that vanishes on honest split (Kapoor & Narayanan, 2022)
Mutable L0/L1	Data cleaned in place; labels silently updated — inference-as-fact and mutable history
Unpinned L2 environment	`transformers>=4.0` resolves differently six months later — L1 instability
Missing commit records	Deployment, labelling, hyperparameters not L0 before run — provenance chain severed
Non-propagating refutation	Replication or retraction does not reach papers, products, and policies that relied on the claim
Computational irreproducibility	Code, data, or environment not available to rerun (Pineau et al., 2021)

Cultural drivers (co-produce architecture gaps)

Driver	Pattern	Effect
Misaligned incentives	Publication over provenance	Score papers and headlines, not rerunnable chains
Short-termism	Provenance deferred	Docs, pins, and sharing land in perpetual “later”
Local optimization	Metric substitution	Lab/team KPI ≠ field reproducibility
Competitive / shame avoidance	Artifact withholding	Code and data stay private while claims travel

Fix (architecture + incentives)

Treat ML lifecycle as event sourcing: measure → commit (labels, config, deployment) → derive (weights, metrics) with immutable append and explicit refs at each step — committed provenance chain. Experiment architecture and pinned epistemic harness (containers, lockfiles, run IDs) make reproduction structural; refutation propagation ensures disproof reaches dependents. Incentive alignment — scoring replication, artifact commits, and contest paths — makes honest capture cheaper than deferral. Same root cause as immutable infrastructure (Article 26).

Key sources

Kapoor & Narayanan (2022) — systematic survey: leakage across 17 fields, 329 affected papers; taxonomy; civil-war prediction case study; model info sheets proposal. arXiv:2207.07048
Pineau et al. (2021) — NeurIPS reproducibility program; checklist for code, data, seeds, and documentation in ML research. arXiv:2003.12206

Corpus stance

A1 — adopted on this site: The crisis is real and both cultural and architectural; fixes require incentive alignment and structural provenance (facets, commits, gates), not “try harder” alone.