On Information and Sufficiency (1951)
Kullback & Leibler — divergence D(P||Q) between distributions
Work
1951 Annals of Mathematical Statistics paper defining Kullback–Leibler divergence — measure of distributional mismatch.
Primary publication. Kullback, S., & Leibler, R. A. (1951). On Information and Sufficiency. Annals of Mathematical Statistics, 22(1), 79–86.
Claims we use in Genesis
Observation quality. Quality(sensor) ≈ 1 − D_KL(P(observation) ‖ P(ground_truth)) — perfect calibration yields zero divergence; uncharacterised sensors have unknown divergence. The requirement to publish observation quality is, formally, to estimate and disclose KL divergence (with Bayesian hierarchy for meta-uncertainty). See What Information Theory Says.
Author
Solomon Kullback (Richard A. Leibler, co-author)