Approximation of the Two-Part MDL Code
Pieter Adriaans, Paul Vitanyi
Abstract
Approximation of the optimal two-part MDL code for given data, through successive monotonically length-decreasing two-part MDL codes, has the following properties: (i) computation of each step may take arbitrarily long; (ii) we may not know when we reach the optimum, or whether we will reach the optimum at all; (iii) the sequence of models generated may not monotonically improve the goodness of fit; but (iv) the model associated with the optimum has (almost) the best goodness of fit. To express the practically interesting goodness of fit of individual models for individual data sets we have to rely on Kolmogorov complexity.
Create a lesson
Related papers
Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO
Yunpeng Ba, Zhi Zheng, Yue Xie et al.
Beyond Parallel Blindness: Information Floors and Model Gaps in Block Drafting
Xinwei Qiang, Xiang Fang, Chang Chen et al.
QuantumBoostNet: A Hybrid Classical-Quantum Architecture for Enhanced Accuracy in Cardiac Ultrasound View Identification
Mihai Udrescu-Milosav, Stefan-Alexandru Jura, Mihai Udrescu et al.
MM-Spectrum: Multimodal Multi-spectral Molecular Structural Elucidation with a Stable MoE Framework
Hai-tao Yu, Nan Min, Zheng Fang et al.
Making Latent Evolution Explicit: Operator-Structured Transitions for World Action Models
Xiaoxiao Lu, Yunlong Dong, Jiahao Shi et al.
Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuit
Sai Adith Senthil Kumar