Beyond Parallel Blindness: Information Floors and Model Gaps in Block Drafting
Xinwei Qiang, Xiang Fang, Chang Chen, Yue Guan, Yufei Ding
Abstract
Block drafters propose several tokens in one forward pass, before earlier target tokens are realised. Their rejection mixes two losses: missing within-block path information and imperfect modelling of observable information. Accepted length cannot distinguish them. We separate the two with an information floor, the minimum expected rejection at a specified conditioning order; rejection above this floor is the model gap. Estimating both from target rollouts across four domains, four open-weight targets, and a frontier API target yields three findings. First, the all-parallel floor reaches 0.286 at the final slot on Qwen3-4B, limiting even the best proposal to 71\% per-slot acceptance. Second, one realised token removes 86--100\% of this floor, a locality also recovered by an independent mutual-information analysis. Third, current drafters remain far above their floors: the final-slot model gap accounts for 43--64\% of DFlash rejection and 85--92\% of DSpark's oracle-conditioned rejection. These findings separate the value of short-range conditioning from proposal quality.
Create a lesson
Related papers
Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO
Yunpeng Ba, Zhi Zheng, Yue Xie et al.
QuantumBoostNet: A Hybrid Classical-Quantum Architecture for Enhanced Accuracy in Cardiac Ultrasound View Identification
Mihai Udrescu-Milosav, Stefan-Alexandru Jura, Mihai Udrescu et al.
MM-Spectrum: Multimodal Multi-spectral Molecular Structural Elucidation with a Stable MoE Framework
Hai-tao Yu, Nan Min, Zheng Fang et al.
Making Latent Evolution Explicit: Operator-Structured Transitions for World Action Models
Xiaoxiao Lu, Yunlong Dong, Jiahao Shi et al.
Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuit
Sai Adith Senthil Kumar
Importance Scoring of Transformer Attention Heads in Learning Tabular Data
Ahmad Jad Allah, Kazi F. Akhter, Md. Kamrozzaman Bhuiyan et al.