Verifier-Guided Self-Consistency: Test-Time Reasoning with Imperfect Generators and Verifiers
Ekaterina Fadeeva, Xueyan Li, Mrinmaya Sachan, Maxim Panov
Abstract
Large language models can solve many multi-step reasoning problems, but their reasoning remains brittle: plausible intermediate steps often lead to incorrect answers. Improving reasoning at test time requires balancing two imperfect signals: the generator, which provides diversity, and the verifier, which guides correctness. Existing approaches largely favor one side of this tradeoff. Verifier-weighted voting relies on trajectories sampled from the generator and cannot steer generation toward correct ones. By contrast, beam-style verifier-guided search can steer generation, but may over-concentrate on narrow trajectory sets and amplify verifier errors. We propose Verifier-Guided Self-Consistency (VGSC), a stochastic decoding framework that combines step-level correctness and future-outcome estimates to guide reasoning during generation. VGSC repeatedly proposes candidate next steps from the base language model and uses both signals in an acceptance/resampling procedure, while maintaining multiple trajectories for final answer-level self-consistency. We provide a probabilistic interpretation of VGSC as approximate sampling from a verifier-tilted trajectory distribution. Across multiple reasoning benchmarks, generator models, and verifier families, VGSC achieves higher average accuracy than representative baselines.
Create a lesson
Related papers
User Embeddings are Superpositions of Interpretable Behavioral Modes
Jonathan Kozaczuk, Zheng Dong, Noyan Evirgen et al.
XStack: A Benchmark for ML Repo Migration
Linrong Cai, Taiming Lu, Haozhe Chen et al.
Same Behavior, Different Mechanism: What Finite Experiments Can Certify About Integrated Information
Linqian Wu, Diaoxuan He
Towards interpreting any residual stream direction by scaling activation inversion
Celeste De Schamphelaere, Juan J. Vazquez, Ari Spiesberger et al.
Adversarial Training with Large Step Sizes: Implicit Bias, Optimization, and Sharpness
Yi Feng, Andrea Paudice
Unpacking the Compute Efficiency of Strong-to-Weak OPD against GRPO
Zeyu Leo Liu, Yang Li, Bo Pang et al.