CPU-ARC: Agentic Reasoning Caching for on-CPU Reasoning Reuse
Peter Belcak, Greg Heinrich, Saurav Muralidharan, Shizhe Diao, Pavlo Molchanov
Abstract
Large language models reinvent the wheel all the time. They conflate task-solving with the execution of the solutions, which forces them to solve every problem from scratch. We show that once a task has been reasoned through, the application of its solution to a new problem instance can be done very efficiently and with little involvement from language models. We introduce agentic reasoning caching, a method for constructing robust CPU-executable agentic reasoning caches (ARCs) from LLM reasoning traces. ARCs show on-par performance with frontier LLMs across a range of difficult mathematical and scientific tasks, are fully explainable, continually trainable, robust to input perturbations, and 16-291x cheaper to invoke than the source model.
Create a lesson
Related papers
On the Reliability of Majority-Vote Pseudo-Labels in Test-Time Reinforcement Learning
Haoyu Wang, Ru Wang, Yuanyuan Wang et al.
Flow-Guided Target-Space Alignment via Path Consistency
Ruizhi Yuan, Zeqiu Yu, Wei Gao et al.
Ensemble Methodology for Reference-Free Confidence Estimation in LLM-based Code Summarization
Marco Colombo, Balaji A Jayaram, Sowmyakumari Selvaraj et al.
Can Agentic Systems Close the Gap in Radiology with Production AI Tools?
Aubin Charley, Julien Khlaut, Charles Corbière et al.
BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design
Dayou Du, Woon Yee Ng, Guangming Sheng et al.
Generative Latent Priors for Interpreting and Steering Protein Language Models
Daniel Hinjos, Lucía Urcelay