\textbf{\textsc{HepUKE}}: Uncertainty-aware Knowledge Evolution for High Energy Physics Analysis Agents

Abstract

Knowledge bases are essential to agents for high energy physics (HEP), supplying few-shot evidence for previously unseen tasks. Their principal utility lies in strategic guidance for novel analyses rather than ready-made answers or reusable code. Prevailing systems, however, treat all retrieved evidence as equally trustworthy and fail to accrue knowledge from the analyses under assistance. In this paper, inspired by the standard HEP treatment of uncertainty, we propose \textbf{\textsc{\underline{HepUKE}}}: an \textbf{\underline{U}}ncertainty-aware \textbf{\underline{K}}nowledge \textbf{\underline{E}}volution system for \textbf{\underline{HEP}} that repurposes information entropy to unify statistical uncertainty from LLM generation and systematic uncertainty from HEP methodological choices. Upon it, we construct a self-evolving domain-specific-language (DSL) knowledge base and release a large-scale expert-annotated HEP physics-analysis benchmark. Per-evidence confidence is quantified via the information entropy of sampled DSL generations: high-confidence precedents constrain program synthesis, whereas analyses lacking close precedents are validated and written back, sustaining continual knowledge evolution. To handle the highly redundant yet logically stringent nature of HEP knowledge, we introduce an early-stopping controller that fuses a backward stationarity test on the answer posterior with a forward marginal information gain estimate. On the 1,000-pair BESIII PhysicsQA benchmark, HepUKE reduces retrieval rounds by 43.0\% while preserving most answer quality, and on the out-of-domain HotpotQA benchmark, it reduces retrieval rounds by 14.5\% with a similar quality--efficiency trade-off. The code and datasets used in this work are available at https://anonymous.4open.science/r/HepUKE-32FF.