Verify the Verifier: Specification-Grounded LLM Cross-Checking for Bugs That Escape UVM Scoreboards
Ana E. Sanchez Villalobos, Michael D. Quinn, Aakash Tyagi, Minh D. Luu, David Kebo Houngninou
Abstract
A passing UVM regression shows the design agrees with the reference model on what the scoreboard checks, not that it meets the specification. Bugs that pass every test yet violate the specification, termed scoreboard escapes, survive mature testbenches. We present Verify the Verifier, to our knowledge the first framework whose LLM-based Anomaly Cross-Checker examines a passing regression's traces against the specification. Of 564 bugs inserted into four OpenTitan IPs, 68 are candidate scoreboard escapes: non-equivalent, exercised by the tests, yet passing. The framework confirmed 41 as scoreboard escapes with assertions failing on the buggy design and proven on the original.
Create a lesson
Related papers
Anticipating Future Build Results with BuildPredictor
Christian Macho, Katharina Stengg
Static Analysis Guidance for Secure LLM Code Generation
Daniele Lunghi, Alberick Euraste Djiré, Zacharie Chenail-Larcher et al.
SWE-Recreation: Benchmarking Coding Agents for Software Recreation
Zhiyao Xu, Zhanjie Ding, Hongyao Yu et al.
SkillBridge: Cost-Aware Diagnosis and Repair of Cross-OS Compatibility Defects in Agent Skills
Anjie Xu, Zixing Wang, Yifeng Cai et al.
Barriers to GenAI Integration During Large-Scale Agile Transformation: An Exploratory Case Study
Nils Brede Moe, Marthe Berntzen, Astri Barbala et al.
Adopting Embedded AI Through an Innovation Team
Viktoria Stray, Kristina Kantima Østvold, Agnes Hunstad Birkemo et al.