RiskReason: A Benchmark for Verifiable Financial Risk Reasoning

Abstract

Robust financial-risk inference requires more than mathematical correctness. Models must accurately apply conditional rules, maintain evolving financial states, and make decisions that respect operational constraints. This work introduces RiskReason, a benchmark for verifiable financial-risk reasoning that spans diverse risk-management mechanisms. The benchmark includes Standard tasks, which assess individual financial mechanisms, and Long-horizon tasks, which combine multiple mechanisms through extended dependencies. Each task provides reference outcomes, formal specifications, reasoning annotations, and dependency representations to support systematic analysis of model performance. Eleven model configurations are evaluated using strict single-attempt accuracy, complemented by structural and qualitative failure analyses. The strongest configurations achieve 80.0--90.2\% overall accuracy, with several maintaining high performance on composed financial scenarios. However, analysis of failures reveals incorrect risk decisions despite accurate intermediate calculations, inconsistent propagation of financial states, and remediation actions that violate binding constraints. In contrast, some strict evaluation failures result from incomplete reporting rather than incorrect financial conclusions. These findings indicate that aggregate accuracy alone is insufficient to characterize financial-risk reasoning reliability and motivate evaluations that distinguish numerical correctness, state consistency, constraint satisfaction, and decision validity.