FPBench: Application-Oriented Error Decomposition for Foundation Potentials

Abstract

Foundation potentials (FPs) have emerged as a new basis for atomistic modeling. While their evaluation using average energy and force errors often indicates near-DFT accuracy, their performance in practical computational studies remains inconsistent. Here, we present FPBench, an application-oriented benchmark that evaluates FPs on representative computational tasks. Multiple state-of-the-art FPs are assessed on three fundamental tasks: force prediction for atomistic simulations, energy ranking for substitutional and vacancy orderings, and ion/vacancy migration. Benchmarking these FPs shows that average force and energy errors often fail to predict task performance, revealing substantial differences in practical reliability. FPBench introduces application-oriented error decomposition through metrics that resolve performance according to the physically consequential quantities and configurations governing computational tasks, including the fractions of highly accurate and large-force-error atoms, far-from-equilibrium atoms, relative phase-stability and convex-hull agreement, and along-path errors in ion migration. These error-decomposition metrics identify where FP errors arise within specific computational tasks, providing targeted guidance for model development. FPBench provides an open benchmark, evaluation code, and a public leaderboard for rigorous FP assessment and development.