Do Tabular Foundation Models Know Physics? Contamination, Units, and the Deterministic Limit
Wassim Tenachi, Yashar Hezaveh, Laurence Perreault Levasseur, Pierre-Luc Bacon
Abstract
Tabular foundation models (TFMs) learn to fill in tables the way language models fill in text, and tables are arguably the format in which most physical measurement arrives. Did they learn any physics in the process? They are Bayesian by construction, so the question is what their prior contains. We probe it directly, evaluating four of them (TabPFN-3, TabICLv2, TabDPT and Real-TabPFN-2.5) against six baselines on datasets sampled from 316 physical equations, in and out of domain. TFMs dominate, out of the box and after tuning. But we show that their prior can represent neither a noiseless mechanism nor physical units, which is why they interpolate physics without yet being able to act as physical models.
Create a lesson
Related papers
A Common Measure of Communication for Speech Brain-Computer Interfaces
Dulhan Jayalath, Benjamin Ballyk, Oiwi Parker Jones
Graph Machine: Towards Better Pretraining via Edges
Lintai Hou
The Implications of Linguistic Illegibility for LLM Security
James Mickens
Post-Training Language Models for Gold-Medal Performance in Coding Competitions
Aleksander Ficek, Sean Narenthiran, Mehrzad Samadi et al.
UE5M3 FP4 Block Scaling for Stable Language Model Pretraining
Robert Hu, Carlo Luschi, Paul Balanca
Cliff: Learning Process Rewards from the First Mistake
Peixuan Han, Runhui Wang, Ketan Ramaneti et al.