Evaluating MFU as a Proxy for GPU Power for Energy-Aware Simulation of LLM Training
Niklas Enskat, Philipp Wiesner
Abstract
High-fidelity performance simulators are essential for designing and configuring efficient AI systems, yet today's tools lack the ability to predict power consumption. Established GPU power models rely on hardware utilization counters, which do not exist until the workload has actually run. This work evaluates whether Model FLOPs Utilization (MFU)-an analytical, software-defined metric relating achieved throughput to peak hardware capability-can serve as a portable, software-defined predictor of GPU power for LLMs. We benchmark almost 3000 single-device training runs across six GPUs, covering different model families, numerical precisions, batch sizes, and context-window lengths. We find that a linear MFU-based power model fits every tested GPU as long as the workload is compute-bound, as in production LLM training. Fitting per-(GPU, dtype, batch size) instead of per-GPU drops the within-cell mean error from around 10% to around 1%, matching the cross-repeat measurement-noise floor.
Create a lesson
Related papers
Optimal Scheduling in Generalized Switch in Heavy Traffic
Runhan Xie, Ziv Scully, Rhonda Righter et al.
Discovering Performance Archetypes: Critical-Path-Aware Pattern Analysis and Regression Detection
Kaveh Shahedi, Heng Li, Maxime Lamothe et al.
Strong aggregation of the Markov chains associated with matching models based on the automorphism group of their compatibility graphs
Moyi Yang, Jean-Michel Fourneau
Accelerating Transfer-Learning-Based Autotuning with Predictive LLVM IR Performance Ranking
Md Arafat Hossain, Thomas Randall, Akash Dutta et al.
Learning Metastable Dynamics
Rupak Majumdar, Mahmoud Salamati, Nikhil Singh et al.
Dissecting GPU Utilization for LLM Inference on Nvidia Hopper
Mohammad Siavashi, Gerald Q. Maguire, Dejan Kostic et al.