A Statistical Framework for Training Data Auditing of Large Language Models
Shengjie Niu, Yeheng Ge, Jian Huang
Abstract
Training-data auditing asks whether data of interest were used to train a large language model (LLM), a question central to privacy auditing, copyright protection, and benchmark-contamination detection. These concerns are particularly relevant to the use of copyrighted books and scientific papers in pre-training, as well as synthetic text and medical question-answer records in fine-tuning. In practice, training membership signals may be concentrated in only a few passages of a long document, making it important to aggregate evidence within each document without diluting these localized signals. Beyond individual documents, auditing a target collection requires an overall assessment based on the document-level evidence. In this paper, we propose a unified statistical framework for document-level membership inference, target-collection global auditing, and suspicious-subset identification. We integrate complementary chunk-level membership evidence with Cauchy combination to construct document scores for document-level membership inference. The document scores and their calibrated p-values further enable an overall assessment of training-data collection use and the identification of suspicious documents through multiple testing. Under the stated exchangeability conditions, we establish finite-sample validity of document-level p-values, type-I error control for global auditing, and false discovery rate control for subset identification. Case studies on BookTection, Pile-arXiv, Cosmopedia, and HealthCareMagic demonstrate the effectiveness and reliability of the proposed framework in auditing pre-training and fine-tuning data.
Create a lesson
Related papers
Bayesian optimization on amortized search space
Tristan Fauvel, Federico Pavone, Titouan Guillou et al.
Exact Bayesian layers via characteristic functions
William H. W. Thompson, Guillaume St-Onge, Laurent Hébert-Dufresne et al.
Flow Matching from Non-Gaussian Sources: A Theory of Efficient Sampling
Yuchen Zhou, Lili Zheng, Yuxin Chen et al.
Conformal Selection for Hierarchical Streaming Data
Zhenyuan Sun, Chengyao Yu, Ruixing Ming et al.
Hellinger Decision Trees for Density Ratio Estimation: Finite-Sample Guarantees
Ren-Rui Liu, Zheng-Chu Guo
Differentially Private Score Estimation with Applications to Score-Based Generative Models
Mengxiao Gao, Elly K. H. Hung, Yuchen Wu et al.