Statistical Challenges with Massive Data Sets in Particle Physics

Abstract

The massive data sets from today's particle physics experiments present a variety of challenges amenable to the tools developed by the statistics community. From the real-time decision of what subset of data to record on permanent storage, to the reduction of millions of channels of electronics to a few dozen high-level variables of primary interest, to the interpretation of these high-level observables in the context of an underlying physical theory, there are many problems that could benefit from a higher-bandwidth exchange of ideas between our fields. Examples of interesting problems from various stages in the collection and interpretation of particle physics data are provided in an attempt to whet the appetite of future collaborators with knowledge of potentially helpful techniques, and to encourage fruitful discussion between the particle physics and statistics communities.