Data-quality monitoring of luminosity blocks with ATLAS Open Data: an open benchmark and a learned within-run detector

Abstract

Data-quality monitoring at the LHC decides, luminosity block by luminosity block, which collision data enter physics analyses, yet its methods cannot be compared openly because the experiments' monitoring data are not public. We construct a public benchmark from the ATLAS Open Data release of the 2015--2016 proton--proton collisions at $\sqrt{s}=13$~TeV: 1289-bin histograms of reconstructed jets, electrons, photons, muons and the missing transverse momentum computed from them, for 39\,828 luminosity blocks of all 217 runs in the Good Run Lists; eight types of detector defects injected at event level at three strengths; real-data labels from the Good Run Lists and the liquid-argon calorimeter error flag; and a time-ordered protocol in which every detector is trained and calibrated on earlier runs only. Histogram shapes differ far more between runs than within a run (median $χ^2/n_\mathrm{dof}$ of 5.9 against 1.1), so references and autoencoders built across runs fail, and all competitive detectors use residuals with respect to a robust reference of each run. A gradient-boosted classifier on position-invariant features of these residuals, trained on synthetic defects and calibrated by split-conformal prediction, detects 43\% of the luminosity blocks with injected defects at a false-positive rate of 1.0\%, against 25\% for a matched-filter scan and 63\% for a single-histogram test that knows the defect pattern. For defect types excluded from its training, its efficiency falls to 18\%; adding pattern-agnostic synthetic detector failures to the training raises it to 34\%, above the scan, at a cost of three percentage points on the known types.