Discard the Dross and Select the Essential: Pre-query Sample Selection for Black-box Membership Inference Attacks

Abstract

Black-box membership inference attacks (MIAs) rely on target-model queries to infer whether candidate samples were used for training. However, membership signals are highly non-uniform across samples: some candidate samples support strong member/non-member separability, whereas many others provide little useful signal. Consequently, indiscriminate querying can incur substantial query cost and increase query-induced exposure, with limited marginal benefit for inference. This raises a key question: which candidate samples are worth querying for black-box MIAs? To address this question, we propose PSS-MIA, a pre-query sample selection framework which can be embedded with any existing MIA methods. PSS-MIA proceeds in two stages: it first ranks candidate samples and selects a subset expected to support stronger membership inference, then queries the selected samples and uses the returned outputs for an existing black-box MIA, thereby reducing query cost and query-induced exposure. In the first stage, we propose Loss-Gap Ranking (LGR), which ranks candidate samples by estimating the strength of their membership signal using loss gaps computed from reference models. Experiments on CIFAR-10, CIFAR-100, and CINIC-10 with five representative black-box MIA methods demonstrate that PSS-MIA with LGR consistently outperforms all other compared methods. Moreover, under a 0.1% FPR constraint, PSS-MIA can save at least 83.1%, 60.6%, and 80.4% of the query budget for the three datasets, respectively.