Funding the runners-up beats a golden ticket

Abstract

A golden ticket allows one reviewer to advance a proposal that the rest of a panel would reject. We assess its score-based rationale by comparing minority-support ranking with other rules for allocating the same number of places. We do so on 10,625 rejected submissions to a computer-science conference between 2017 and 2024, using later citation percentiles as the outcome. Minority-support ranking measured the gap between the two highest scores after adjustment for the panel mean and reviewer count. At a budget of 10% of the year's accepted count, panel-mean ranking selected submissions with a mean citation percentile 7.75 points higher than minority-support ranking (95% bootstrap interval 4.35 to 11.09). The difference was larger in the other sample periods and citation windows examined. At that budget, minority-support ranking was equivalent to a lottery within a margin of plus or minus 5 percentile points. Two further analyses, adjusted highest-score ranking and a matched comparison, found no clear citation advantage from minority support once the panel mean was accounted for. Ranking on score variance also outperformed minority-support ranking in a paired comparison. Estimates cover the 69.8% of eligible rejections whose later citations could be verified. These comparisons identify panel-mean ranking as a benchmark for evaluating whether golden tickets select work that would otherwise be overlooked.