CPU Simulation with Ranked Set Sampling and Repeated Subsampling

Abstract

Computer system simulation studies routinely rely on executing a limited number of short application regions, since full end-to-end simulation is prohibitively time-consuming. To preserve representativeness, existing methods employ either random sampling or phase-based characterization to identify representative regions. In this work, we revisit random sampling in the context of computer architecture simulation. To assess how the confidence level varies with different micro-architectural configurations, we examine how the sample standard deviation relates to the sample mean. We show that the ranked set sampling (RSS) technique - well established in the statistical literature - maps naturally to architectural simulation and yields significantly tighter confidence intervals than simple random sampling. Across our experiments, RSS reduces the confidence interval width by up to 50%. We further introduce a repeated subsampling scheme that identifies representative simulation regions for future studies. For a fixed sample size, this approach reduces the maximum observed error from 35% to 10%. Evaluating two selection criteria, we find that more informed subsample selection provides additional accuracy gains. Overall, our method achieves an average error below 2% and a maximum error of 3.5% across individual SPEC CPU 2017 Integer applications when simulating 30 regions of 1 million instructions each.