Pickle: Precise, Flexible Cross-Core Last-level Cache Data Prefetching for Irregular Memory Accesses

Abstract

Graph analytics and sparse scientific workloads are dominated by parallel chains of data-dependent, long-latency memory accesses whose patterns are difficult for hardware to infer yet straightforward to express in software. Conventional hardware prefetchers attempt to recover this information from address streams alone, but false positives lead to substantial memory traffic overhead. Software-assisted approaches offer greater flexibility but still consume core limited resources. We propose Pickle, a software-defined, hardware-managed lastlevel cache (LLC) prefetcher that follows the decoupled access/execute philosophy. Pickle serves as an independent access engine, fully decoupled from core resources, that executes prefetch kernels sliced from the original application to bring data into the shared LLC ahead of demand. We evaluate Pickle using full-system, cycle-level simulation of a cluster of 8 high-performance cores, running all GAP benchmark suite algorithms across nine real-world graphs and irregular-access dominated scientific applications from the NAS parallel benchmark suite. Over a no-prefetching baseline, Pickle achieves 1.49x geomean speedup with only 2% DRAM traffic overhead on graph algorithms, and 1.53x with a 4.5% memory traffic reduction on NAS scatter/gather kernels. For reference, the state-of-the-art coreprivate indirect prefetcher achieves 1.40x but incurs 43% DRAM traffic overhead on graph workloads, and 1.36x at zero traffic overhead on scatter/gather kernels, illustrating the challenge of inferring irregular access patterns without application-level context. Pickle also composes transparently with private cache prefetchers: combining it with the state-of-the-art indirect or a simple stride prefetcher yields 1.65x-1.66x and 1.72x-1.84x geomean speedup on graph and NAS scatter/gather workloads, respectively.