Evaluating and Improving Weak Scalability Analysis of Visualization Algorithms

Abstract

Research on visualizing large-scale datasets traditionally relies on empirical evaluation of scalability, determining the effectiveness of specific computation methods, algorithmic strategies, or implementations. Weak scalability, which assesses the algorithm's performance as problem size and computing resources increase, is a valuable indicator for a method's applicability at scale. However, sufficiently large data sets with increasing size are needed for weak scalability studies. To this end, it is customary to use simple scaling techniques to increase problem size by generating larger input data sets from a base data set. Nevertheless, many visualization algorithms' workload depends on factors beyond input size, such as input data complexity or output size, leading to inaccuracies in the attributed weak scalability. In this work, we highlight different common data scaling methods on multiple algorithms and data sets, recognizing that the suitability of scaling approaches varies across algorithms and data sets. We present a method that effectively mitigates the observed inconsistencies in workload increases for the different scaling methods in a shared-memory setting. With this work, we aim to further the discussion on how to evaluate and report scalable visualization research.