Estimation of Multiple Quantiles in Dynamically Varying Data Streams

Abstract

In this paper we consider the problem of estimating quantiles when data are received sequentially (data stream). For real life data streams, the distribution of the data typically varies with time making estimation of quantiles challenging. We present a method that simultaneously maintain estimates of multiple quantiles of the data stream distribution. The method is based on making incremental updates of the quantile estimates every time a new sample from the data stream is received. The method is memory and computationally efficient since it only stores one value for each quantile estimate and only performs one operation per quantile estimate when a new sample is received from the data stream. The estimates are realistic in the sense that the monotone property of quantiles is satisfied in every iteration. Experiments show that the method efficiently tracks multiple quantiles and outperforms state of the art methods.