Taxonomy of Prediction

Abstract

A prediction makes a claim about a system's future given knowledge of its past. A retrodiction makes a claim about its past given knowledge of its future. The bidirectional machine is an ambidextrous hidden Markov chain that does both optimally by making explicit in its state structure all statistical correlation in a stochastic process. We introduce an informational taxonomy to profile these correlations via a suite of multivariate information measures. While prior results laid out the different kinds of information contained in isolated measurement of a bit, the associated informations were challenging to calculate explicitly. Overcoming this via bidirectional machine states, we expand that analysis to prediction and retrodiction. The result highlights fourteen new interpretable and calculable measures that characterize a process' informational structure. In addition, we introduce a labeling and indexing scheme that systematizes information-theoretic analyses of complex multivariate systems. Operationalizing this, we provide algorithms to directly calculate all of these quantities in closed form for finitely-modeled processes.