A rapid algorithm to calculate joint probability matrices for joint entropies of arbitrary order

Abstract

There is no closed form analytical equation or quick method to calculate probabilities based only on the entropy of a signal or process. Except in the cases where there are constraints on the state probabilities, one must typically derive the underlying probabilities through search algorithms. These become more computationally expensive as entropies of higher orders are investigated. In this paper, a method to calculate a joint probability matrix based on the entropy for any order is elaborated. With this method, only first order entropies need to be successfully calculated while the others are derived via multiplicative cascades.