On shared and multiple information

Abstract

We address three outstanding problems in information theory. Problem one is the definition of a non-negative decomposition of the information conveyed by two or more sources about a target variable into the specific contribution of each possible combination of the sources. Problem two is the definition of a measure of information shared by several sources about the target variable. Problem three is the definition of a measure of multiple mutual information, that is, the extension of mutual information to more than two variables.