A Partial Order on Uncertainty and Information

Abstract

Information and uncertainty are closely related and extensively studied concepts in a number of scientific disciplines such as communication theory, probability theory, and statistics. Increasing the information arguably reduces the uncertainty on a given random subject. Consider the uncertainty measure as the variance of a random variable. Given the information that its outcome is in an interval, the uncertainty is expected to reduce when the interval shrinks. This proposition is not generally true. In this paper, we provide a necessary and sufficient condition for this proposition when the random variable is absolutely continuous or integer valued. We also give a similar result on Shannon information.