A latent-observed dissimilarity measure

Abstract

Quantitatively assessing relationships between latent variables and observed variables is important for understanding and developing generative models and representation learning. In this paper, we propose latent-observed dissimilarity (LOD) to evaluate the dissimilarity between the probabilistic characteristics of latent and observed variables. We also define four essential types of generative models with different independence/conditional independence configurations. Experiments using tractable real-world data show that LOD can effectively capture the differences between models and reflect the capability for higher layer learning. They also show that the conditional independence of latent variables given observed variables contributes to improving the transmission of information and characteristics from lower layers to higher layers.