Mean Field Variational Bayesian Inference and Statistical Mechanics of Gaussian Mixture Model

Abstract

One of the main modeling in many data science applications is the Gaussian Mixture Model (GMM), and Mean Field Variational Bayesian Inference (MFVBI) is classically used for approximate fast computation. In this paper, our aim is to lay a mathematical foundation for a rigorous analysis of the MFVBI applied to the GMM. Several fundamental key concepts from statistical mechanics surge naturally throughout our process. It turns out that GMM can be considered as a generalization of Curie-Weiss model in statistical mechanics. The standard quantities like partition function, Legendre transform and free energy come into operation. The initial system of equation \eqref{introp} reduces to a simpler \eqref{bonyi}. We introduce a temperature parameter in order to accommodate a phase transition phenomena which can finally guarantee the accuracy of the solutions to MFVBI.