Course Project Report: Comparing MCMC and Variational Inference for Bayesian Probabilistic Matrix Factorization on the MovieLens Dataset

Abstract

This is a course project report with complete methodology, experiments, references and mathematical derivations. Matrix factorization [1] is a widely used technique in recommendation systems. Probabilistic Matrix Factorization (PMF) [2] extends traditional matrix factorization by incorporating probability distributions over latent factors, allowing for uncertainty quantification. However, computing the posterior distribution is intractable due to the high-dimensional integral. To address this, we employ two Bayesian inference methods: Markov Chain Monte Carlo (MCMC) [3, 4] and Variational Inference (VI) [5, 6] to approximate the posterior. We evaluate their performance on MovieLens dataset [7] and compare their convergence speed, predictive accuracy, and computational efficiency. Experimental results demonstrate that VI offers faster convergence, while MCMC provides more accurate posterior estimates.