Variational Bayesian hierarchical regression for data analysis

Abstract

Collected data, which is used for analysis or prediction tasks, often have a hierarchical structure, for example, data from various people performing the same task. Modeling the data's structure can improve the reliability of the derived results and prediction performance of newly unobserved data. Bayesian modeling provides a tool-kit for designing hierarchical models. However, Markov Chain Monte Carlo methods which are commonly used for parameter estimation are computationally expensive. This often renders its use for many applications not applicable. However, variational Bayesian methods allow to derive an approximation with much less computational effort. This document describes the derivation of a variational approximation for a hierarchical linear Bayesian regression and demonstrates its application to data analysis.