Mixtures of Gaussian process priors

Abstract

Nonparametric Bayesian approaches based on Gaussian processes have recently become popular in the empirical learning community. They encompass many classical methods of statistics, like Radial Basis Functions or various splines, and are technically convenient because Gaussian integrals can be calculated analytically. Restricting to Gaussian processes, however, forbids for example the implemention of genuine nonconcave priors. Mixtures of Gaussian process priors, on the other hand, allow the flexible implementation of complex and situation specific, also nonconcave "a priori" information. This is essential for tasks with, compared to their complexity, a small number of available training data. The paper concentrates on the formalism for Gaussian regression problems where prior mixture models provide a generalisation of classical quadratic, typically smoothness related, regularisation approaches being more flexible without having a much larger computational complexity.