Higher-Order Bayesian Networks, Exactly (Extended version)

Abstract

Bayesian networks (BNs) are graphical \emph{first-order} probabilistic models that allow for a compact representation of large probability distributions, and for efficient inference, both exact and approximate. We introduce a \emph{higher-order} programming language -- in the idealized form of a $\lambda$-calculus -- which we prove \emph{sound and complete} w.r.t. BNs: each BN can be encoded as a term, and conversely each (possibly higher-order and recursive) program of ground type \emph{compiles} into a BN. The language allows for the specification of recursive probability models and hierarchical structures. Moreover, we provide a \emph{compositional} and \emph{cost-aware} semantics which is based on factors, the standard mathematical tool used in Bayesian inference. Our results rely on advanced techniques rooted into linear logic, intersection types, rewriting theory, and Girard's geometry of interaction, which are here combined in a novel way.