Selection Criterion for Log-Linear Models Using Statistical Learning Theory

Abstract

Log-linear models are a well-established method for describing statistical dependencies among a set of n random variables. The observed frequencies of the n-tuples are explained by a joint probability such that its logarithm is a sum of functions, where each function depends on as few variables as possible. We obtain for this class a new model selection criterion using nonasymptotic concepts of statistical learning theory. We calculate the VC dimension for the class of k-factor log-linear models. In this way we are not only able to select the model with the appropriate complexity, but obtain also statements on the reliability of the estimated probability distribution. Furthermore we show that the selection of the best model among a set of models with the same complexity can be written as a convex optimization problem.