Towards Automated Confidence Bound Provers and Searchers

Abstract

In this work we lay the groundwork for automating the process of finding and proving the validity of lower confidence bounds of the mean. Our key finding is based on the observation that finding an optimal confidence bound under certain conditions can be formulated as an optimization problem. We use this observation to show that any valid confidence bound (such as Hoeffding's) must be a relaxation of a certain optimization problem. To automatically find and prove confidence bounds, we need to automate the process of defining and finding such a relaxation. We define a family of relaxations parameterized by a function called the order function. This family of relaxations can approximate any other target relaxation such as Hoeffding's by using the target as the order function. When the order function is linear (such as the sample mean in the case of Hoeffding's), our relaxation is a linear-size mixed-integer linear program.