Learning Based Control Policy and Regret Analysis for Online Quadratic Optimization with Asymmetric Information Structure

Abstract

In this paper, we propose a learning approach to analyze dynamic systems with asymmetric information structure. Instead of adopting a game theoretic setting, we investigate an online quadratic optimization problem driven by system noises with unknown statistics. Due to information asymmetry, it is infeasible to use classic Kalman filter nor optimal control strategies for such systems. It is necessary and beneficial to develop a robust approach that learns the probability statistics as time goes forward. Motivated by online convex optimization (OCO) theory, we introduce the notion of regret, which is defined as the cumulative performance loss difference between the optimal offline known statistics cost and the optimal online unknown statistics cost. By utilizing dynamic programming and linear minimum mean square biased estimate (LMMSUE), we propose a new type of online state feedback control policies and characterize the behavior of regret in finite time regime. The regret is shown to be sub-linear and bounded by O(ln T). Moreover, we address an online optimization problem with output feedback control policies.