Parameter learning: stochastic optimal control approach with reinforcement learning

Abstract

In this study, we develop a stochastic optimal control approach with reinforcement learning structure to learn the unknown parameters appeared in the drift and diffusion terms of the stochastic differential equation. By choosing an appropriate cost functional, based on a classical optimal feedback control, we translate the original optimal control problem to a new control problem which takes place the unknown parameter as control, and the related optimal control can be used to estimate the unknown parameter. We establish the mathematical framework of the dynamic equation for the exploratory state, which is consistent with the existing results. Furthermore, we consider the linear stochastic differential equation case where the drift or diffusion term with unknown parameter. Then, we investigate the general case where both the drift and diffusion terms contain unknown parameters. For the above cases, we show that the optimal density function satisfies a Gaussian distribution which can be used to estimate the unknown parameter. Based on the obtained estimation of the parameters, we can do empirical analysis for a given model.