Data-Based Optimal Control of Multi-Agent Systems: A Reinforcement Learning Design Approach

Abstract

This paper studies optimal consensus tracking problem of heterogeneous linear multi-agent systems. By introducing tracking error dynamics, the optimal tracking problem is reformulated as finding a Nash-equilibrium solution of a multi-player games, which can be done by solving associated coupled Hamilton-Jacobi (HJ) equations. A data-based error estimator is designed to obtain the data-based control for the multi-agent systems. Using the quadratic functional to approximate the every agent's value function, we can obtain the optimal cooperative control by input-output (I/O) $Q$-learning algorithm with value iteration technique in the least-square sense. The control law solves the optimal consensus problem for multi-agent systems with measured input-output information, and does not rely on the model of multi-agent systems. A numerical example is provided to illustrate the effectiveness of the proposed algorithm.