Timely Ground-Station Status Delivery via UAV-Assisted Data Collection and Hierarchical RL

Abstract

We consider a UAV-assisted status-update system in which a rotary-wing unmanned aerial vehicle (UAV) collects updates from multiple sensors, stores the newest update of each sensor, and forwards selected buffered updates to a ground station (GS). We minimize a weighted sum of the time-average GS-side age of information (AoI) and UAV energy consumption over a finite horizon. The optimization is subject to half-duplex operation, sensor and UAV energy-causality constraints, transmission-rate and power limits, and a terminal-location constraint. Because collection changes the information age at the UAV but does not reduce the GS-side AoI until the buffered update is forwarded, mobility, collection, and forwarding decisions are coupled over time. We use a hierarchical AoI-energy actor-critic (HAE-AC) controller in which a coarse-timescale actor selects return-feasible UAV movements, while a deterministic per-slot scheduler constructs feasible collection or forwarding sets and assigns the minimum transmit power required for each scheduled transmission. In a 100-sensor deployment, HAE-AC reduces the average GS-side AoI by about 22% relative to an adaptive staleness-gated benchmark. Compared with forward-upon-collection, selective buffered forwarding reduces the number of downlink bursts by roughly 30% while increasing the number of collections per sensor by more than 23%.