Learning Policies with External Memory
Leonid Peshkin, Nicolas Meuleau, Leslie Kaelbling
Abstract
In order for an agent to perform well in partially observable domains, it is usually necessary for actions to depend on the history of observations. In this paper, we explore a stigmergic approach, in which the agent's actions include the ability to set and clear bits in an external memory, and the external memory is included as part of the input to the agent. In this case, we need to learn a reactive policy in a highly non-Markovian domain. We explore two algorithms: SARSA(λ), which has had empirical success in partially observable domains, and VAPS, a new algorithm due to Baird and Moore, with convergence guarantees in partially observable domains. We compare the performance of these two algorithms on benchmark problems.
Create a lesson
Related papers
Calibrated RF-Fingerprinting Under Interference With Heterogeneous Transmission Protocols
Tariq Abdul-Quddoos, Xiangfang Li, Lijun Qian
Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation
Haocheng Xi, Yiming Xie, Hexu Zhao et al.
Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL
Juzheng Zhang, Disha Makhija, Manoj Ghuhan Arivazhagan et al.
RISC-V and machine learning: a survey
Shriman Keshri, Apparna Singh, Chinmaya Kumar Palo et al.
Epidemiological Causal Graph Identification: Challenges, Identifiability and Algorithms
Sambit Mishra, Yingying Wang, Christine K. Johnson et al.
Multi-center Medical Data Mining with FL-Net - A One-stop Shop for Federated Learning
Simon Süwer, Julian Klemm, Elisa Acitelli et al.