Lucid Dreaming for Experience Replay: Refreshing Past States with the Current Policy
arXiv Preprint Archive September 29, 2020 Yunshu Du, Garrett Warnell, Assefaw Gebremedhin et al.
A new framework called Lucid Dreaming for Experience Replay (LiDER) refreshes past experiences in a replay buffer by having the agent revisit a past state and simulate new actions using its current policy. If the simulated experience is better than the original, it replaces the old memory. This approach improves data efficiency and performance in off-policy reinforcement learning. In tests on six Atari 2600 games, LiDER consistently outperformed the baseline actor-critic algorithm. The method is designed for easy integration into multi-worker RL algorithms that use experience replay.