Slow and Steady Wins the Race: Maintaining Plasticity with Hare and Tortoise Networks

This study investigates the loss of generalization ability in neuralnetworks, revisiting warm-starting experiments from Ash Adams. Our empiricalanalysis reveals that common methods designed to enhance plasticity bymaintaining trainability provide limited benefits to generalization. Whilereinitializing the network can be effective, it also risks losing valuableprior knowledge. To this end, we introduce the Hare Tortoise, inspired by thebrain’s complementary learning system. Hare Tortoise consists of twocomponents: the Hare network, which rapidly adapts to new informationanalogously to the hippocampus, and the Tortoise network, which graduallyintegrates knowledge akin to the neocortex. By periodically reinitializing theHare network to the Tortoise’s weights, our method preserves plasticity whileretaining general knowledge. Hare Tortoise can effectively maintain thenetwork’s ability to generalize, which improves advanced reinforcement learningalgorithms on the Atari-100k benchmark. The code is available athttps://github.com/dojeon-ai/hare-tortoise.

Further reading#

• Access Paper in arXiv.org