Continual Lifelong Learning with Neural Networks: A Review.

Humans and animals have the ability to continually acquire and fine-tuneknowledge throughout their lifespan. This ability is mediated by a rich set ofneurocognitive functions that together contribute to the early development andexperience-driven specialization of our sensorimotor skills. Consequently, theability to learn from continuous streams of information is crucial forcomputational learning systems and autonomous agents (inter)acting in the realworld. However, continual lifelong learning remains a long-standing challengefor machine learning and neural network models since the incrementalacquisition of new skills from non-stationary data distributions generallyleads to catastrophic forgetting or interference. This limitation represents amajor drawback also for state-of-the-art deep neural network models thattypically learn representations from stationary batches of training data, thuswithout accounting for situations in which the number of tasks is not known apriori and the information becomes incrementally available over time. In thisreview, we critically summarize the main challenges linked to continuallifelong learning for artificial learning systems and compare existing neuralnetwork approaches that alleviate, to different extents, catastrophicinterference. Although significant advances have been made in domain-specificcontinual lifelong learning with neural networks, extensive research effortsare required for the development of general-purpose artificial intelligence andautonomous agents. We discuss well-established research and recentmethodological trends motivated by experimentally observed lifelong learningfactors in biological systems. Such factors include principles of neurosynapticstability-plasticity, critical developmental stages, intrinsically motivatedexploration, transfer learning, and crossmodal integration.