Emergence of Structured Behaviors from Curiosity-Based Intrinsic Motivation.

Infants are experts at playing, with an amazing ability to generate novelstructured behaviors in unstructured environments that lack clear extrinsicreward signals. We seek to replicate some of these abilities with a neuralnetwork that implements curiosity-driven intrinsic motivation. Using a simplebut ecologically naturalistic simulated environment in which the agent can moveand interact with objects it sees, the agent learns a world model predictingthe dynamic consequences of its actions. Simultaneously, the agent learns totake actions that adversarially challenge the developing world model, pushingthe agent to explore novel and informative interactions with its environment.We demonstrate that this policy leads to the self-supervised emergence of aspectrum of complex behaviors, including ego motion prediction, objectattention, and object gathering. Moreover, the world model that the agentlearns supports improved performance on object dynamics prediction andlocalization tasks. Our results are a proof-of-principle that computationalmodels of intrinsic motivation might account for key features of developmentalvisuomotor learning in infants.