Transfer learning in reinforcement learning has been an active area of research over the past decade. In transfer learning, training on a source task is leveraged to speed up or otherwise improve learning on a target task. This paper presents the more ambitious problem of curriculum learning in reinforcement learning, in which the goal is to design a sequence of source tasks for an agent to train on, such that final performance or learning speed is improved. We take the position that each stage of such a curriculum should be tailored to the current ability of the agent in order to promote learning new behaviors. Thus, as a first step towards creating a curriculum, the trainer must be able to create novel, agent-specific source tasks. We explore how such a space of useful tasks can be created using a parameterized model of the domain and observed trajectories on the target task. We experimentally show that these methods can be used to form components of a curriculum and that such a curriculum can be used successfully for transfer learning in 2 challenging multiagent reinforcement learning domains.

PDF, HTML

In Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2016), Singapore, May 2016.

@inproceedings{AAMAS16-Narvekar, title={Source Task Creation for Curriculum Learning}, author={Sanmit Narvekar and Jivko Sinapov and Matteo Leonetti and Peter Stone}, booktitle={Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2016)}, month={May}, address={Singapore}, url="http://nn.cs.utexas.edu/?narvekar:aamas16", year={2016} }

Slides (PDF)