Generalization of Agent Behavior through Explicit Representation of Context (2021)
In order to deploy autonomous agents in digital interactive environments, they must be able to act robustly in unseen situations. The standard machine learning approach is to include as much variation as possible into training these agents. The agents can then interpolate within their training, but they cannot extrapolate much beyond it. This paper proposes a principled approach where a context module is coevolved with a skill module in the game. The context module recognizes the temporal variation in the game and modulates the outputs of the skill module so that the action decisions can be made robustly even in previously unseen situations. The approach is evaluated in the Flappy Bird and LunarLander video games, as well as in the CARLA autonomous driving simulation. The Context+Skill approach leads to significantly more robust behavior in environments that require extrapolation beyond training. Such a principled generalization ability is essential in deploying autonomous agents in real-world tasks, and can serve as a foundation for continual adaptation as well.
View:
PDF
Citation:
In Proceedings of the 3rd IEEE Conference on Games( ), 2021.
Bibtex:

Presentation:
Video
Suhaib Abdulquddos Masters Alumni suhaib [at] cs utexas edu
Risto Miikkulainen Faculty risto [at] cs utexas edu
Cem C Tutum Former Research Scientist tutum [at] cs utexas edu
ContextSkillCARLA Code relating to the experiments involved in studying the effectiveness of the Context+Skill neural network applied to t... 2022

ContextSkillFlappyBall Download at GitHub.

Context-skill model for extrapolati...

2021