Design Principles for Creating Human-Shapable Agents (2009)
In order for learning agents to be useful to non-technical users, it is important to be able to teach agents how to perform new tasks using simple communication methods. We begin this paper by describing a framework we recently developed called Training an Agent Manually via Evaluative Reinforcement (TAMER), which allows a human to train a learning agent by giving simple scalar reinforcement signals while observing the agent perform the task. We then discuss how this work fits into a general taxonomy of methods for human-teachable (HT) agents and argue that the entire field of HT agents could benefit from an increased focus on the human side of teaching interactions. We then propose a set of conjectures about aspects of human teaching behavior that we believe could be incorporated into future work on HT agents.
In AAAI Spring 2009 Symposium on Agents that Learn from Human Teachers, March 2009.

Ian Fasel ianfasel [at] cs utexas edu
W. Bradley Knox bradknox [at] mit edu
Peter Stone pstone [at] cs utexas edu