This paper focuses on learning in the presence of a Markovian teammate in Ad hoc teams. A Markovian teammate's policy is a function of a set of discrete feature values derived from the joint history of interaction, where the feature values transition in a Markovian fashion on each time step. We introduce a novel algorithm ``Learning to Cooperate with a Markovian teammate'', or LCM, that converges to optimal cooperation with any Markovian teammate, and achieves safety with any arbitrary teammate. The novel aspect of LCM is the manner in which it satisfies the above two goals via efficient exploration and exploitation. The main contribution of this paper is a full specification and a detailed analysis of LCM's theoretical properties.

PDF, PS, HTML

In Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), May 2013.

@inproceedings{AAMAS13-chakrado, title={Cooperating with a Markovian Ad Hoc Teammate}, author={Doran Chakraborty and Peter Stone}, booktitle={Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems (AAMAS)}, month={May}, url="http://nn.cs.utexas.edu/?AAMAS13-chakrado", year={2013} }