As a classic example of imperfect information games, Heads-Up No-limit Texas Holdem (HUNL) has been studied extensively in recent years. While state-of-the-art approaches based on Nash equilibrium have been successful, they lack the ability to model and exploit opponents effectively. This paper presents an evolutionary approach to discover opponent models based on recurrent neural networks (LSTM) and Pattern Recognition Trees. Experimental results showed that poker agents built in this method can adapt to opponents they have never seen in training and exploit weak strategies far more effectively than Slumbot 2017, one of the cutting-edge Nash-equilibrium-based poker agents. In addition, agents evolved through playing against relatively weak rule-based opponents tied statistically with Slumbot in heads-up matches. Thus, the proposed approach is a promising new direction for building high-performance adaptive agents in HUNL and other imperfect information games.

PDF

In Proceedings of The Genetic and Evolutionary Computation Conference (GECCO 2018), Kyoto, Japan, July 2018. ACM.

@inproceedings{li:gecco18, title={Opponent Modeling and Exploitation in Poker Using Evolved Recurrent Neural Networks}, author={Xun Li and Risto Miikkulainen}, booktitle={Proceedings of The Genetic and Evolutionary Computation Conference (GECCO 2018)}, month={July}, address={Kyoto, Japan}, publisher={ACM}, url="http://nn.cs.utexas.edu/?li:gecco18", year={2018} }