Multi-objective Neuroevolution of NPCs (2008)
Author: Jacob Schrum

Neuroevolution with multiple objectives allows for populations of agents to evolve different solutions based on various trade-offs inherent in any given domain. In particular, these methods can be used to evolve complex NPC behavior for games involving multiple objectives.

Multi-objective neuroevolution has currently been applied to a battle game domain in which a player fights against several attacking monster agents by using a bat as a weapon. As work on this project continues, multi-objective methods will be applied to increasingly complicated/difficult games with more objectives.

The player controls the green agent, and evolved neural networks control the yellow monsters. The monsters try to attack the player, and the player tries to hit them with a bat. The challenge is to evolve complex behavior for the monsters that combines several objectives, such as attacking, assisting others in attacking, staying alive, and avoiding getting hit.

In the movies below, the player is controlled by a static computer strategy. It constantly moves towards the nearest monster in front of it while swinging its bat.

Baiting Strategy

A monster being chased turns by small amounts to avoid facing the player as it backs up. The player turns to pursue it, which enables the player to eventually catch up with the monster, since turning slightly while moving puts less distance between player and monster than moving straight. However, since the player also turns to pursue the monster, the monsters chasing the player eventually catch up. Sometimes the monster being pursued will incur damage as a result of these slight turns, but regardless of whether the bait takes damage or not, any monsters pursuing from behind are eventually able to catch up and attack. One monster takes a risk so that the rest can catch up. Once the player is hit, the monsters are usually able to bounce the player back and forth for a few hits before it regains enough control to start pursuing a new monster.

Charging Strategy

Although monsters using the baiting strategy avoid damage and do a good job inflicting damage, they waste a lot of simulation time avoiding the player when they could be hitting it.

In some simulations a riskier strategy was learned that does not have this problem. It turns out that if the player moves forward after being hit, the monster that hit it can rush the player and hit it again in mid-swing. Being hit cancels the player's swing and knocks it back again. As long as the player rushes forward it is vulnerable to attack, because moving forward gives the monster just enough time to reach the player before the bat reaches the monster.

Wait and Strike Strategy

Recent experiments have revealed another effective strategy learned by multi-objective neuroevolution in this domain. This strategy involves waiting for just the right moment to rush in and strike the player.

Monsters start out surrounding the player, and approach it if they are not at risk of being hit by the bat. If the player is swinging at them they try to spread out off to the sides. If a monster can angle itself directly towards the player's left side, it will rush in to hit. This works because the player's bat swings from right to left, leaving a small opening on the left side. The manner in which the monsters spread out around the player also makes it easier for other monsters to find their chance to rush in.

After rushing in, these monsters will sometimes knock the player back several times the way that charging monsters do.
Jacob Schrum Ph.D. Alumni schrum2 [at] cs utexas edu
Risto Miikkulainen Faculty risto [at] cs utexas edu
Constructing Complex NPC Behavior via Multi-Objective Neuroevolution Jacob Schrum and Risto Miikkulainen In Proceedings of the Fourth Artificial Intelligence and Interactive Digital Entertainment Confer... 2008

IJCNN-2013 Tutorial on Evolution of Neural Networks Risto Miikkulainen To Appear In 2013. Tutorial slides.. 2013

BREVE Monsters BREVE is a system for designing Artificial Life simulations available at http://spiderlan... 2010