Neuroevolution with multiple objectives allows for populations of agents to evolve different solutions based on various trade-offs inherent in any given domain. In particular, these methods can be used to evolve complex NPC behavior for games involving multiple objectives.
Multi-objective neuroevolution has currently been applied to a battle game domain in which a player fights against several attacking monster agents by using a bat as a weapon. As work on this project continues, multi-objective methods will be applied to increasingly complicated/difficult games with more objectives.
The player controls the green agent, and evolved neural
networks control the yellow monsters. The monsters
try to attack the player, and the player tries to hit them with a
bat. The challenge is to evolve complex behavior for the monsters
that combines several objectives, such as attacking, assisting
others in attacking, staying alive, and avoiding getting
In the movies below, the player is controlled by a static computer
strategy. It constantly moves towards the nearest monster in front
of it while swinging its bat.
A monster being chased turns by
small amounts to avoid facing the player as it backs up. The player turns to
pursue it, which enables the player to eventually catch up with the
monster, since turning slightly while moving puts less distance between
player and monster than moving straight. However, since the player also
turns to pursue the monster, the monsters chasing the player
eventually catch up. Sometimes the monster being pursued will incur damage as a result
of these slight turns, but regardless of whether the bait takes damage or not, any monsters
pursuing from behind are eventually able to catch up and attack.
One monster takes a risk so that the rest can catch up. Once the player is hit,
the monsters are usually able to bounce the player back and forth for a few hits
before it regains enough control to start pursuing a new monster.
Although monsters using the baiting strategy avoid damage and
do a good job inflicting damage, they waste
a lot of simulation time avoiding the player when they could be hitting it.
In some simulations a riskier strategy was learned that does not have this problem.
It turns out that if the player moves forward after being hit,
the monster that hit it can rush the player and hit it again in mid-swing. Being hit
cancels the player's swing and knocks it back again. As long as the player
rushes forward it is vulnerable to attack,
because moving forward gives the monster just enough
time to reach the player before the bat reaches the monster.
Wait and Strike Strategy
Recent experiments have revealed another effective strategy learned by
multi-objective neuroevolution in this domain. This strategy involves waiting for
just the right moment to rush in and strike the player.
Monsters start out surrounding the player, and approach it if they are not
at risk of being hit by the bat. If the player is swinging at them they try
to spread out off to the sides. If a monster can angle itself directly
towards the player's left side, it will rush in to hit. This works because
the player's bat swings from right to left, leaving a small opening on the
left side. The manner in which the monsters spread out around the player
also makes it easier for other monsters to find their chance to rush in.
After rushing in, these monsters will sometimes knock the player
back several times the way that charging monsters do.