Multi-modal Approaches to Evolving Behavior for Multi-task Games (2011)
Author: Jacob Schrum
This series of videos demonstrate the benefit of multimodal networks in evolving behavior for "multitask domains", defined here as domains in which agents face multiple separate tasks in separate evaluations: i.e. an agent is evaluated in one task and receives a fitness score(s), and then in a completely different task for which it receives another score(s). The combined scores from all tasks define the performance of the agent. Evolving multimodal neural networks helps discover agents capable of performing multiple tasks. In order to account for multiple scores from multiple tasks, Pareto-based multiobjective evolution is used.

Four methods are shown:

Control: Traditional neuroevolution where each network has only one output mode.
MM(P): Neuroevolution with a mode mutation in which new modes are connected to previous modes. Mode mutation adds a new set of output modes to a network. Each mode has a "preference neuron", the output of which indicates the network's preference (relative to the other modes) for using that mode on a given time step of evaluation. The network can learn to switch between different modes for different situations, which can help it develop different modes for the different tasks. With this type of mode mutation, all new mode inputs come from the previous existing network mode, to assure similarity to an existing mode.
MM(R): Mode mutation in which new inputs come from random hidden layer nodes in the network. This approach tends to make new modes that are very different from existing modes, which helps quickly develop new behaviors. MM(R) also supports and uses a mode deletion mutation, which deletes the least-used mode from the previous evaluation. Such deletion is not possible with MM(P) because it would disconnect modes from the network, but this operation is always safe with MM(R).
Multitask: Networks have explicit knowledge of the number of tasks, and know which one they are performing at any given time. This knowledge allows them to have a dedicated mode for each task. The domains below each consist of two tasks, so multitask networks each have two modes. When in task one, the networks only use the outputs from mode one, and in task two they use the outputs from mode two.

Two domains are used to evaluate agents. Each domains consists of two separate tasks.

Front/Back Ramming Domain

In each of these tasks, the evolved agents have rams (white orbs) that they must use as weapons to strike the scripted opponent. The evolved agents are also vulnerable to attacks from the scripted agent, which attempts to sneak around the rams to hit the evolved agents. The difference in the two tasks is the positioning of the ram. In the Front Ramming task, the ram is on the front of the agents, and in the Back Ramming task it is on the backs of the agents. Evolved agents must learn to orient themselves according to the task, so that they properly use the rams.

Front/Back Ramming: Control

Front/Back Ramming: MM(P)

Front/Back Ramming: MM(R)

Front/Back Ramming: Multitask

Predator/Prey Domain

In the Predator task, a team of four evolved agents try to hit a scripted prey agent while preventing it from escaping. In the Prey task, the evolved team must run away from the same scripted agent, which now behaves as a predator.

Predator/Prey: Control

Predator/Prey: MM(P)

Predator/Prey: MM(R)

Predator/Prey: Multitask

Jacob Schrum Ph.D. Student schrum2 [at] cs utexas edu
IJCNN-2013 Tutorial on Evolution of Neural Networks Risto Miikkulainen To Appear In 2013. Tutorial slides.. 2013

Evolving Multimodal Networks for Multitask Games Jacob Schrum and Risto Miikkulainen In Proceedings of the IEEE Conference on Computational Intelligence and Games (CIG 2011), 102... 2011

Evolving Multimodal Networks for Multitask Games Jacob Schrum and Risto Miikkulainen Simon M. Lucas, editors, IEEE Transactions on Computational Intelligence and AI in Games, 4(2... 2012

BREVE Monsters BREVE is a system for designing Artificial Life simulations available at http://spiderlan... 2010