Peon Demos (from about year 2000)

This page is based on research in Real-Time Interactive Neuro-Evolution by Adrian Agogino and Kenneth Stanley

Here is a link to a program for viewing .avi movies on unix: XAnim Home Page (LINK EXPIRED)

NEW: Press here to try an interactive Java demo with Peons (LINK EXPIRED)

What is this?

The following demos depict real-time neuroevolution in action. They are the result of research in the area of real-time interactive neuro-evolution, which basically means evolving brains in a live-action scenario, where a user can interact with them. The movie clips are actual captures of the evolution occuring, offering an unusually clear glimpse at the process. If you find this interesting, take a look at our paper. Note that the actual animation of the peons is smoother in the program than in the movie clips. This allows them to be more compact.

What is going on in the movies?

  • The demos actually depict a simple game.
  • There are a bunch of small dots, called peons.
    (Note that occasioanlly we see a larger peon, shaped as a square)
  • There is a larger, red dot, called "the enemy."
  • There are one or two large yellow circles, called "goldmines."
  • There is a square in the middle, called "the base."

    The peons always leave from the base. If they don't travel to one of the goldmines in an alotted amount of time, they die. Also, if the enemy touches a peon, it dies. When a successful peon makes it to a mine, it is transported back to the base to try again. Thus, a good peon can dodge the enemy and find a mine in a short timespan.

    Each peon has a neural network brain, which is a simple feed-forward network. The input to the network is visual information about where the enemy is, and where the mines are. The output is a movement decision. However, keep in mind that the brains are initially completely random in their connections, so peons initially do not know anything about their environment, or how to interpret their sensors.

    As the game progresses, peons are given a fitness rating. In the movies below, the highest ranking peon at any given time is represented as a large box, so that they stand out. High ranking peons are allowed to reproduce to replace lower ranking peons that have died. When peons reproduce, mutations occur. Hence, we get evolution.

    Here are the peon movies:

    Untrained Peons

    The population is composed entirely of peons with randomly connected brains, so we don't expect to see anything intelligent. Not surprisingly, we see a lot of peons scattering in random directions. This is how an initial population looks before any adaptation occurs.

    Naive Peons

    In only a matter of seconds, the peon population has learned to seek out the nearest mine. However, this is a naive strategy- the enemy never moved while they found this technique, and therefore they are not prepared to react appropriately when the enemy finally attacks. This illustrates the tendency of evolution to produce populations that are deceptively simplistic, yet still successful.

    Peons Devastated

    Suddenly, the enemy attacks the naive population from the above clip. The population has not learned to deal with a moving enemy, and therefore it is quickly devastated. You can actually see the population dying out here as its favorite strategy is completely outwitted.

    Peons Employ New Strategy

    In only 20 or 30 seconds, the peons devise a new strategy which beats the enemy's strategy. (Note that the enemy's strategy is fixed in this example.) This illustrates how flexible online adaptation is, and how it is possible in real-time. The strategy employed appears to confuse the enemy, rendering it harmless.

    Note: The following two demos are with a new peon population starting from scratch

    Orbital Training an Untrained Set

    Once again we are observing an untrained population. However, notice that the mine and the enemy are both "orbiting" the peons' home base. This is a training scenario we developed to obtain a population that could not rely on the destination being in any one location, yet still had to avoid the enemy. This is a complicated prospect for the untrained peons. Notice near the end of the movie where one aimless peon accidentally wanders into the passing mine. This peon will now be honored with higher selectivity in the future. Interestingly, its strategy is most likely almost useless, since it just knows to wander in a certain direction, which won't work with an orbiting mine. The situation will tend to select for peons that are a little sensitive to the location to of the mine, and eventually strengthen that sensitivity into a following instinct.

    Peons after Orbital Training

    The peons in this clip have developed a very sophisticated and reactive neural network to deal with a very quickly changing situation. Please compare this clip to the previous one to see how far they have come. Attaining this level of intelligence took a few minutes of training with exactly the right parameters for fitness selection. (If the parameters are changed, the peons are not able to attain this level.) Notice the elegant balance between enemy avoidance and mine following that the peons display. Please note that the mine stops midway through the clip. This is part of the confusion the peons deal with- the mine is set to stop moving at random times and then start up again later.

    Contact authors here:

    kstanley@cs.utexas.edu
    agogino@ece.utexas.edu