The project aims at constructing intelligent agents in sophisticated
simulated worlds through biologically inspired computation
methods. Despite successes in structured domains like board games and
medical diagnosis, traditional artificial intelligence (AI) techniques
are unlikely to lead to agents that can operate in the physical world
around us. The real world is noisy, dynamic, high-dimensional, and
only partially observable---very different from the structured worlds
where logic and search have been so successful. However, recent
increases in computing power provide a new opportunity, for two
reasons. First, it is now possible to simulate the physical world in
great detail, providing realistic challenges for AI in fully known and
controllable environments. Second, biologically inspired computation
techniques, such as neural networks, evolutionary computation, and
reinforcement learning, have become practical in complex
domains. Applying them to realistic simulations is a major step
towards building intelligent agents for the real world.
In this project, we aim to take advantage of this opportunity. In the
past several years, we have developed methods for evolving neural
networks in partially observable Markov decision tasks (e.g. NEAT),
and have built a 3D simulation platform where complex behavior can be
evaluated (i.e. NERO). In the proposed work, these technologies are
brought together to construct intelligent agents.
The challenge is that while NEAT is good at discovering sophisticated
low-level control behaviors, it is difficult for it to learn
high-level strategic behaviors. Such behaviors often depend on crucial
detail in the input and are multimodal, i.e. composed of distinctly
different behaviors at different times. The proposed solution is
twofold: (1) to evolve networks to construct their own high-level
features to represent such detail, and (2) to evolve useful component
behaviors and their effective combinations explicitly in two separate
populations. This approach has proven viable in preliminary
experiments.
In this project, the above two ideas will first be generalized
and combined into an integrated neuroevolution learning method. This
method will be then tested in several challenging benchmark tasks and
compared to other methods for implementing high-level strategic
behavior. Finally, the method will be scaled up to a robotic soccer
simulation in NERO, where it will be compared with existing hand-coded
and learned teams, and evaluated in human-subject experiments. The end
result of the project will be a general method for learning strategic
high-level behavior, with a thorough experimental understanding of
what makes it work and how it compares with other approaches.
The technology developed in the project will be immediately useful for
building high-level control systems for robots and virtual
agents. Such agents learn effective behavioral strategies and adapt
them when the world changes. This ability makes it possible to build
video games that are more engaging and entertaining than current
games, including games that can serve as training environments for
people. In the long term, the technology should lead to safer and more
efficient vehicle, traffic, and robotic control, improved process and
manufacturing optimization, and more efficient computer and
communication systems. In so doing, it will take us a step closer to
deploying artificial agents into the real world.
This research is supported by the Texas Higher Education
Coordinating Board under grant 003658-0036-2007.