neural networks research group
areas
people
projects
demos
publications
software/data
Reinforcement Learning
Reinforcement Learning tasks are learning problems where the desired behavior is not known; only sparse feedback on how well the agent is doing is provided. Reinforcement Learning techniques include value-function and policy iteration methods (note that although evolutionary computation and neuroevolution can also be seen as reinforcement learning methods, they are presented separately in this area hierarchy). Our research in reinforcement learning includes real-world applications such as packet routing, satellite communication, robot navigation, and sensorimotor embedding.
People
Adrian Agogino
Collaborator
adrian@email.arc.nasa.gov
Igor V. Karpov
Ph.D. Student
ikarpov@cs.utexas.edu
Shailesh Kumar
Masters Student (Alumni)
ShaileshKumar@fairisaac.com
Risto Miikkulainen
Professor
risto@cs.utexas.edu
Jefferson Provost
Ph.D. Student (Alumni)
jefferson.provost@gmail.com
Jacob Schrum
Ph.D. Student
schrum2@cs.utexas.edu
Projects
The OpenNERO AI Research and Education Platform
Since 2009
Learning Strategic Behavior in Sequential Decision Tasks
2009 - 2012
Controlling a Finless Rocket Through Neuroevolution
2002 - 2003
Adaptive Packet Routing: The Confidence-Based Dual Reinforcement Q-Learning Algorithm
1998 - 2000
Realtime Continuous Adaptive Behavior: The Rodney System
1997 - 1997
On-Line Adaptation of a Signal Predistorter through Dual Reinforcement Learning
1995 - 1997
Demos
Multi-modal Approaches to Evolving Behavior for Multi-task Games
Jacob Schrum
2011
Fitness-based Shaping in Multi-objective Domains
Jacob Schrum
2010
Learning in Fractured Domains
Nate Kohl
2009
Multi-modal Behavior in NPCs
Jacob Schrum
2009
Multi-objective Neuroevolution of NPCs
Jacob Schrum
2008
Finless Rocket Control
Faustino Gomez
2003
Neuroevolution of Augmenting Topologies Demos
Kenneth Stanley
2003
Double Pole Balancing with ESP
Faustino Gomez
1999
Publications (90)
Accelerating Evolution via Egalitarian Social Learning
Wesley Tansey, Eliana Feasley, and Risto Miikkulainen
To Appear In
Proceedings of the 14th Annual Genetic and Evolutionary Computation Conference (GECC...
2012
Evolution of a Communication Code in Cooperative Tasks
Aditya Rawal, Padmini Rajagopalan, Risto Miikkulainen and Kay Holekamp
To Appear In
Artificial Life (13th International Conference on the Synthesis and Simulation of Li...
2012
Evolving Multimodal Networks for Multitask Games
Jacob Schrum and Risto Miikkulainen
In
Proceedings of the IEEE Conference on Computational Intelligence and Games (CIG 2011)
, 102...
2011
Human-Assisted Neuroevolution Through Shaping, Advice and Examples
Igor V. Karpov, Vinod K. Valsalam and Risto Miikkulainen
In
Proceedings of the 13th Annual Genetic and Evolutionary Computation Conference (GECCO 2011)
2011
UT^2: Human-like Behavior via Neuroevolution of Combat Behavior and Replay of Human Traces
Jacob Schrum, Igor V. Karpov and Risto Miikkulainen
In
Proceedings of the IEEE Conference on Computational Intelligence and Games (CIG 2011)
, 329...
2011
Evolving Agent Behavior In Multiobjective Domains Using Fitness-Based Shaping
Jacob Schrum and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2010)
, 439--446,...
2010
Neuroevolution
Risto Miikkulainen
In
Encyclopedia of Machine Learning
, New York, 2010. Springer.
2010
Learning in Fractured Problems for Constructive Neural Network Algorithms
Nate Kohl
PhD Thesis, Department of Computer Sciences, University of Texas at Austin, Austin, TX, 2009.
2009
Competition Between Reinforcement Learning Methods in a Predator-Prey Grid World
Jacob Schrum
Technical Report AI08-9, The University of Texas at Austin, Department of Computer Sciences, Novembe...
2008
Online Kernel Selection for Bayesian Reinforcement Learning
Joseph Reisinger and Peter Stone and Risto Miikkulainen
In
Proceedings of the Twenty-Fifth International Conference on Machine Learning
, July 2008.
2008
Acquiring Evolvability through Adaptive Representations
Joseph Reisinger and Risto Miikkulainen
In
Proceeedings of the Genetic and Evolutionary Computation Conference
, 1045-1052, 2007.
2007
Coevolution of Role-Based Cooperation in Multi-Agent Systems
Chern Han Yong and Risto Miikkulainen
Technical Report AI07-338, Department of Computer Sciences, The University of Texas at Austin, 2007.
2007
Reinforcement Learning in High-Diameter, Continuous Environments
Jefferson Provost
PhD Thesis, Computer Sciences Department, University of Texas at Austin, Austin, TX, 2007.
2007
Self-Organizing Distinctive State Abstraction Using Options
Jefferson Provost, Benjamin J. Kuipers, and Risto Miikkulainen
In
Proceedings of the 7th International Conference on Epigenetic Robotics
, 2007.
2007
Developing navigation behavior through self-organizing distinctive state abstraction
Jefferson Provost, Benjamin J. Kuipers, and Risto Miikkulainen
Connection Science
, 18:159-172, 2006.
2006
Integration and Evaluation of Exploration-Based Learning in Games
Igor V. Karpov, Thomas D'Silva, Craig Varrichio, Kenneth O. Stanley, Risto Miikkulainen
In
Proceedings of the {IEEE} Symposium on Computational Intelligence and Games
, Reno, NV, 20...
2006
Academic AI and Video Games: A Case Study of Incorporating Innovative Academic Research into a Video Game Prototype
Aliza Gold
In
Proceedings of the IEEE 2005 Symposium on Computational Intelligence and Games (CIG'05)
, 2...
2005
Constructing Good Learners Using Evolved Pattern Generators
Vinod K. Valsalam, James A. Bednar, and Risto Miikkulainen
In H.-G. Beyer and others, editors,
Proceedings of the Genetic and Evolutionary Computation Confe...
2005
Effective Image Compression Using Evolved Wavelets
Uli Grasemann and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference
, 2005.
2005
Evolving Neural Network Agents in the NERO Video Game
Kenneth O. Stanley, Bobby D. Bryant, and Risto Miikkulainen
In
Proceedings of the IEEE 2005 Symposium on Computational Intelligence and Games (CIG'05)
, P...
2005
Evolving Neural Network Ensembles for Control Problems
David Pardoe, Michael Ryoo, and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference
, 2005.
2005
Incorporating Advice into Evolution of Neural Networks
Chern Han Yong, Kenneth O. Stanley, and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2005)
, 2005. Lat...
2005
Learning Basic Navigation for Personal Satellite Assistant Using Neuroevolution
Yiu Fai Sit and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference
, 2005.
2005
Neuroevolution of an Automobile Crash Warning System
Kenneth Stanley, Nate Kohl, Rini Sherony, and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference
, 2005.
2005
Real-Time Learning in the NERO Video Game
Kenneth O. Stanley, Ryan Cornelius, Risto Miikkulainen, Thomas D'Silva, and Aliza Gold
In
Proceedings of the Artificial Intelligence and Interactive Digital Entertainment Conference (A...
2005
Real-time Neuroevolution in the NERO Video Game
Kenneth O. Stanley, Bobby D. Bryant, and Risto Miikkulainen
IEEE Transactions on Evolutionary Computation
:653-668, 2005. IEEE.
2005
Retaining Learned Behavior During Real-Time Neuroevolution
Thomas D'Silva, Roy Janik, Michael Chrien, Kenneth O. Stanley and Risto Miikkulainen
Artificial Intelligence and Interactive Digital Entertainment
, 2005. American Association for...
2005
Towards an Empirical Measure of Evolvability
Joseph Reisinger, Kenneth O. Stanley, Risto Miikkulainen
In
Genetic and Evolutionary Computation Conference {(GECCO2005)} Workshop Program
, 257-264, W...
2005
Self-Organizing Perceptual and Temporal Abstraction for Robot Reinforcement Learning
Jefferson Provost, Benjamin J. Kuipers and Risto Miikkulainen
In
AAAI-04 Workshop on Learning and Planning in Markov Processes
, 2004.
2004
Competitive Coevolution through Evolutionary Complexification
Kenneth O. Stanley and Risto Miikkulainen
Journal of Artificial Intelligence Research
, 21:63-100, 2004.
2004
Efficient Allele Fitness Assignment with Self-organizing Multi-agent System
Adrian Agogino and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004) Workshop Progr...
2004
Efficient Evolution of Neural Networks Through Complexification
Kenneth O. Stanley
PhD Thesis, Department of Computer Sciences, The University of Texas at Austin, 2004.
2004
Evolving a Roving Eye for Go
Kenneth O. Stanley and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004)
, Berlin, 2...
2004
Evolving Reusable Neural Modules
Joseph Reisinger, Kenneth O. Stanley, and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference
, 2004.
2004
Evolving Wavelets using a Coevolutionary Genetic Algorithm and Lifting
Uli Grasemann and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference
, 969-980, San Francisco...
2004
Exploiting Morphological Conventions for Genetic Reuse
Kenneth O. Stanley, Joseph Reisinger, and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference ({GECCO}-2004) Workshop Pr...
2004
The Constructivist Learning Architecture: A Model of Cognitive Development for Robust Autonomous Robots
Harold H. Chaput
PhD Thesis, Department of Computer Sciences, The University of Texas at Austin, August 2004. Also Te...
2004
Transfer of Neuroevolved Controllers in Unstable Domains
Faustino J. Gomez and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference
, Berlin, 2004. Springer...
2004
A Taxonomy for Artificial Embryogeny
Kenneth O. Stanley and Risto Miikkulainen
Artificial Life
, 9(2):93-130, 2003.
2003
Achieving High-Level Functionality through Evolutionary Complexification
Kenneth O. Stanley and Risto Miikkulainen
In
Proceedings of the AAAI-2003 Spring Symposium on Computational Synthesis
, Stanford, CA, 20...
2003
Active Guidance for a Finless Rocket Using Neuroevolution
Faustino J. Gomez and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference
, 2084-2095, San Francis...
2003
Evolving Adaptive Neural Networks with and Without Adaptive Synapses
Kenneth O. Stanley, Bobby D. Bryant, and Risto Miikkulainen
In
Proceedings of the 2003 Congress on Evolutionary Computation
, Piscataway, NJ, 2003. IEEE.
2003
Neuroevolution for Adaptive Teams
Bobby D. Bryant and Risto Miikkulainen
In
Proceedings of the 2003 Congress on Evolutionary Computation (CEC 2003)
, 2194-2201, Pisca...
2003
Robust Non-Linear Control through Neuroevolution
Faustino J. Gomez
PhD Thesis, Department of Computer Sciences, The University of Texas at Austin, 2003.
2003
Utilizing Domain Knowledge in Neuroevolution
James Fan, Raymond Lau, and Risto Miikkulainen
Proceedings of the Twentieth International Conference on Machine Learning (ICML-03, Was...
2003
Adaptive Control Utilising Neural Swarming
Alex v. E. Conradie, Risto Miikkulainen, and Christiaan Aldrich
In William B. Langdon and Erick Cantu-Paz and Keith E. Mathias and Rajkumar Roy and David Davis and...
2002
Continual Coevolution Through Complexification
Kenneth O. Stanley and Risto Miikkulainen
In William B. Langdon and Erick Cantu-Paz and Keith E. Mathias and Rajkumar Roy and David Davis and ...
2002
Cultural Enhancement Of Neuroevolution
Paul H. McQuesten
PhD Thesis, Department of Computer Sciences, The University of Texas at Austin, Austin, TX, 2002. Te...
2002
Efficient Evolution Of Neural Network Topologies
Kenneth O. Stanley and Risto Miikkulainen
In William B. Langdon and Erick Cantu-Paz and Keith E. Mathias and Rajkumar Roy and David Davis and ...
2002
Efficient Reinforcement Learning Through Evolving Neural Network Topologies
Kenneth O. Stanley and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2002)
, 9, San Fr...
2002
Eugenic Evolution Utilizing A Domain Model
Matthew Alden, Aard-Jan van Kesteren, and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2002)
, 279-286, ...
2002
Evolving Neural Networks Through Augmenting Topologies
Kenneth O. Stanley and Risto Miikkulainen
Evolutionary Computation
, 10(2):99-127, 2002.
2002
Intelligent Process Control Utilizing Symbiotic Memetic Neuro-Evolution
Alex v. E. Conradie, Risto Miikkulainen, and Christiaan Aldrich
In
Proceedings of the 2002 Congress on Evolutionary Computation
, 6, 2002.
2002
Numerical Optimization With Neuroevolution
Brian Greer, Henri Hakonen, Risto Lahdelma, and Risto Miikkulainen
In
Proceedings of the 2002 Congress on Evolutionary Computation
, 361-401, Piscataway, NJ, 20...
2002
The Dominance Tournament Method of Monitoring Progress in Coevolution
Kenneth O. Stanley and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference ({GECCO}-2002) Workshop Pr...
2002
A Neuroevolution Method For Dynamic Resource Allocation On A Chip Multiprocessor
Faustino J. Gomez, Doug Burger, and Risto Miikkulainen
In
Proceedings of the {INNS-IEEE} International Joint Conference on Neural Networks
, 2355-23...
2001
Abrupt And Gradual Sound Change In An Expanding Lexicon
Melissa A. Redford and Risto Miikkulainen
Technical Report AI01-289, Department of Computer Sciences, The University of Texas at Austin, 2001.
2001
Applying ESP And Region Specialists To Neuro-Evolution For Go
Andres Santiago Perez-Bergquist
Technical Report CSTR01-24, Department of Computer Science, University of Texas at Austin, May 2001....
2001
Co-Evolving A Go-Playing Neural Network
Alex Lubberts and Risto Miikkulainen
In
Coevolution: {T}urning Adaptive Algorithms Upon Themselves, Birds-of-a-Feather Workshop, Gene...
2001
Constrained Emergence Of Universals And Variation In Syllable Systems
Melissa A. Redford, Chun Chi Chen, and Risto Miikkulainen
Language and Speech
:27-56, 2001. Manuscript.
2001
Cooperative Coevolution Of Multi-Agent Systems
Chern Han Yong and Risto Miikkulainen
Technical Report AI07-338, Department of Computer Sciences, The University of Texas at Austin, 2001.
2001
Creating Melodies With Evolving Recurrent Neural Networks
Chun-Chi J. Chen and Risto Miikkulainen
In
Proceedings of the {INNS-IEEE} International Joint Conference on Neural Networks
, 2241-22...
2001
Evolving Populations Of Expert Neural Networks
Joseph Bruce and Risto Miikkulainen
In Kristiina Jokinen and Dirk Heylen and Anton Nijholt, editors,
Learning to Behave: Proceedings ...
2001
Eugenic Neuro-Evolution For Reinforcement Learning
Daniel Polani and Risto Miikkulainen
In
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000)
, 1041-1046...
2000
Neuro-Evolution And Natural Deduction
Nirav S. Desai and Risto Miikkulainen
In
Proceedings of The First {IEEE} Symposium on Combinations of Evolutionary Computation and Neu...
2000
Online Interactive Neuro-Evolution
Adrian Agogino, Kenneth O. Stanley, and Risto Miikkulainen
Neural Processing Letters
:29-38, 2000.
2000
Confidence Based Dual Reinforcement Q-Routing: An Adaptive On-Line Routing Algorithm
Shailesh Kumar and Risto Miikkulainen
In
16th International Joint Conference on Artificial Intelligence (IJCAI-99)
, 758--763, Stock...
1999
Solving Non-Markovian Control Tasks With Neuroevolution
Faustino J. Gomez and Risto Miikkulainen
1999. Dissertation Proposal, Computer Science Department, University of Texas at Austin.
1999
2-D Pole Balancing With Recurrent Evolutionary Networks
Faustino Gomez and Risto Miikkulainen
In
Proceedings of the International Conference on Artificial Neural Networks (ICANN-98)
, 425-...
1998
Confidence Based Dual Reinforcement Q-Routing: An On-Line Adaptive Network Routing Algorithm
Shailesh Kumar
Masters Thesis, Department of Computer Sciences, the University of Texas at Austin., May 1998. 108. ...
1998
Confidence Based Q-Routing: An On-Line Adaptive Network Routing Algorithm
Shailesh Kumar and Risto Miikkulainen
C. H. Dagli and M. Akay and O. Ersoy and B. R. Fernandez and A. Smith, editors,
Smart Engineering...
1998
Eugenic Evolution For Combinatorial Optimization
John W. Prior
Masters Thesis, Department of Computer Sciences, The University of Texas at Austin, 1998. 126. Techn...
1998
Evolving Neural Networks To Play Go
Norman Richards, David Moriarty, and Risto Miikkulainen
In Thomas B{"a}ck, editors,
Proceedings of the Seventh International Conference on Genetic Algori...
1998
Hierarchical Evolution Of Neural Networks
David E. Moriarty and Risto Miikkulainen
In
Proceedings of the 1998 IEEE Conference on Evolutionary Computation (ICEC98)
, 428-433, Anc...
1998
Modeling The Emergence Of Syllable Systems
Melissa A. Redford, Chun Chi Chen, and Risto Miikkulainen
In Morton Ann Gernsbacher and Sharon J. Derry, editors,
Proceedings of the 20th Annual Conference...
1998
Culling And Teaching In Neuro-Evolution
Paul McQuesten and Risto Miikkulainen
In Thomas B{"a}ck, editors,
Proceedings of the Seventh International Conference on Genetic Algori...
1997
Dual Reinforcement Q-Routing: An On-Line Adaptive Routing Algorithm
Shailesh Kumar and Risto Miikkulainen
C. H. Dagli, M. Akay, O. Ersoy, B. R. Fernandez and A. Smith, editors,
Smart Engineering Systems:...
1997
Forming Neural Networks Through Efficient And Adaptive Coevolution
David E. Moriarty and Risto Miikkulainen
Evolutionary Computation
, 5:373--399, 1997.
1997
Incremental Evolution Of Complex General Behavior
Faustino Gomez and Risto Miikkulainen
Adaptive Behavior
(5):317-342, 1997.
1997
Symbiotic Evolution Of Neural Networks In Sequential Decision Tasks
David E. Moriarty
PhD Thesis, Department of Computer Sciences, The University of Texas at Austin, 1997. 117. Technical...
1997
Efficient Reinforcement Learning Through Symbiotic Evolution
David E. Moriarty and Risto Miikkulainen
In Leslie Pack Kaelbling, editors,
Recent Advances in Reinforcement Learning
(AI94-224), 11-32...
1996
Evolving Obstacle Avoidance Behavior In A Robot Arm
David E. Moriarty and Risto Miikkulainen
In Pattie Maes and Maja J. Mataric and Jean-Arcady Meyer and Jordan Pollack and Stewart W. Wilson, ...
1996
On-Line Adaptation Of A Signal Predistorter Through Dual Reinforcement Learning
Patrick Goetz, Shailesh Kumar and Risto Miikkulainen
In Lorenza Saitta, editors,
Machine Learning: Proceedings of the 13th Annual Conference (Bari, It...
1996
Discovering Complex Othello Strategies Through Evolutionary Neural Networks
David E. Moriarty and Risto Miikkulainen
Connection Science
, 7:195--209, 1995.
1995
Learning Sequential Decision Tasks
David E. Moriarty and Risto Miikkulainen
Technical Report AI95-229, Department of Computer Sciences, The University of Texas at Austin, 1995.
1995
Evolutionary Neural Networks For Value Ordering In Constraint Satisfaction Problems
David E. Moriarty and Risto Miikkulainen
Technical Report AI94-218, Department of Computer Sciences, The University of Texas at Austin, 1994.
1994
Evolving Neural Networks To Focus Minimax Search
David E. Moriarty and Risto Miikkulainen
In
Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI-94)
, 1371-...
1994
Grounding Robotic Control With Genetic Neural Networks
Diane Law and Risto Miikkulainen
Technical Report AI93-223, Department of Computer Sciences, The University of Texas at Austin, 1994.
1994
Searle, Subsymbolic Functionalism And Synthetic Intelligence
Diane Law
Technical Report, Department of Computer Sciences, The University of Texas at Austin, Austin, TX, 19...
1994
Using Marker-Based Genetic Encoding Of Neural Networks To Evolve Finite-State Behaviour
Brad Fullmer and Risto Miikkulainen
In Francisco J. Varela and Paul Bourgine, editors,
Toward a Practice of Autonomous Systems: {P}ro...
1991
Software/Data
OpenNERO
OpenNERO is a general research and education platform for artificial intelligence. The platform is based on a simulatio...
2010