The behavior of a realistic robotic agent takes place in high-dimensional continuous sensory, state, and motor spaces. Autonomous learning of effective behaviors requires autonomous learning of useful abstractions for these spaces. The concept of distinctive state from the topological mapping literature can be used to learn such actions from the agent's own experience, without prior knowledge provided by an external designer. In the approach taken in this paper, Self-Organizing Distinctive-state Abstraction (SODA), a variant of self-organizing maps defines a finite set of distinctive sensory prototypes; distinctive states are then defined as local maxima of the activation function for the leading prototype. Hierarchical reinforcement learning is then used to learn options that move the agent among distinctive states with increasing reliability. This state-action abstraction is learned autonomously, and reflects only the environment and the agent's sensorimotor capabilities, without external direction. Using SODA, a robot can learn to navigate in large environments that are intractable to learn in using primitive motor commands.

PDF

In Proceedings of the 7th International Conference on Epigenetic Robotics, 2007.

@inproceedings{provost:epigenetic07, title={Self-Organizing Distinctive State Abstraction Using Options}, author={Jefferson Provost and Benjamin J. Kuipers and Risto Miikkulainen}, booktitle={Proceedings of the 7th International Conference on Epigenetic Robotics}, url="http://nn.cs.utexas.edu/?provost:epigenetic07", year={2007} }