I study learning in robots and other situated, embodied agents. This interest includes both scaling up existing learning algorithms to the rich, high-dimensional, continuous sensorimotor systems of such agents, and integrating learning and adaptation into their general decision-making processes. My research seeks to understand how an agent with a rich, realistic sensorimotor system can learn to perceive, act, and achieve a broad range of goals, with minimal prior knowledge of itself and its world. My dissertation presents Self-Organizing Distinctive-state Abstraction (SODA), a generic method by which a robot in a continuous world can learn a set of high-level perceptual features and temporally-extended actions for navigating in large environments. Using SODA an agent has learned to perform a navigation task requiring hundreds small-scale, local actions using as few as nine new, temporally-extended actions, significantly improving learning time over navigating with local actions. My dissertation advisors are Ben Kuipers and Risto Miikkulainen.