Intrinsically Motivated Affordance Discovery and Modeling
In this chapter, we argue that a single intrinsic motivation function for affordance discovery can guide long-term learning in robot systems. To these ends, we provide a novel definition of “affordance” as the latent potential for the closed-loop control
- PDF / 673,482 Bytes
- 22 Pages / 439.36 x 666.15 pts Page_size
- 31 Downloads / 202 Views
Abstract In this chapter, we argue that a single intrinsic motivation function for affordance discovery can guide long-term learning in robot systems. To these ends, we provide a novel definition of “affordance” as the latent potential for the closed-loop control of environmental stimuli perceived by sensors. Specifically, the proposed intrinsic motivation function rewards the discovery of such control affordances. We will demonstrate how this function has been used by a humanoid robot to learn a number of general purpose control skills that address many different tasks. These skills, for example, include strategies for finding, grasping, and placing simple objects. We further show how this same intrinsic reward function is used to direct the robot to build stable models of when the environment affords these skills.
1 Introduction Computational approaches for accumulating long-term control knowledge have eluded psychologists and roboticists since the beginning of research in artificial intelligence. In order for an agent to learn over a lifetime, it must figure out how to take actions that have some measurable effect on the world and be motivated to exercise such actions to understand their consequences. In this chapter, we examine a single fixed intrinsic reward function that addresses these issues in a unified framework.
S. Hart () Manufacturing Systems Research Laboratory, General Motors R&D, 30500 Mound Road, Warren, MI 48090, USA e-mail: [email protected] R. Grupen Department of Computer Science, University of Massachusetts Amherst, 140 Governors Drive, Amherst, MA 01003, USA e-mail: [email protected] G. Baldassarre and M. Mirolli (eds.), Intrinsically Motivated Learning in Natural and Artificial Systems, DOI 10.1007/978-3-642-32375-1 12, © Springer-Verlag Berlin Heidelberg 2013
279
280
S. Hart and R. Grupen
Intrinsically motivated learning is an area of research that has recently provided new tools for autonomous systems to learn over the long term. It alleviates the intractability of more traditional adaptive control learning approaches in which human programmers are required to supply specific (extrinsic) reward criteria for every task. However, much of the existing work in the intrinsic motivation literature has focused on how an agent can explore its environment to gather as much information as possible (cf., Oudeyer et al. 2007). In contrast, we propose a framework in which the robot is intrinsically motivated to control interactions with the environment—learning new control programs and learning how to predict their outcomes in a large variety of run-time situations. The intrinsic motivator we propose was previously introduced in work by the authors (Hart 2009b; Hart and Grupen 2011; Hart et al. 2008a,b) in which we demonstrated how to use this motivator to learn generalizable control strategies for uncovering environmental affordances. In this chapter, we will review this work and extend it by showing how the same motivator can create probabilistic models concerning the conditions in which
Data Loading...
