Novelty and Beyond: Towards Combined Motivation Models and Integrated Learning Architectures
For future intrinsically motivated agents to combine multiple intrinsic motivation or behavioural components, there is a need to identify fundamental units of motivation models that can be reused and combined to produce more complex agents. This chapter r
- PDF / 722,488 Bytes
- 25 Pages / 439.36 x 666.15 pts Page_size
- 72 Downloads / 190 Views
Abstract For future intrinsically motivated agents to combine multiple intrinsic motivation or behavioural components, there is a need to identify fundamental units of motivation models that can be reused and combined to produce more complex agents. This chapter reviews three existing models of intrinsic motivation, novelty, interest and competence-seeking motivation, that are based on the neural network framework of a real-time novelty detector. Four architectures are discussed that combine basic units of the intrinsic motivation functions in different ways. This chapter concludes with a discussion of future directions for combined motivation models and integrated learning architectures.
1 Introduction The study of motivation in natural systems has a long history, including work by Aristotle, Jean Piaget and Sigmund Freud. Over the years, a broad spectrum of different motivation theories has been proposed for natural systems (Heckhausen and Heckhausen 2008). A subset of motivation theories that form the focus of this book are intrinsic motivations (Baldassarre 2011; Deci and Ryan 1985; Kaplan and Oudeyer 2007; Oudeyer and Kaplan 2007). Intrinsic motivations fall in the category of cognitive motivation theories, which includes theories of the mind that tend to be abstracted from the biological system of the behaving organism. Examples include novelty-seeking behaviour and curiosity (Berlyne 1960),
K.E. Merrick () School of Engineering and Information Technology, University of New South Wales, Australian Defence Force Academy, Northcott Drive Canberra, ACT Australia e-mail: [email protected] G. Baldassarre and M. Mirolli (eds.), Intrinsically Motivated Learning in Natural and Artificial Systems, DOI 10.1007/978-3-642-32375-1 9, © Springer-Verlag Berlin Heidelberg 2013
209
210
K.E. Merrick
competence-seeking motivation (White 1959), achievement, affiliation and power motivation (Heckhausen and Heckhausen 2008). Artificial intelligence researchers seek to achieve a scientific understanding of the mechanisms underlying thought and intelligent behaviour in order to embody them in machines. This embodiment is achieved through abstract computational structures such as states, goals, and actions that form the fundamental units of computational models of cognition and motivation. Various different kinds of artificial models have been proposed for intrinsic motivation. Oudeyer and Kaplan (2007) provide a typology of different approaches and mechanisms from psychological and computational perspectives. Likewise, this book considers the broad classes of prediction-based (Schmidhuber 1991, 2010) models in Part II, novelty-based models (Marsland et al. 2000; Merrick and Maher 2009; Saunders 2001) in Part III and competence-based models (Barto et al. 2004; Schembri et al. 2007) in Part IV. Alternatively, the functions and mechanisms of intrinsic motivation can also be distinguished according to knowledge-based (including prediction and noveltybased) views and competence-based views as in Mirolli and Baldassarre (2012)
Data Loading...
