The Role of the Basal Ganglia in Discovering Novel Actions
Our interest is in the neural circuitry which supports the discovery and encoding of novel actions. We discuss the significant existing literature which identifies the basal ganglia, a complex of subcortical nuclei, as important in both the selection of a
- PDF / 481,571 Bytes
- 22 Pages / 439.36 x 666.15 pts Page_size
- 92 Downloads / 201 Views
Abstract Our interest is in the neural circuitry which supports the discovery and encoding of novel actions. We discuss the significant existing literature which identifies the basal ganglia, a complex of subcortical nuclei, as important in both the selection of actions and in reinforcement learning. We discuss the complementarity of these problems of action selection and action learning. Two basic mechanisms of biasing action selection are identified: (a) adjusting the relative strengths of competing inputs and (b) adjusting the relative sensitivity of the receiver of reinforced inputs. We discuss the particular importance of the phasic dopamine signal in the basal ganglia and its proposed role in conveying a reward prediction error. Temporal constraints of this signal limit the information it can convey to immediately surprising sensory events, thus—we argue—making it inappropriate to convey information regarding the economic value of actions (as proposed by the reward prediction error hypothesis). Rather, we suggest this signal is ideal to support the identification of novel actions and their encoding via the biasing of future action selection.
1 Introduction This chapter will consider the brain mechanisms that support the discovery of novel actions. Action acquisition is viewed as a fundamental competence on which much of the intrinsically motivated cumulative learning by robots will rest. Given that most animals are born without detailed instructions of how to behave in any given circumstance, yet in comparatively short times develop numerous actions P. Redgrave () K. Gurney T. Stafford M. Thirkettle J. Lewis Adaptive Behaviour Research Group, Department of Psychology, University of Sheffield, Sheffield, UK e-mail: [email protected]; [email protected]; [email protected]; [email protected]; [email protected] G. Baldassarre and M. Mirolli (eds.), Intrinsically Motivated Learning in Natural and Artificial Systems, DOI 10.1007/978-3-642-32375-1 6, © Springer-Verlag Berlin Heidelberg 2013
129
130
P. Redgrave et al.
and patterns of adaptive behaviour, our approach is to learn as much as we can from action development in biological systems. In the contemporary neuroscience literature, a recurring theme is that one of the brain’s fundamental processing units, the basal ganglia, is associated with action selection (Grillner et al. 2005; Hikosaka 2007; Humphries et al. 2006; Mink 1996; Prescott et al. 2006; Redgrave 2007; Redgrave et al. 1999b) and reinforcement learning (Berridge 2007; Houk 2005; Salamone and Correa 2002; Schultz 1998, 2006; Wickens 1993; Wickens et al. 2003; Wise 2004). It appears likely that it is within the circuits of the basal ganglia and associated structures that behavioural output is related to outcome. This document will therefore (a) identify selection as a fundamental problem faced by all multifunctional agents, (b) indicate how the architecture of the basal ganglia can provide a generic solution to a variety of selection problems, (
Data Loading...
