Computational Neural Mechanisms of Goal-Directed Planning and Problem Solving
- PDF / 3,404,280 Bytes
- 22 Pages / 595.276 x 790.866 pts Page_size
- 72 Downloads / 221 Views
ORIGINAL PAPER
Computational Neural Mechanisms of Goal-Directed Planning and Problem Solving Justin M. Fine 1 & Noah Zarr 1 & Joshua W. Brown 1 Accepted: 2 November 2020 / Published online: 13 November 2020 # Society for Mathematical Psychology 2020
Abstract The question of how animals and humans can solve arbitrary goal-driven problems remains open. Reinforcement learning (RL) methods have approached goal-directed control problems through model-based algorithms. However, RL focus on maximizing long-term reward is inconsistent with the psychological notion of planning to satisfy homeostatic drives, which involves setting goals first, then planning actions to achieve them. Optimal control theory suggests a solution: animals can learn a model of the world, learn where goals can be fulfilled, set a goal, and then act to minimize the difference between actual and desired world states. Here, we present a purely localist neural network model that can autonomously learn the structure of an environment and then achieve any arbitrary goal state in a changing environment without relearning reward values. The model, GOLSA, achieves this through a backwards spreading activation that propagates goal-values to an agent. The model elucidates how neural inhibitory mechanisms can support competition between goal representations, serving to push needs-based planning versus exploration. The model performs similar to humans in canonical revaluation tasks used to classify human and rodent behavior as goal-directed. The model revaluates optimal actions when goals, goal-values, world structure, and need to fulfill drive changes. The model also clarifies a number of issues inherent in other RL-based representations such as policy dependence in successor representations, while elucidating biological constraints such as the role of oscillations in gating information flow for learning versus action. Together, our proposed model suggests a biologically grounded framework for multi-step planning behaviors through consideration of how goal representations compete for behavioral expression in planning. Keywords Neural network . Goal-directed decision making . Planning . Model-based . Cognitive control . Systems neuroscience
Introduction Actions derived from habits and reflexes offer a surprising amount of power for producing successful behavior, and they require minimal cognitive control for production. This reduced complexity, however, renders them inflexible because they are driven predominantly in a stimulus reactive mode within an environment. Such inflexibility is insufficient for supporting all behaviors, particularly when the cached actions will not lead to successful goal-driven outcomes (Dolan and Dayan 2013). The capacity for goal-oriented and flexible control behavior is abundantly recognized across species and thought to involve learning an internal model of the * Joshua W. Brown [email protected] 1
Dept. of Psychological and Brain Sciences, Indiana University, 1101 E Tenth St, Bloomington, IN 47405, USA
environment (Dayan and
Data Loading...
