ERIC Number: EJ1104820
Record Type: Journal
Publication Date: 2016-Jul
Pages: 17
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1072-0502
EISSN: N/A
Active Inference, Epistemic Value, and Vicarious Trial and Error
Pezzulo, Giovanni; Cartoni, Emilio; Rigoli, Francesco; io-Lopez, Léo; Friston, Karl
Learning & Memory, v23 n7 p322-338 Jul 2016
Balancing habitual and deliberate forms of choice entails a comparison of their respective merits--the former being faster but inflexible, and the latter slower but more versatile. Here, we show that arbitration between these two forms of control can be derived from first principles within an Active Inference scheme. We illustrate our arguments with simulations that reproduce rodent spatial decisions in T-mazes. In this context, deliberation has been associated with vicarious trial and error (VTE) behavior (i.e., the fact that rodents sometimes stop at decision points as if deliberating between choice alternatives), whose neurophysiological correlates are "forward sweeps" of hippocampal place cells in the arms of the maze under consideration. Crucially, forward sweeps arise early in learning and disappear shortly after, marking a transition from deliberative to habitual choice. Our simulations show that this transition emerges as the optimal solution to the trade-off between policies that maximize reward or extrinsic value (habitual policies) and those that also consider the epistemic value of exploratory behavior (deliberative or epistemic policies)--the latter requiring VTE and the retrieval of episodic information via forward sweeps. We thus offer a novel perspective on the optimality principles that engender forward sweeps and VTE, and on their role on deliberate choice.
Descriptors: Interference (Learning), Epistemology, Physiology, Neurology, Brain Hemisphere Functions, Simulation, Correlation, Decision Making, Habituation, Behavior Patterns, Role, Memory, Bayesian Statistics, Animals, Rewards, Task Analysis, Models
Cold Spring Harbor Laboratory Press. 500 Sunnyside Boulevard, Woodbury, NY 11797-2924. Tel: 800-843-4388; Tel: 516-367-8800; Fax: 516-422-4097; e-mail: cshpres@cshl.edu; Web site: http://learnmem.cshlp.org
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A