NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1115929
Record Type: Journal
Publication Date: 2009-Dec
Pages: 21
Abstractor: As Provided
ISSN: EISSN-1932-6246
Modeling Human Performance in Restless Bandits with Particle Filters
Yi, Sheng Kung M.; Steyvers, Mark; Lee, Michael
Journal of Problem Solving, v2 n2 Article 5 p81-101 Dec 2009
Bandit problems provide an interesting and widely-used setting for the study of sequential decision-making. In their most basic form, bandit problems require people to choose repeatedly between a small number of alternatives, each of which has an unknown rate of providing reward. We investigate restless bandit problems, where the distributions of reward rates for the alternatives change over time. This dynamic environment encourages the decision-maker to cycle between states of exploration and exploitation. In one environment we consider, the changes occur at discrete, but hidden, time points. In a second environment, changes occur gradually across time. Decision data were collected from people in each environment. Individuals varied substantially in overall performance and the degree to which they switched between alternatives. We modeled human performance in the restless bandit tasks with two particle filter models: one that can approximate the optimal solution to a discrete restless bandit problem, and another simpler particle filter that is more psychologically plausible. It was found that the simple particle filter was able to account for most of the individual differences. (This work was funded by award FA95500710082 from the Air Force Office of Scientific Research.)
Purdue University Press. Stewart Center Room 370, 504 West State Street, West Lafayette, IN 47907. Tel: 800-247-6553; Fax: 419-281-6883; e-mail: pupress@purdue,edu; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: US Air Force (DOD), Office of Scientific Research (AFOSR)
Authoring Institution: N/A
Identifiers - Location: California (Irvine)
Grant or Contract Numbers: FA95500710082