Developing a cognitive model of task performance is an important and often overlooked phase in assessment design; failing to establish such a model can threaten the validity of the inferences made from the scores produced by an assessment (e.g., Leighton, 2004). Conducting think aloud interviews (TAIs), where students think aloud while completing and interacting with a task, is an ideal approach for developing a cognitive model of task performance. The objective of this research was to pilot the use of think aloud interviews (TAIs) to inform item development and strengthen the validity argument for a new AP World History exam that is currently being developed. The results indicate that we are seeing evidence in students' verbal reports of intended historical thinking skills. Also, there are several different features (e.g., degree of familiarity) that contribute to the perceived difficulty of these items. [Slides] presented at the Annual Meeting of the American Educational Research Association (AERA) and National Council on Measurement in Education (NCME) in New Orleans, LA in April 2011.