This paper provides an example of the application of the cognitive interview, a qualitative tool for pre-testing a survey instrument to check its cognitive validity, that is, whether the items mean to respondents what they mean to the item designers. The instrument is the survey used in the final phase of the International Study of Principal Preparation (ISPP http://www.ucalgary.ca/~cwebber/ISPP/index.htm). This study involving researchers in 13 countries investigates those aspects of principals' work perceived by them to be most challenging in their early years in the position and the extent to which principals believe they were prepared for these challenges. Both probing and think aloud approaches to the cognitive interview were used and revealed a small number of items requiring amendment.