NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ842306
Record Type: Journal
Publication Date: 2009
Pages: 10
Abstractor: As Provided
Reference Count: 74
ISSN: ISSN-0731-1745
Generalizability of Cognitive Interview-Based Measures across Cultural Groups
Solano-Flores, Guillermo; Li, Min
Educational Measurement: Issues and Practice, v28 n2 p9-18 Sum 2009
We addressed the challenge of scoring cognitive interviews in research involving multiple cultural groups. We interviewed 123 fourth- and fifth-grade students from three cultural groups to probe how they related a mathematics item to their personal lives. Item meaningfulness--the tendency of students to relate the content and/or context of an item to activities in which they are actors--was scored from interview transcriptions with a procedure similar to the scoring of constructed-response tasks. Generalizability theory analyses revealed a small amount of score variation due to the main and interaction effect of rater but a sizeable magnitude of measurement error due to the interaction of person and question (context). Students from different groups tended to draw on different sets of contexts of their personal lives to make sense of the item. In spite of individual and potential cultural communication style differences, cognitive interviews can be reliably scored by well-trained raters with the same kind of rigor used in the scoring of constructed-response tasks. However, to make valid generalizations of cognitive interview-based measures, a considerable number of interview questions may be needed. Information obtained with cognitive interviews for a given cultural group may not be generalizable to other groups.
Blackwell Publishing. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8599; Fax: 781-388-8232; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Education; Elementary Secondary Education; Grade 4; Grade 5
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A