NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1109877
Record Type: Journal
Publication Date: 2013-Oct
Pages: 106
Abstractor: As Provided
ISSN: EISSN-2330-8516
Cognitive Interviews as a Tool for Investigating the Validity of Content Knowledge for Teaching Assessments. Research Report. ETS RR-13-19
Howell, Heather; Phelps, Geoffrey; Croft, Andrew J.; Kirui, David; Gitomer, Drew
ETS Research Report Series, Oct 2013
This report provides a description of a cognitive interview study investigating validity of assessments designed to measure "content knowledge for teaching" (CKT). The report is intended both to provide information on the validity of the CKT measures and to provide guidance to researchers interested in replicating the design. The study takes an argument-based approach to investigating validity by first articulating interpretive arguments that are central to the CKT measurement theory and then using the cognitive interview data to evaluate these arguments (Kane, 2006). The study is based on 30 interviews of elementary mathematics teachers and 30 interviews of elementary English language arts teachers. Teachers were selected using previous CKT assessment scores to represent high- and low-scoring groups for each subject. The cognitive interviews were conducted separately for each subject and responses were coded and then analyzed to investigate the scoring and extrapolation inferences for the validity argument. Findings strongly support the scoring inference, providing evidence that the item keying for the items is correct. Results also indicate that the participants reasoned about the item in ways that conformed with the reasoning outlined in the "task design rationales" (TDR) for each item. These TDRs represent what reasoning should look like for each of these items for a respondent drawing on the desired CKT knowledge. As such, conformity with the TDRs supports the extrapolation inference, providing evidence that the reasoning used by the participant represents the underlying knowledge and skill domain we intend to measure through CKT assessments. The study design, instruments, methods, and results are described in detail, with discussion included to support researchers interested in replicating or capitalizing on the study design.
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research; Tests/Questionnaires
Education Level: Elementary Education; Grade 4; Intermediate Grades; Grade 5; Middle Schools; Grade 6
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A