NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ866997
Record Type: Journal
Publication Date: 2009
Pages: 20
Abstractor: As Provided
Reference Count: 46
ISSN: ISSN-1543-4303
Q-Matrix Construction: Defining the Link between Constructs and Test Items in Large-Scale Reading and Listening Comprehension Assessments
Sawaki, Yasuyo; Kim, Hae-Jin; Gentile, Claudia
Language Assessment Quarterly, v6 n3 p190-209 2009
In cognitive diagnosis a Q-matrix (Tatsuoka, 1983, 1990), which is an incidence matrix that defines the relationships between test items and constructs of interest, has great impact on the nature of performance feedback that can be provided to score users. The purpose of the present study was to identify meaningful skill coding categories that reflect core language skills and processes assessed in the Reading and Listening sections of the Test of English as a Foreign Language[TM] Internet-based Test (TOEFL[R] iBT). The study was conducted as part of a research activity to explore the possibility of developing a detailed score report for low-stakes use by taking a cognitive diagnosis approach. Content experts conducted a test content analysis to develop draft Q-matrices, while measurement experts empirically analyzed the draft Q-matrices with examinee performance data using a cognitively diagnostic psychometric model called the fusion model (DiBello, Stout, & Roussos, 1995; Hartz, 2002). The draft Q-matrices were refined by repeating fusion model analysis and revision of skill definitions and item coding. This resulted in a set of Q-matrices that represented substantively meaningful score reporting categories of a suitable grain size for score reporting, while maintaining an acceptable level of examinee classification consistency. (Contains 7 footnotes, 4 tables, and 1 figure.)
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Test of English as a Foreign Language