NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1129759
Record Type: Journal
Publication Date: 2017
Pages: 6
Abstractor: As Provided
ISSN: ISSN-1543-4303
A Response to Holster and Lake Regarding Guessing and the Rasch Model
Stewart, Jeffrey; McLean, Stuart; Kramer, Brandon
Language Assessment Quarterly, v14 n1 p69-74 2017
Stewart questioned vocabulary size estimation methods proposed by Beglar and Nation for the Vocabulary Size Test, further arguing Rasch mean square (MSQ) fit statistics cannot determine the proportion of random guesses contained in the average learner's raw score, because the average value will be near 1 by design. He illustrated this by demonstrating this is true even of entirely random data. Holster and Lake appear to misinterpret this as a claim that Rasch analyses cannot distinguish random data from real responses. To test this, they compare real data to random and note that, predictably, the statistic easily distinguishes the two and that reliability for random data is near zero. However, while certainly true, this fact is not relevant to Stewart's argument that multiple-choice options inflate the test's size estimates and that MSQ fit statistics cannot be used to detect this. We further illustrate this by showing real data retains average MSQ values near 1, even when unknown items skipped by learners are imputed with random guesses. Furthermore, the imputed data do not exhibit "problematic guessing" under Holster & Lake's own criteria, despite size inflation under Beglar and Nation's suggested scoring. We conclude by discussing uses of the 3PL model.
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A