NotesFAQContact Us
Search Tips
ERIC Number: ED545460
Record Type: Non-Journal
Publication Date: 2013-Apr
Pages: 34
Abstractor: As Provided
Reference Count: 17
An Application of Reverse Engineering to Automatic Item Generation: A Proof of Concept Using Automatically Generated Figures
LoriƩ, William A.
Online Submission
A reverse engineering approach to automatic item generation (AIG) was applied to a figure-based publicly released test item from the Organisation for Economic Cooperation and Development (OECD) Programme for International Student Assessment (PISA) mathematical literacy cognitive instrument as part of a proof of concept. The author created an item template from which three items were randomly generated from within each of six types defined by a feature deemed to be most likely to affect item difficulty, for a total of eighteen distinct items. To assess their equivalence, these items were embedded in otherwise identical test forms and administered to human intelligence task workers on the Amazon Mechanical Turk system. One level of the type-defining feature appeared to affect item difficulty systematically. The author provides a task requirement rationale for removing this level. Implications for AIG theory and practice are discussed. An appendix presents a sample HTT.
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Program for International Student Assessment