NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1092427
Record Type: Journal
Publication Date: 2016-Feb
Pages: 14
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-1527-9316
EISSN: N/A
Validity and Reliability of Scores Obtained on Multiple-Choice Questions: Why Functioning Distractors Matter
Ali, Syed Haris; Carr, Patrick A.; Ruit, Kenneth G.
Journal of the Scholarship of Teaching and Learning, v16 n1 p1-14 Feb 2016
Plausible distractors are important for accurate measurement of knowledge via multiple-choice questions (MCQs). This study demonstrates the impact of higher distractor functioning on validity and reliability of scores obtained on MCQs. Freeresponse (FR) and MCQ versions of a neurohistology practice exam were given to four cohorts of Year 1 medical students. Consistently non-functioning multiple-choice distractors (<5% selection frequency) were replaced with those developed from incorrect responses on FR version of the items, followed by administration of the revised MCQ version to subsequent two cohorts. Validity was assessed by comparing an index of expected MCQ difficulty with an index of observed MCQ difficulty, while reliability was assessed via Cronbach's alpha coefficient before and after replacement of consistently non-functioning distractors. Pre-intervention, effect size (Cohen's d) of the difference between mean expected and observed MCQ difficulty indices was noted to be 0.4-0.59. Post-intervention, this difference reduced to 0.15 along with an increase in Cronbach's alpha coefficient of scores obtained on MCQ version of the exam. Through this study, we showed that multiple-choice distractors developed from incorrect responses on free-response version of the items enhance the validity and reliability of scores obtained on MCQs.
Indiana University. 755 West Michigan Street UL 1180D, Indianapolis, IN 46202. Tel: 317-274-5647; Fax: 317-278-2360; e-mail: josotl@iupui.edu; Web site: http://www.iupui.edu/~josotl
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: North Dakota
Grant or Contract Numbers: N/A