ERIC Number: EJ1141408
Record Type: Journal
Publication Date: 2017-Jun
Abstractor: As Provided
How Question Types Reveal Student Thinking: An Experimental Comparison of Multiple-True-False and Free-Response Formats
Hubbard, Joanna K.; Potts, Macy A.; Couch, Brian A.
CBE - Life Sciences Education, v16 n2 Article 26 Jun 2017
Assessments represent an important component of undergraduate courses because they affect how students interact with course content and gauge student achievement of course objectives. To make decisions on assessment design, instructors must understand the affordances and limitations of available question formats. Here, we use a crossover experimental design to identify differences in how multiple-true-false (MTF) and free-response (FR) exam questions reveal student thinking regarding specific conceptions. We report that correct response rates correlate across the two formats but that a higher percentage of students provide correct responses for MTF questions. We find that MTF questions reveal a high prevalence of students with mixed (correct and incorrect) conceptions, while FR questions reveal a high prevalence of students with partial (correct and unclear) conceptions. These results suggest that MTF question prompts can direct students to address specific conceptions but obscure nuances in student thinking and may overestimate the frequency of particular conceptions. Conversely, FR questions provide a more authentic portrait of student thinking but may face limitations in their ability to diagnose specific, particularly incorrect, conceptions. We further discuss an intrinsic tension between question structure and diagnostic capacity and how instructors might use multiple formats or hybrid formats to overcome these obstacles.
Descriptors: Test Format, Questioning Techniques, Undergraduate Students, Objective Tests, Testing, Performance Based Assessment, Statistical Analysis, Error Patterns, Correlation, Predictor Variables, Predictive Measurement, Comparative Analysis, Scores, Achievement Rating
American Society for Cell Biology. 8120 Woodmont Avenue Suite 750, Bethesda, MD 20814-2762. Tel: 301-347-9300; Fax: 301-347-9310; e-mail: email@example.com; Website: http://www.ascb.org
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A
Identifiers - Location: Nebraska
Grant or Contract Numbers: N/A