NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1015697
Record Type: Journal
Publication Date: 2013
Pages: 12
Abstractor: As Provided
ISSN: ISSN-1554-9178
Assessing Class-Wide Consistency and Randomness in Responses to True or False Questions Administered Online
Pawl, Andrew; Teodorescu, Raluca E.; Peterson, Joseph D.
Physical Review Special Topics - Physics Education Research, v9 n2 p020102-1-020102-12 Jul-Dec 2013
We have developed simple data-mining algorithms to assess the consistency and the randomness of student responses to problems consisting of multiple true or false statements. In this paper we describe the algorithms and use them to analyze data from introductory physics courses. We investigate statements that emerge as outliers because the class has a preference for the incorrect answer and also those that emerge as outliers because the students are randomly changing their responses. These outliers are found to include several statements that are known in the literature to expose student misconceptions. Combining this fact with comments made by students and results of complementary assessments provides evidence that the tendency of a group of students to change their answer to a true or false statement or to remain consistent can serve as indicators of whether the class has understood the relevant concept. Our algorithms enable teachers to employ problems of the type described as a tool to identify specific aspects of a course that require improvement. They also enable researchers to employ such problems in experiments designed to probe aspects of students' thought processes and behavior. Additionally, our results demonstrate that at least one category of research-inspired problems (ranking tasks) can be adapted to the linked true or false format and productively used as an assessment tool in an online setting. (Contains 1 table and 13 figures.)
American Physical Society. One Physics Ellipse 4th Floor, College Park, MD 20740-3844. Tel: 301-209-3200; Fax: 301-209-0865; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Massachusetts; Wisconsin
Grant or Contract Numbers: N/A