ERIC Number: EJ825100
Record Type: Journal
Publication Date: 2008
Abstractor: As Provided
Reference Count: 30
Randomised Items in Computer-Based Tests: Russian Roulette in Assessment?
Marks, Anthony M.; Cronje, Johannes C.
Educational Technology & Society, v11 n4 p41-50 2008
Computer-based assessments are becoming more commonplace, perhaps as a necessity for faculty to cope with large class sizes. These tests often occur in large computer testing venues in which test security may be compromised. In an attempt to limit the likelihood of cheating in such venues, randomised presentation of items is automatically programmed into testing software, such that neighbouring screens present different items to the test-taker. This article argues that randomisation of test items can be a disadvantage to students who were randomly presented with difficult items first. Such disadvantage would violate the American Psychological Association's published guidelines concerning testing and assessment that call for the principle of fairness for test-takers across diverse test modes. Owing to the smallness of the chance of a student being randomly assigned difficult items first, it may be hard to prove such disadvantage. However, even if only one test-taker is affected once during a high-stakes test, the principle of fairness is compromised. This article reports on four instances out of about 400 in which students may either have been unfairly advantaged or disadvantaged by being given a series of easy or difficult items at the beginning of the test. Although the results are not statistically significant, we conclude that more research needs to be done before one can ignore what we have named the Item Randomisation Effect. (Contains 1 table and 4 figures.)
Descriptors: Educational Assessment, Educational Testing, Research Needs, Test Items, Computer Assisted Testing, Computer Software, Educational Technology, Cheating, Testing Problems, Test Bias, Test Construction, Multiple Choice Tests, Foreign Countries, Evaluation Methods, Student Evaluation, Test Anxiety, College Students, Veterinary Medical Education, Evaluation Research
International Forum of Educational Technology & Society. Athabasca University, School of Computing & Information Systems, 1 University Drive, Athabasca, AB T9S 3A3, Canada. Tel: 780-675-6812; Fax: 780-675-6973; Web site: http://www.ifets.info
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A
Identifiers - Location: South Africa