ERIC Number: EJ907201
Record Type: Journal
Publication Date: 2010
Reference Count: 10
Validate High Stakes Inferences by Designing Good Experiments, Not Audit Items: A Comment on "Self-Monitoring Assessments Educational Accountability Systems"
Briggs, Derek C.
Measurement: Interdisciplinary Research and Perspectives, v8 n4 p185-190 2010
The use of large-scale assessments for making high stakes inferences about students and the schools in which they are situated is premised on the assumption that tests are sensitive to good instruction. An increase in the quality of classroom instruction should cause, on the average, an increase in test scores. In work with a number of colleagues over the past decade, Dan Koretz has argued that the validity of high stakes inferences from school-level improvements in test performance can be threatened by strategic instructional activities that serve to "inflate" test scores (Koretz & Beguin, 2010; Koretz & Hamilton, 2006; Koretz, McCaffrey, & Hamilton, 2001). This might lead people to erroneously conclude that students have learned and that the underlying tests serve as a good barometer for this when all that has been demonstrated is that the tests are sensitive to innovative methods of coaching. As a method of evaluating this phenomenon, Koretz and Beguin (2010) have proposed a system of "self-monitoring assessments" (SMA) in which "audit items" are incorporated into the operational design of existing tests. While the author appreciates the underlying conceptual framework for score inflation being presented, he is pessimistic about the utility of the SMA approach. In this comment the author lays out his objections and sketches out an alternative. (Contains 1 figure and 1 footnote.)
Descriptors: Measurement, High Stakes Tests, Inferences, Scores, Test Validity, Audits (Verification), Test Items, Educational Testing
Psychology Press. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Opinion Papers
Education Level: N/A
Authoring Institution: N/A