NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1002205
Record Type: Journal
Publication Date: 2013
Pages: 3
Abstractor: ERIC
ISSN: ISSN-1536-6367
The Need for a Principled Approach for Examining Indirect Effects of Test Use
Lane, Suzanne
Measurement: Interdisciplinary Research and Perspectives, v11 n1-2 p44-46 2013
In Shepard's (1997) discussion on the importance of test use and consequences in a validity argument for educational assessments, she reflected on Cronbach and Meehl's (1955) perspective on the role of test developers in providing consequential evidence. In the following year, a special issue in "Educational Measurement: Issues and Practice" tackled the difficult question of where the responsibility lies for investigating consequences of state assessment and accountability systems (Yen, 1998) and provided a framework for collecting such evidence (Lane, Parke, & Stone, 1998). In Kane's (2006) seminal chapter on validation, he also maintained that the test developer is responsible for collecting evidence for the intended uses of tests and that the test users, "those who make the policy decision to use a test for some purpose," have an obligation to be involved in the examination of the consequences for that purpose as well as collecting evidence for uses of tests that are not proposed by the test developer. Nichols and Williams (2009) provided a comprehensive review of the responsibilities of test users and developers collecting evidence of consequences for test use. They also provided a framework delineating who is responsible according to different conditions, demonstrating how the responsibility shifts as the context of test score use shifts, including the proximity of test score use to the intended test use. Haertel reminds measurement professionals that they have not clearly articulated the indirect mechanisms of actions nor have they been effective at designing a principled approach to studying both the intended and unintended effects of test use. Haertel's framework for classifying mechanisms of intended testing effects as direct versus indirect helps clarify one's thinking on the validity evidence needed for assessment and accountability systems. As described by Haertel, indirect effects, including directing student effort, focusing the system, and shaping public perceptions, have no direct dependence on the information provided by test scores but are linked closely to the purposes of assessments. As indicated by the Race to the Top Assessment Program (U.S. Department of Education, 2009), an important goal of the assessments that are being developed by the two consortia, Partnership for Assessment of Readiness for College and Careers (PARCC) and Smarter Balanced Assessment Consortium (SBAC), is to "play a critical role in educational systems; provide administrators, educators, parents, and students with the data and information needed to continuously improve teaching and learning." This indirect effect of focusing the system requires a clearly delineated validity plan for the collection of evidence to examine the effects of these assessments on improving instruction and learning.
Psychology Press. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site:
Publication Type: Journal Articles; Opinion Papers
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A