ERIC Number: ED565847
Record Type: Non-Journal
Publication Date: 2012-Feb
Abstractor: As Provided
Reference Count: 35
Assessing the Rothstein Test: Does It Really Show Teacher Value-Added Models Are Biased? Working Paper 5
Goldhaber, Dan; Chaplin, Duncan
Mathematica Policy Research, Inc.
In a provocative and influential paper, Jesse Rothstein (2010) finds that standard value-added models (VAMs) suggest implausible future teacher effects on past student achievement, a finding that obviously cannot be viewed as causal. This is the basis of a falsification test (the Rothstein falsification test) that appears to indicate bias in VAM estimates of current teacher contributions to student learning. Rothstein's finding is significant because there is considerable interest in using VAM teacher effect estimates for high-stakes teacher personnel policies, and the results of the Rothstein test cast considerable doubt on the notion that VAMs can be used fairly for this purpose. However, in this paper, we illustrate--theoretically and through simulations--plausible conditions under which the Rothstein falsification test rejects VAMs even when there is no bias in estimated teacher effects, and even when students are randomly assigned conditional on the covariates in the model. On the whole, our findings show that the "Rothstein falsification test" is not definitive in showing bias, which suggests a much more encouraging picture for those wishing to use VAM teacher effect estimates for policy purposes. The following are appended: (1) When Falsification Tests Fail; and (2) Negatively Correlated Errors Need Not Cause Bias.
Descriptors: Value Added Models, Academic Achievement, Teacher Effectiveness, Correlation, Regression (Statistics), Simulation, Teachers, Statistical Bias, Computation, Error Patterns, Statistical Significance, Test Bias, Models
Mathematica Policy Research, Inc. P.O. Box 2393, Princeton, NJ 08543-2393. Tel: 609-799-3535; Fax: 609-799-0005; e-mail: firstname.lastname@example.org; Web site: http://www.mathematica-mpr.com
Publication Type: Reports - Descriptive
Education Level: N/A
Authoring Institution: Mathematica Policy Research, Inc.
IES Cited: ED544205