ERIC Number: EJ888463
Record Type: Journal
Publication Date: 2010
Abstractor: As Provided
Reference Count: 0
Proceed with Caution: Interactive Rules and Teacher Work Sample Scoring Strategies, an Ethnomethodological Study
Bullough, Robert V., Jr.
Teachers College Record, v112 n3 p775-810 2010
Background: Facing growing accountability pressures, Teacher Work Samples (TWSs) as a model of performance-based assessment is of growing significance in teacher education. Developed at Western Oregon University and widely adopted and adapted, proponents claim that the model is "real," "natural," "meaningful," and "helpful" (G. R. Girod, 2002). Research Questions: The study addresses three questions: (1) How do sample raters understand their responsibilities? (2) What are the underlying interactive rules and strategies used by raters to achieve their aims, and how are they employed? (3) What issues or concerns should teacher educators interested in using TWS methods address as they seek to demonstrate candidate quality and program value? Research Design: Conversation analysis, ethnomethodology. Data Collection and Analysis: Ten TWS scoring conversations conducted by four teams were recorded and analyzed to identify interactive rules and strategies. Scoring teams were composed of one tenure-track elementary teacher education faculty member and one clinical teacher education faculty member. Excerpts from a TWS case judged marginal are presented and analyzed. Findings: From the case, a set of interactive rules (tenure-track faculty speak first; the efficiency and equivalence rules; and scorers are prepared) and strategies (splitting the difference; rubric simplification; previewing scores; and rubric stretching) are identified, and implications of their use are discussed for assessment validity, fairness, content quality and coverage, meaningfulness, and cognitive complexity (R. L. Linn, E. L. Baker, & S. B. Dunbar, 1991). Conclusions: This study raises a number of concerns about the expectations for and use of Teacher Work Samples and cautions about their use for high-stakes assessment.
Descriptors: Work Sample Tests, Student Evaluation, Teacher Education, Evaluators, Scoring, Teacher Educators, Scoring Rubrics, Test Validity
Teachers College, Columbia University. P.O. Box 103, 525 West 120th Street, New York, NY 10027. Tel: 212-678-3774; Fax: 212-678-6619; e-mail: firstname.lastname@example.org; Web site: http://www.tcrecord.org
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A