ERIC Number: ED355284
Record Type: Non-Journal
Publication Date: 1992-Dec-4
Reference Count: N/A
The Reliability of Scores from the 1992 Vermont Portfolio Assessment Program. Interim Report.
Koretz, Daniel; And Others
Since 1988, Vermont has been developing an innovative performance assessment program that relies substantially on portfolios of student work. This interim report presents findings about the reliability of scores from the first statewide implementation of the portfolio program in the 1991-92 school year. The focus is not on program impact as an educational intervention, but on the program's quality as an assessment tool. Students' writing was scored on five criteria, each of which was scored on a four-point scale. Mathematics samples were scored on seven criteria on four-point scales. The extent of agreement between raters about the quality of both writing and mathematics was on the average low. The overall pattern was one of low reliability, and in no instance was the scoring highly reliable. The reliability coefficients (ranging from 0.33 to 0.43) are low enough to seriously limit the use of the 1992 assessment results. Despite the unreliability of individual student scores, statewide average scores are quite reliable because of the large numbers of students represented. Some suggestions are given to increase the reliability of portfolio scores, such as simplifying scoring systems and providing better training for scorers. Thirteen tables present study findings. (SLD)
Publication Type: Reports - Research
Education Level: N/A
Sponsor: Office of Educational Research and Improvement (ED), Washington, DC.
Authoring Institution: National Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA.; Rand Corp., Santa Monica, CA. Inst. on Education and Training.
Identifiers - Location: Vermont