ERIC Number: ED411310
Record Type: Non-Journal
Publication Date: 1997-Mar
Reference Count: N/A
Computer-Assisted Portfolio Scoring: Can Technology Enhance the Process of Scoring Portfolios?
Solano-Flores, Guillermo; Raymond, Bruce; Schneider, Steven A.
The need for effective ways of monitoring the quality of scoring of portfolios resulted in the development of a software package that provides scoring leaders with updated information on their assessors' scoring quality. Assessors with computers enter data as they score, and this information is analyzed and reported to scoring leaders. The developed structural scoring approach, PerformAce, was tested with responses of teachers to the portfolio component of a teacher certification assessment developed by the National Board for Professional Teaching Standards (NBPTS). Assessors used the PerformAce software as an electronic scoring form with screens that display the description of an attribute at different levels of performance attached to a scale. When all attributes are scored, the program asks assessors for a holistic score for the examinee's performance. Both attribute and holistic scores are stored in the assessor's computer and then transferred to a central computer to allow scoring leaders to identify patterns of ineffective or inefficient scoring. The approach was used with the field test of an NBPTS art teacher examination with eight portfolio entries and four assessment center exercises. Assessors were 127 art teachers who rated 928 portfolio responses. The best possible estimate of the reliability of the assessment was 0.81, and using computers did not affect this reliability. Almost all of the assessors (94%) agreed that the approach helped them make informed decisions and 73% of the scoring leaders thought the software helped them preserve the quality of scoring. Results suggest that the computer-assisted scoring approach does not have an adverse impact on interrater reliability and that computers can be useful tools to manage scoring sessions. (Contains 1 table, 4 figures, and 12 references.) (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Sponsor: National Board for Professional Teaching Standards, Detroit, MI.
Authoring Institution: N/A