ERIC Number: EJ697793
Record Type: Journal
Publication Date: 2005
Reference Count: N/A
Cognitive Representations in Raters' Assessment of Teacher Portfolios
van der Schaaf, Marieke; Stokking, Karel; Verloop, Nico
Studies in Educational Evaluation, v31 n1 p27-55 2005
Portfolios are frequently used to assess teachers' competences. In portfolio assessment, the issue of rater reliability is a notorious problem. To improve the quality of assessments insight into raters' judgment processes is crucial. Using a mixed quantitative and qualitative approach we studied cognitive processes underlying raters' judgments and the reliability of these judgments. Six raters systematically assessed 18 portfolios. The interrater reliability of 12 portfolios was satisfactory. Variance analysis showed slight rater effects. We used the Correspondent Inference Theory (Jones & Davis, 1965) and the Associative Systems Theory (Carlston, 1992; 1994) to analyse judgment forms and retrospective verbal protocols. Raters' cognitive representations on the dimensions "abstract-concrete" and "positive-negative" were significantly related to the judgments given and to the reliability of these judgments.
Descriptors: Portfolios (Background Materials), Systems Approach, Cognitive Processes, Portfolio Assessment, Interrater Reliability
Elsevier Customer Service Department, 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126 (Toll Free); Fax: 407-363-1354; e-mail: firstname.lastname@example.org.
Publication Type: Journal Articles
Education Level: N/A
Authoring Institution: N/A