NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ920297
Record Type: Journal
Publication Date: 2011-May
Pages: 12
Abstractor: As Provided
Reference Count: 21
ISBN: N/A
ISSN: ISSN-0146-6216
The Impact of Statistically Adjusting for Rater Effects on Conditional Standard Errors of Performance Ratings
Raymond, Mark R.; Harik, Polina; Clauser, Brian E.
Applied Psychological Measurement, v35 n3 p235-246 May 2011
Prior research indicates that the overall reliability of performance ratings can be improved by using ordinary least squares (OLS) regression to adjust for rater effects. The present investigation extends previous work by evaluating the impact of OLS adjustment on standard errors of measurement ("SEM") at specific score levels. In addition, a cross-validation (i.e., resampling) design was used to determine the extent to which any improvements in measurement precision would be realized for new samples of examinees. Conditional "SEM"s were largest for scores toward the low end of the score distribution and smallest for scores at the high end. Conditional "SEM"s for adjusted scores were consistently less than conditional "SEM"s for observed scores, although the reduction in error was not uniform throughout the distribution. The improvements in measurement precision held up for new samples of examinees at all score levels. (Contains 2 tables, 2 figures, and 1 note.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A