NotesFAQContact Us
Search Tips
ERIC Number: ED465778
Record Type: Non-Journal
Publication Date: 2002-Apr
Pages: 8
Abstractor: N/A
The Occasion Effect for "At-Home" Angoff Ratings.
Wiley, Andrew; Guille, Robin
This exploratory study extends the work done by B. Plake and others (2000) and R. Guille and others (2001) by investigating whether a negligible occasion facet would still be found when ratings for licensure and certification examinations were completed in isolation. A set of items was sent to a standard-setting committee to be reviewed at home, completely independently of all other members of the committee. Seven to nine raters reviewed each item. The examination was a medical certification examination administered once a year. Approximately half the 200 items had Angoff ratings already assigned from use on a previous examination, so only half of the items needed Angoff values assigned to them. For the study, a set of items needing Angoff ratings was sent to committee members at home, along with a set of 13 anchor items that had previously been rated. For the at-home ratings, the mean and standard deviation for the items were compared with the values obtained at the most recent standard-setting meeting. Results provide consistent evidence of a slight item-occasion interaction that appears to be the result of the increased variability in the ratings during the at-home session as compared to the normal session. Findings do provide preliminary support for the idea that Angoff ratings obtained at home may not be significantly different from ratings provided during traditional standard setting meetings. (Contains 1 table and 14 references.) (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A