ERIC Number: ED463746
Record Type: Non-Journal
Publication Date: 2000
Reference Count: N/A
Comparing Onsite and Online Standard Setting Methods for Multiple Levels of Standards.
Harvey, Anne L.
The Web-based standard setting (WBSS) system described in Harvey and Way (1999) provides an online, Internet-based alternative for testing programs that use onsite judgmental standard setting studies to set their cut scores. Standard setting studies using the WBSS system were compared to similar onsite studies of two exams in the College-Level Examination Program. The online and onsite groups felt similarly about the usefulness of the discussions in making the final ratings and about the overall experience of the study. Results indicated significant differences in the perceptions of group process and working conditions, with the online studies scoring lower on these aspects. Two methods of standard setting were implemented in both the online and onsite modes: an Angoff method and a no/yes method. No significant differences were found in the ratings for the no/yes method. Substantial differences in the average ratings for the Angoff method were found for one of the exams, but not for the other exam. Further research is needed in the aspects of these studies that cause a difference in one case but not the other. (Author/MES)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: College Level Examination Program