NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ797384
Record Type: Journal
Publication Date: 2008-Apr
Pages: 19
Abstractor: Author
Reference Count: 7
Examining Panelist Data from a Bilingual Standard Setting Study
Rodeck, Elaine M.; Chin, Tzu-Yun; Davis, Susan L.; Plake, Barbara S.
Journal of Applied Testing Technology, v9 n3 p1-19 Apr 2008
This study examined the relationships between the evaluations obtained from standard setting panelists and changes in ratings between different rounds of a standard setting study that involved setting standards on different language versions of an exam. We investigated panelists' evaluations to determine if their perceptions of the standard setting were related to adjustments they made in their recommended cut scores across rounds of the process. The standard setting was conducted for a high school mathematics test composed of multiple-choice and constructed response items. The test was designed for a population of students who speak and receive primary instruction in either English or French. Results indicated panelists' ratings of their ratings and their comfort with the process were related to how their ratings changed across sequential rounds of the process. Differences in the degree to which the evaluations influenced the standard setting judgments were observed across the English and French panelists, with the French group reporting increasing comfort across rounds in contrast to the English group that had relatively higher comfort at the beginning of the process. The results illustrate how standard setting evaluation data can provide insight into factors that affect panelists' ratings. (Contains 1 figure and 7 tables.)
Association of Test Publishers. 601 Pennsylvania Avenue NW, South Building, Suite 900, Washington, DC 20004. Tel: 866-240-7909; Fax: 717-755-8962; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: Elementary Secondary Education; Grade 9; High Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A