NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1081192
Record Type: Journal
Publication Date: 2015-Dec
Pages: 24
Abstractor: As Provided
ISSN: ISSN-0013-1644
Effects of Design Properties on Parameter Estimation in Large-Scale Assessments
Hecht, Martin; Weirich, Sebastian; Siegle, Thilo; Frey, Andreas
Educational and Psychological Measurement, v75 n6 p1021-1044 Dec 2015
The selection of an appropriate booklet design is an important element of large-scale assessments of student achievement. Two design properties that are typically optimized are the "balance" with respect to the positions the items are presented and with respect to the mutual occurrence of pairs of items in the same booklet. The purpose of this study is to investigate the effects of these two design properties on bias and root mean square error of item parameter estimates from the Rasch model. First, position effects are estimated using data from a large-scale assessment study measuring the competencies of 19,107 ninth graders in science. These results were then used for a simulation study with 1,540 booklet designs with systematically varied position balance and cluster pair balance. The simulation results showed a small effect of position balancing on bias and root mean square error of the item parameter estimates while the cluster pair balance was ignorable. This null effect is actually good news for test designers since it allows for deliberately reducing the degree of cluster pair balance without negative effects on item parameter estimates. However, it is recommended to try to achieve a high position balance when designing large-scale assessment studies.
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 9; Junior High Schools; Middle Schools; Secondary Education; High Schools
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Germany
Grant or Contract Numbers: N/A