ERIC Number: EJ1185024
Record Type: Journal
Publication Date: 2018-Aug
Pages: 29
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0013-1644
EISSN: N/A
The Stabilizing Influences of Linking Set Size and Model-Data Fit in Sparse Rater-Mediated Assessment Networks
Wind, Stefanie A.; Jones, Eli
Educational and Psychological Measurement, v78 n4 p679-707 Aug 2018
Previous research includes frequent admonitions regarding the importance of establishing connectivity in data collection designs prior to the application of Rasch models. However, details regarding the influence of characteristics of the linking sets used to establish connections among facets, such as locations on the latent variable, model-data fit, and sample size, have not been thoroughly explored. These considerations are particularly important in assessment systems that involve large proportions of missing data (i.e., sparse designs) and are associated with high-stakes decisions, such as teacher evaluations based on teaching observations. The purpose of this study is to explore the influence of characteristics of linking sets in sparsely connected rating designs on examinee, rater, and task estimates. A simulation design whose characteristics were intended to reflect practical large-scale assessment networks with sparse connections were used to consider the influence of locations on the latent variable, model-data fit, and sample size within linking sets on the stability and model-data fit of estimates. Results suggested that parameter estimates for examinee and task facets are quite robust to modifications in the size, model-data fit, and latent-variable location of the link. Parameter estimates for the rater, while still quite robust, are more sensitive to reductions in link size. The implications are discussed as they relate to research, theory, and practice.
Descriptors: Data Collection, Goodness of Fit, Computation, Networks, Sample Size, Teacher Evaluation, Rating Scales, Monte Carlo Methods
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A