ERIC Number: EJ1022214
Record Type: Journal
Publication Date: 2013
Abstractor: As Provided
An Application of Generalizability Theory to Evaluate the Technical Quality of an Alternate Assessment
Taylor, Melinda Ann; Pastor, Dena A.
Applied Measurement in Education, v26 n4 p279-297 2013
Although federal regulations require testing students with severe cognitive disabilities, there is little guidance regarding how technical quality should be established. It is known that challenges exist with documentation of the reliability of scores for alternate assessments. Typical measures of reliability do little in modeling multiple sources of error, which are characteristic of alternate assessments. Instead, Generalizability theory (G-theory) allows researchers to identify sources of error and analyze the relative contribution of each source. This study demonstrates an application of G-theory to examine reliability for an alternate assessment. A G-study with the facets rater type, assessment attempts, and tasks was examined to determine the relative contribution of each to observed score variance. Results were used to determine the reliability of scores. The assessment design was modified to examine how changes might impact reliability. As a final step, designs that were deemed satisfactory were evaluated regarding the feasibility of adapting them into a statewide standardized assessment and accountability program.
Descriptors: Generalizability Theory, Alternative Assessment, Test Reliability, Scores, Error of Measurement, Disabilities, Grade 5, Grade 8, Grade 10, Reading Tests, Mathematics Tests, Statistical Analysis, Test Construction
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Secondary Education; Elementary Education; Intermediate Grades; Middle Schools; Junior High Schools; High Schools; Secondary Education; Grade 5; Grade 8; Grade 10
Authoring Institution: N/A