ERIC Number: EJ1113667
Record Type: Journal
Publication Date: 2016-Sep
Abstractor: As Provided
Generalizability Theory Reliability of Written Expression Curriculum-Based Measurement in Universal Screening
Keller-Margulis, Milena A.; Mercer, Sterett H.; Thomas, Erin L.
School Psychology Quarterly, v31 n3 p383-392 Sep 2016
The purpose of this study was to examine the reliability of written expression curriculum-based measurement (WE-CBM) in the context of universal screening from a generalizability theory framework. Students in second through fifth grade (n = 145) participated in the study. The sample included 54% female students, 49% White students, 23% African American students, 17% Hispanic students, 8% Asian students, and 3% of students identified as 2 or more races. Of the sample, 8% were English Language Learners and 6% were students receiving special education. Three WE-CBM probes were administered for 7 min each at 3 time points across 1 year. Writing samples were scored for commonly used WE-CBM metrics (e.g., correct minus incorrect word sequences; CIWS). Results suggest that nearly half the variance in WE-CBM is related to unsystematic error and that conventional screening procedures (i.e., the use of one 3-min sample) do not yield scores with adequate reliability for relative or absolute decisions about student performance. In most grades, three 3-min writing samples (or 2 longer duration samples) were required for adequate reliability for relative decisions, and three 7-min writing samples would not yield adequate reliability for relative decisions about within-year student growth. Implications and recommendations are discussed.
Descriptors: Generalizability Theory, Reliability, Written Language, Curriculum Based Assessment, Screening Tests, Student Evaluation, Elementary School Students, Error of Measurement, Scores, Writing Evaluation, At Risk Students, Data Analysis, Scoring Rubrics, Benchmarking
American Psychological Association. Journals Department, 750 First Street NE, Washington, DC 20002. Tel: 800-374-2721; Tel: 202-336-5510; Fax: 202-336-5502; e-mail: firstname.lastname@example.org; Web site: http://www.apa.org
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Education
Authoring Institution: N/A
Identifiers - Location: Texas (Houston)
Grant or Contract Numbers: N/A