ERIC Number: EJ1167877
Record Type: Journal
Publication Date: 2015-Sep
Abstractor: As Provided
Multi-Item Direct Behavior Ratings: Dependability of Two Levels of Assessment Specificity
Volpe, Robert J.; Briesch, Amy M.
School Psychology Quarterly, v30 n3 p431-442 Sep 2015
Direct Behavior Rating-Multi-Item Scales (DBR-MIS) have been developed as formative measures of behavioral assessment for use in school-based problem-solving models. Initial research has examined the dependability of composite scores generated by summing all items comprising the scales. However, it has been argued that DBR-MIS may offer assessment of 2 levels of behavioral specificity (i.e., item-level, global composite-level). Further, it has been argued that scales can be individualized for each student to improve efficiency without sacrificing technical characteristics. The current study examines the dependability of 5 items comprising a DBR-MIS designed to measure classroom disruptive behavior. A series of generalizability theory and decision studies were conducted to examine the dependability of each item ("calls out," "noisy," "clowns around," "talks to classmates" and "out of seat"), as well as a 3-item composite that was individualized for each student. Seven graduate students rated the behavior of 9 middle-school students on each item over 3 occasions. Ratings were based on 10-min video clips of students during mathematics instruction. Separate generalizability and decision studies were conducted for each item and for a 3-item composite that was individualized for each student based on the highest rated items on the first rating occasion. Findings indicate favorable dependability estimates for 3 of the 5 items and exceptional dependability estimates for the individualized composite.
Descriptors: Behavior Rating Scales, Outcome Measures, Progress Monitoring, Student Behavior, Formative Evaluation, Accuracy, Middle School Students, Mathematics Instruction, Generalizability Theory, Reliability, Behavior Problems, Statistical Analysis, Psychometrics, Intervention
American Psychological Association. Journals Department, 750 First Street NE, Washington, DC 20002. Tel: 800-374-2721; Tel: 202-336-5510; Fax: 202-336-5502; e-mail: email@example.com; Web site: http://www.apa.org
Publication Type: Journal Articles; Reports - Research
Education Level: Middle Schools
Authoring Institution: N/A
Grant or Contract Numbers: N/A