ERIC Number: EJ1020983
Record Type: Journal
Publication Date: 2014-Mar
Abstractor: As Provided
Direct Behavior Rating: Considerations for Rater Accuracy
Harrison, Sayward E.; Riley-Tillman, T. Chris; Chafouleas, Sandra M.
Canadian Journal of School Psychology, v29 n1 p3-20 Mar 2014
Direct behavior rating (DBR) offers users a flexible, feasible method for the collection of behavioral data. Previous research has supported the validity of using DBR to rate three target behaviors: academic engagement, disruptive behavior, and compliance. However, the effect of the base rate of behavior on rater accuracy has not been established. In addition, although teachers report frequent use of DBR-like tools, little is known about what type of training is necessary to ensure accurate usage. The present study sought to determine (a) whether training with practice and feedback improves rater accuracy when rating academic engagement, disruptive behavior, and compliance and (b) whether rater accuracy varies at low, medium, and high levels of these behaviors. Participants ("N" = 67) were randomly assigned to one of three training conditions: brief familiarization, brief training with practice and feedback, or extensive training with practice and feedback. Following training, participants watched videos of elementary students and rated behavior with DBRs. Results indicated that practice with feedback improved rater accuracy for disruptive behavior but not for the behaviors of academic engagement and compliance. In addition, rater accuracy was found to vary significantly at different levels of behavior. Academic engagement was rated most accurately when high rates of the behavior were displayed. Disruptive behavior and compliance were rated most accurately at low or high base rates, while raters had more difficulty rating medium levels of behaviors. Limitations, implications, and future directions for research are discussed.
Descriptors: Behavior Rating Scales, Accuracy, Learner Engagement, Behavior Problems, Compliance (Psychology), Student Behavior, Training, Feedback (Response), Undergraduate Students, Instructional Effectiveness, Questionnaires
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: firstname.lastname@example.org; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A
Grant or Contract Numbers: N/A