ERIC Number: EJ1134358
Record Type: Journal
Publication Date: 2017-Mar
Abstractor: As Provided
Dependability of Data Derived from Time Sampling Methods with Multiple Observation Targets
Johnson, Austin H.; Chafouleas, Sandra M.; Briesch, Amy M.
School Psychology Quarterly, v32 n1 p22-34 Mar 2017
In this study, generalizability theory was used to examine the extent to which (a) time-sampling methodology, (b) number of simultaneous behavior targets, and (c) individual raters influenced variance in ratings of academic engagement for an elementary-aged student. Ten graduate-student raters, with an average of 7.20 hr of previous training in systematic direct observation and 58.20 hr of previous direct observation experience, scored 6 videos of student behavior using 12 different time-sampling protocols. Five videos were submitted for analysis, and results for observations using momentary time-sampling and whole-interval recording suggested that the majority of variance was attributable to the rating occasion, although results for partial-interval recording generally demonstrated large residual components comparable with those seen in prior research. Dependability coefficients were above .80 when averaging across 1 to 2 raters using momentary time-sampling, and 2 to 3 raters using whole-interval recording. Ratings derived from partial-interval recording needed to be averaged over 3 to 7 raters to demonstrate dependability coefficients above 0.80.
Descriptors: Generalizability Theory, Sampling, Elementary School Students, Learner Engagement, Graduate Students, Observation, Student Behavior, Video Technology, Scoring, Interrater Reliability
American Psychological Association. Journals Department, 750 First Street NE, Washington, DC 20002. Tel: 800-374-2721; Tel: 202-336-5510; Fax: 202-336-5502; e-mail: email@example.com; Web site: http://www.apa.org
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Education; Higher Education; Postsecondary Education
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R324A110017