NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1298978
Record Type: Journal
Publication Date: 2021
Pages: 11
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0731-1745
EISSN: N/A
Boolean Analysis of Interobserver Agreement: Formal and Functional Evidence Sampling in Complex Coding Endeavors
Solano-Flores, Guillermo
Educational Measurement: Issues and Practice, v40 n2 p26-36 Sum 2021
This article proposes a Boolean approach to representing and analyzing interobserver agreement in dichotomous coding. Building on the notion that observations are samples of a universe of observations, it submits that coding can be viewed as a process in which observers sample pieces of evidence on constructs. It distinguishes between formal and functional interobserver agreement based on whether events coded by observers are, respectively, the same or different but equivalent samples of the same classes of events. Formal and functional agreement are defined as Boolean functions. The article reports on an investigation in which a Boolean approach was used to examine agreement in the coding of teacher-student social interactions. Agreement maps (devices that represent the relative frequencies of coding cases--different combinations of 1s and 0s in the observers' coding) allowed detail examination of how interobserver agreement was shaped by the nature of the events coded and the characteristics of the individuals whose behaviors were coded. Results provide empirical evidence of formal and functional agreement as distinguishable forms of interobserver agreement. Results also show that Boolean coding analysis allows detection of ways in which coding procedures confound absence of evidence with evidence of absence. Practical and methodological implications are discussed.
Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A