NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ846843
Record Type: Journal
Publication Date: 2005-Jul
Pages: 12
Abstractor: As Provided
Reference Count: 25
ISBN: N/A
ISSN: ISSN-1492-3831
Sources of Difference in Reliability: Identifying Sources of Difference in Reliability in Content Analysis of Online Asynchronous Discussions
Murphy, Elizabeth; Ciszewska-Carr, Justyna
International Review of Research in Open and Distance Learning, v6 n2 Jul 2005
This paper reports on a case study which identifies and illustrates sources of difference in agreement in relation to reliability in a context of quantitative content analysis of a transcript of an online asynchronous discussion (OAD). Transcripts of 10 students in a month-long online asynchronous discussion were coded by two coders using an instrument with two categories, five processes, and 19 indicators of Problem Formulation and Resolution (PFR). Sources of difference were identified in relation to: coders; tasks; and students. Reliability values were calculated at the levels of categories, processes, and indicators. At the most detailed level of coding on the basis of the indicator, findings revealed that the overall level of reliability between coders was 0.591 when measured with Cohen's kappa. The difference between tasks at the same level ranged from 0.349 to 0.664, and the difference between participants ranged from 0.390 to 0.907. Implications for training and research are discussed. (Contains 3 tables.)
Athabasca University. 1200, 10011 - 109 Street, Edmonton, AB T5J 3S8, Canada. Tel: 780-421-2536; Fax: 780-497-3416; e-mail: irrodl@athabascau.ca; Web site: http://www.irrodl.org
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A