NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ935394
Record Type: Journal
Publication Date: 2009
Pages: 22
Abstractor: As Provided
Reference Count: 30
ISSN: ISSN-0834-1516
Interrater Reliability in Content Analysis of Healthcare Service Quality Using Montreal's Conceptual Framework
Leclerc, Bernard-Simon; Dassa, Clement
Canadian Journal of Program Evaluation, v24 n2 p81-102 Fall 2009
This study examines the usefulness of the Montreal Service Concept framework of service quality measurement, when it was used as a predefined set of codes in content analysis of patients' responses. As well, the study quantifies the interrater agreement of coded data. Two raters independently reviewed each of the responses from a mail survey of ambulatory patients about the quality of care and recorded whether or not a patient expressed each concern. Interrater agreement was measured in three ways: the percent crude agreement, Cohen's kappa, and the coefficient of the generalizability theory. We found all levels of interrater code-specific agreement to be over 96%. All kappa values were above 0.80, except four codes associated with rarely observed characteristics. A coefficient of generalizability equal to 0.93 was obtained. All indices consistently revealed substantial agreement. We empirically showed that the content categories of the Montreal Service Concept were exhaustive and reliable in a well-defined content-analysis procedure. (Contains 1 figure and 3 tables.)
Canadian Evaluation Society. 1485 Laperriere Avenue, Ottawa, ON K1Z 7S8, Canada. Tel: 613-725-2526; Fax: 613-729-6206; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Canada (Montreal)