NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ868310
Record Type: Journal
Publication Date: 2009
Pages: 4
Abstractor: As Provided
Reference Count: 5
ISSN: ISSN-0894-1912
Interrater Reliability to Assure Valid Content in Peer Review of CME-Accredited Presentations
Quigg, Mark; Lado, Fred A.
Journal of Continuing Education in the Health Professions, v29 n4 p242-245 Aut 2009
Introduction: The Accreditation Council for Continuing Medical Education (ACCME) provides guidelines for continuing medical education (CME) materials to mitigate problems in the independence or validity of content in certified activities; however, the process of peer review of materials appears largely unstudied and the reproducibility of peer-review audits for ACCME accreditation and designation of American Medical Association Category 1 CreditTM is unknown. Methods: Categories of presentation defects were constructed from discussions of the CME committee of the American Epilepsy Society: (1) insufficient citation, (2) poor formatting, (3) nonacknowledgment of non-FDA-approved use, (4) misapplied data, (5) 1-sided data, (6) self- or institutional promotion, (7) conflict of interest/commercial bias, (8) other, or (9) no defect. A PowerPoint lecture (n = 29 slides) suitable for presentation to general neurologists was purposefully created with the above defects. A multirater, multilevel kappa statistic was determined from the number and category of defects. Results: Of 14 reviewers, 12 returned completed surveys (86%) identifying a mean plus or minus standard deviation 1.6 plus or minus 1.1 defects/slide. The interrater kappa equaled 0.115 (poor reliability) for number of defects/slides. No individual categories achieved kappa greater than 0.38. Discussion: Interrater reliability on the rating of durable materials used in subspecialty CME was poor. Guidelines for CME appropriate content are too subjective to be applied reliably by raters knowledgeable in their specialty field but relatively untrained in the specifics of CME requirements. The process of peer review of CME materials would be aided by education of physicians on validation of materials appropriate for CME. (Contains 1 table and 1 figure.)
John Wiley & Sons, Inc. Subscription Department, 111 River Street, Hoboken, NJ 07030-5774. Tel: 800-825-7550; Tel: 201-748-6645; Fax: 201-748-6021; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A