NotesFAQContact Us
Search Tips
ERIC Number: ED513784
Record Type: Non-Journal
Publication Date: 2010-Oct
Pages: 22
Abstractor: As Provided
Reference Count: 11
The Elusive Nature of Reliability: Problems and Pitfalls in Scoring Clinical Practice Action Research Projects
Moffett, David W.; Reid, Barbara K.
Online Submission, Paper presented at the Annual Meeting of the Georgia Educational Research Association (Savannah, GA, Oct 22-23, 2010)
The Investigators studied scoring reliability of Candidates' ten day unit plans of instruction through prescribed action research projects, across three academic years. Scoring of the projects in year one provided opportunities for further refinement of the action research evaluation methods in year two. Across three terms in years one and two scoring reliability was realized. However, although reliability was realized in term two of year two, new scoring issues arose. Subsequently in year three of the study reliability was not realized across rater and interraters due to a lack of updating points earned on projects in mastery learning, lack of abiding by the rubric for scoring, and informal discussions between the rater and some interrators affecting scoring of projects. Discussion includes changes that likely led to reaching early satisfactory reliability levels, as well as reasons for not realizing reliability in the most recent year. Recommendations follow that may be helpful to education Units seeking to learn more about reliability issues when scoring Candidate items. (Contains 7 tables.)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A