ERIC Number: ED452205
Record Type: Non-Journal
Publication Date: 2000-Apr-25
Reference Count: N/A
Assessing the Impact of Standardized Patient Variability on Examination Mastery-Level Decision Consistency Rates.
De Champlain, Andre F.; Gessaroli, Marc E.; Floreck, Lisa M.
The purpose of this study was to estimate the extent to which recording variability among standardized patients (SPs) has an impact on classification consistency with data sets simulated to reflect performances on a large-scale clinical skills examination. SPs are laypersons trained to portray patients in clinical encounters (cases) and to record as well as rate examinee behaviors using case-specific checklists and rating scales. The conditions modeled were intended to approximate those that might occur with SP testing as part of the United States Medical Licensing Examination with populations of U.S. medical school graduates and populations that contain U.S. graduates and international medical graduates. Initially, proficiencies were randomly generated for a n(0,1) distribution for 100,000 simulees. Conditions that were simulated, in terms of the characteristics of percent-correct score distributions, overall failure rate, and amount of variability among recordings were similar to those noted in past research with this type of examination. Results suggest that the impact of recording discrepancies on overall misclassification rates is minimal for a (homogeneous) population resembling first time U.S. medical school graduates. Findings differed for the more heterogeneous population, and the total misclassification rate was 2.6% higher in the baseline condition. Results suggest that errors of commission similar to those simulated in this investigation could have serious consequences on the false positive rate for a more heterogeneous group of examinees unless some adjustments are made to account for SP rater variability. (Contains 2 tables and 30 references.) (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A