ERIC Number: EJ1094528
Record Type: Journal
Publication Date: 2016-May
Abstractor: As Provided
Defining Equivalence in Medical Education Evaluation and Research: Does a Distribution-Based Approach Work?
Rusticus, Shayna A.; Eva, Kevin W.
Advances in Health Sciences Education, v21 n2 p359-373 May 2016
Educators often seek to demonstrate the equivalence of groups, such as whether or not students achieve comparable success regardless of the site at which they trained. A methodological consideration that is often underappreciated is how to operationalize equivalence. This study examined whether a distribution-based approach, based on effect size, can identify an appropriate equivalence threshold for medical education data. Thirty-nine individuals rated program site equivalence on a series of simulated pairwise bar graphs representing one of four measures with which they had prior experience: (1) undergraduate academic achievement, (2) a student experience survey, (3) an Objective Structured Clinical Exam global rating scale, or (4) a licensing exam. Descriptive statistics and repeated measures ANOVA examined the effects on equivalence ratings of (a) the difference between means, (b) variability in scores, and (c) which program site (the larger or smaller) scored higher. The equivalence threshold was defined as the point at which 50% of participants rated the sites as non-equivalent. Across the four measures, the equivalence thresholds converged to average effect size of Cohen's d = 0.57 (range of 0.50-0.63). This corresponded to an average mean difference of 10% (range of 3-13%). These results are discussed in reference to findings from the health-related quality of life field that has demonstrated that d = 0.50 represents a consistent threshold for perceived change. This study provides preliminary empirically-based guidance for defining an equivalence threshold for researchers and evaluators conducting equivalence tests.
Descriptors: Medical Education, Effect Size, Simulation, Undergraduate Students, Academic Achievement, Statistical Analysis, Student Surveys, Rating Scales, Licensing Examinations (Professions), Scores, Quality of Life, Guidance, Equivalency Tests, Medical Students, Prior Learning
Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: email@example.com; Web site: http://www.springerlink.com
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A
Grant or Contract Numbers: N/A