Participant attrition may be a significant threat to the generalizability of the results of educational research studies if participants who do not persist in a study differ from those who do in ways that can affect the experimental outcomes. A multi-center trial of the efficacy of different computer-based instructional strategies gave us the opportunity to observe institutional and student factors linked to attrition from a study and the ways in which they altered the participation profile. The data is from a randomized controlled trial conducted at seven US medical schools investigating the educational impact of different instructional designs for computer-based learning modules for surgical clerks. All students undertaking their surgical clerkships at the participating schools were invited participate and those that consented were asked to complete five study measures during their surgery clerkship. Variations in study attrition rates were explored by institution and by participants' self-regulation, self-efficacy, perception of task value, and mastery goal orientation measured on entry to the study. Of the 1,363 invited participants 995 (73%) consented to participate and provided baseline data. There was a significant drop in the rate of participation at each of the five study milestones with 902 (94%) completing at least one of two module post-test, 799 (61%) both module post-tests, 539 (36%) the mid-rotation evaluation and 252 (25%) the final evaluation. Attrition varied between institutions on survival analysis (p < 0.001). Small but statistically significant differences in self-regulation (p = 0.01), self-efficacy (p = 0.02) and task value (p = 0.04) were observed but not in mastery or performance goal orientation measures (p = NS). Study attrition was correlated with lower achievement on the National Board of Medical Examiners subject exam. The results of education trials should be interpreted with the understanding that students who persist may be somewhat more self-regulated, self-efficacious and higher achievers than their peers who drop out and as such do not represent the class as a whole.