Simulation-based surgical skills training during preclinical education is a persistent challenge due to time constraints of trainees and instructors alike. Self-directed practice is resource-efficient and flexible; however, insight into technical proficiency among trainees is often lacking. The purpose of this study is to prospectively assess the accuracy of self-assessments among medical students learning basic surgical suturing. Over seven weekly practice sessions, preclinical medical students performed serial repetitions of a simulation-based suturing task under one-on-one observation by one of four trainers. Following each task repetition, self- and trainer-assessments (SA-TA) were performed using a 36-point weighted checklist of technical standards developed a priori by expert consensus. Upon study completion, agreement between SA and TA was measured using weighted Cohen's kappa coefficients. Twenty-nine medical students each performed a median of 25 suture task repetitions (IQR 21.5-28). Self-assessments tended to overestimate proficiency during the first tertile of practice attempts. Agreement between SA and TA improved with experience, such that the weighted kappa statistics for the two-handed and instrument ties were >0.81 after 18-21 task attempts. Inexperienced trainees frequently overestimate technical proficiency through self-assessments. However, this bias diminishes with repetitive practice. Only after trainees have attained the capacity to accurately self-assess can effective self-directed learning take place.