Background: Medical knowledge is essential for appropriate patient care; however, the accuracy of internal medicine (IM) residents' assessment of their medical knowledge is unknown. Methods: IM residents predicted their overall percentile performance 1 week (on average) before and after taking the in-training exam (ITE), an objective and well accepted method to assess medical knowledge to study resident assessment accuracy. Ordinary least squares regression was used to study the association between the absolute accuracy of their predictions of their percentile performance on the ITE examination and their actual percentile performance. Results: Ninety-three percent of our 28 residents participated. Residents were highly inaccurate in predicting their percentile performance. Only 31% had ITE scores that were within 10 points of their predictions. On average, most residents were pessimistic about their overall percentile performance with 18 (69%) underestimating their performance. Having just taken the examination and previous experience with the examination did not improve predictions of percentile performance. Conclusion: IM residents did not accurately predict their own level of medical knowledge. Taking the examination and experience with previous exams does not appear to improve the ability of these IM residents to predict their performance. Residents need to be taught to rely on the results of standardized examinations such as the ITE to best assess their overall medical knowledge. The factors important for accurate self-assessment of medical knowledge in individual clinical situations remain unexplored.