This study aims to examine the potential impacts on repeat examinees' performance by reusing simulation-based items in a high-stakes standardized assessment. We examined change patterns of item scores, ability estimate, score pattern change, response time and compared the performance of repeat examinees who have received repeat items and those who haven't. Results suggest that there are limited benefits from encountering the same items. The practical implications to licensing/certification assessments are discussed.