Social work educators are compelled to use reliable and valid methods to assess student learning outcomes. This study adapted a clinical simulation by integrating traditional role-play of case scenarios and elements of the Objective Structured Clinical Examination, which is often used to assess students' practice skills. Master of Social Work (MSW) students' engagement skills were assessed at the end of the first semester of their foundation year using the clinical simulation. Participants were first-year MSW students from two theory and practice courses. The study demonstrated the value of using a clinical simulation in practice courses to assess engagement skills. Correlations of self, peer, and instructor performance scores were compared and interrater reliability was assessed. Peers rated their classmates' performance the highest, and some peer ratings were negatively correlated with instructor ratings. There was higher correspondence in self- and instructor ratings in some skills and poor correspondence in others. Peers tended to rate their classmates' performance as showing intermediate proficiency, while self- and instructor ratings of engagement skills indicated basic proficiency. Overall, there was poor interrater reliability. Challenges relating to the implementation of a clinical simulation and its implications for social work education are examined.