In 2000, the Accreditation Council of Graduate Medical Education (ACGME) laid out a definition of competence that included six specific areas of focus: patient care (including clinical reasoning), medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice. The ACGME's intention was to mandate increasingly reliable and valid assessment measures to be used by all training programs over the next decade, with the goal of providing "more credible, accurate, reliable and useful educational outcome data." In this article, the authors will review definitions of competency according to the new ACGME standards and examine the assessment tools currently available, including global evaluations, 360-degree evaluations, checklists, standardized examinations and direct observations. The authors will provide an overview of critical considerations in different assessment methods, including timing, psychometric properties, benchmarks, and feedback. Finally, the authors will discuss the relevant literature concerning the strengths and weaknesses of these various assessment tools. Throughout, the authors will comment on the applicability of the literature on assessment to the field of psychiatry and consider directions for competency assessment within and beyond psychiatric training. Following the paper is an annotated bibliography of the literature for those wishing to explore this topic more deeply and a list of web-based resources that may be used by those wishing to access available instruments.