The teaching of interviewing skills is increasingly incorporated into clinical medicine courses in American medical schools, yet the attempts to evaluate the effectiveness of these efforts have been woefully inadequate. Typical outcome measures have included paper and pencil tests of knowledge, preceptor evaluations of clinical performance, and pre- and post-course attitude ratings. A performance-based method for evaluation of interviewing skills, using videotaped interviews of medical students with simulated patients, was compared with these more traditional measures in a third-year psychiatric clerkship. Each videotaped interview (pre- and post-course) was rated independently by one of five raters using a modified version of the Arizona Clinical Interview Rating Scale (ACIRS) and an instrument providing a content score based on a case-specific checklist of symptoms. The analysis of the data showed a significant improvement in student attitudes towards the psychological and social aspects of medicine as a result of the clerkship, and a significant rise in interviewing performance in the simulated interviews. However, there was little correlation between the changes in these two measures, and none between each and the preceptor evaluations normally used in this course. The results suggest that indirect outcome measures are a poor substitute for behaviorally-based evaluation methods, and do not predict to future clinical performance. (Author/JAC)