Validity is an important issue when measuring teaching quality using student evaluations. This study examined the effects of psychology students' subjective clarity about study contents and their prior interest in the subject as variables possibly biasing the evaluations of psychology courses. German students (N = 292) evaluated lectures and seminars over five years with a standardized questionnaire, yielding 3,348 data points. In cross-classified multilevel models, we separated the total variance into the variance components of course, teacher, student, and the Teacher x Student interaction and found evidence for biasing effects with regard to the students' clarity about study contents and prior subject interest. These effects were small overall and were stronger for lectures than for seminars. These results suggest that the validity of evaluations of teaching in psychology can be improved by creating realistic expectations of what psychology is about before students choose psychology as a study subject.