This article investigates if online evaluation leads to different results than paper-and-pencil evaluation. Given that most previous studies confound the evaluation mode (online versus paper) with the evaluation situation (in class versus after class), we expected that evaluation results would be influenced only by the evaluation situation, whereas the very evaluation mode would not be relevant for the evaluation results. Analysing evaluations of a training programme for university teachers, we compared paper evaluations after class to either paper evaluations in class or online evaluations after class. The response rates were higher for the in-class compared to the after-class evaluations. Likewise, the scale ratings showed that paper evaluations after class were more negative than paper evaluations in class. In addition, participants wrote longer negative comments, but not positive comments, when evaluating after class compared to in class, independent of evaluation mode. The data support the hypothesis that only the evaluation situation, but not the evaluation mode, influences evaluation results.