The flipped classroom has shown a positive effect on academic performance and student satisfaction. However, there are relatively few studies that help us understand when or why this method has a positive effect on students, so as to aid the design of a flipped class. In this study, an instrument for assessing the student learning experience in flipped courses is proposed and validated, comparing the results of their application over two semesters in a university-level programming class with three flipped sections (n = 151) and four conventional ones (n = 226). We found that with a similar score in terms of learning experience, the achievement in a flipped course is slightly higher than in a lecture-based course, F(1,108) = 4.20, p = 0.04, d = 0.19. A multiple regression analysis was conducted for the effect on achievement of GPA, level of challenge, feedback and enjoyment. By controlling for these factors, students in flipped classes achieved higher scores than students in lecture-based courses, ß = 0.1, t(370) = 2.4, p = 0.02. The expected improvement in achievement increases when features of the flipped classroom improve the student experience. Our results suggest that the design of a flipped class should consider the effect of different implementation features on student experience in order to select the most appropriate ones for a particular context.