Much evidence shows that instruction that actively engages students with learning materials is more effective than traditional, lecture-centric instruction. These "active learning" models comprise an extremely heterogeneous set of instructional methods: they often include collaborative activities, flipped classrooms, or a combination of both. To date, it is unclear whether active learning is more effective if it combines collaboration support with flipped classroom methods. We conducted a quasi-experiment as part of an advanced general chemistry course with 413 undergraduate students. We tested whether active learning is more effective than traditional instruction if it includes collaboration support only or a combination of collaboration support and flipped classrooms. Further, we explored effects on students' attitude. Our results show that only the combination of collaboration support and flipped classroom methods led to significantly higher learning outcomes than traditional instruction. Furthermore, our results reveal potential negative effects of active learning interventions on student attitudes.