In this quasi-experimental mixed methods study, we examined the effectiveness of a faculty-to-faculty mentoring program to increase student success rates in online courses at an American university. Over one semester, 24 faculty mentees worked with 6 faculty mentors on improving course organization and implementing student engagement techniques. Using methodological triangulation, we found positive results for two of our effectiveness measures (i.e., mentor and mentee feedback and mentors' ratings of mentees' courses). However, in terms of our third measure (i.e., student learning data), although institutional data showed that there was a 4.1% aggregate increase in online student success after the intervention, the intervention group made no significantly greater gains than two control groups. Our findings highlight some benefits and shortcomings of these types of university initiatives and also emphasize the importance of using triangulation to integrate participant feedback with formal measures of student success.