The transition from school to university mathematics is a challenging process for many students. This phenomenon is reflected by high dropout rates from mathematics programs especially during the first year at university that may be related to the development of students' mathematical interest and self-concept. Taking a learning psychological perspective, it is a relevant question if all students have similar development patterns of these variables or if students who are at risk to drop out show an unfavorable development. Moreover, for didactical issues, it is of particular relevance which factors influence this development. Within a longitudinal study, 556 first-semester students stated their interest and self-concept at the beginning and the middle of the first semester; dropout was measured at the beginning of the second year. By using regression analyses, we found that the development of students' interest and self-concept is related to dropout. It seems that interest in university mathematics and mathematical self-concept of students, who dropped out, decreased significantly in contrast to the characteristics of students who continued their studies. In an exploratory follow-up study, we identified factors which probably influenced the development of students' self-concept by asking 18 students in a first-semester mathematics course "linear algebra" about learning situations in which they experienced the emergence or the decline of self-concept. Using qualitative content analysis, we found that most of these situations occurred when students worked on mathematical homework tasks or got criterial feedback on their solutions. Based on the presented results, we discuss implications for teaching and learning of undergraduate mathematics.