This study aimed to develop an automatic classification system, namely ACTIVE, for generating immediate and individualised feedback on students' reflective entries about their second language (L2) learning experiences. It also aimed to explore students' attitudes towards using the system to support the development of their reflective skills in L2 learning. A total of 466 undergraduate students took part in the study. One hundred and twenty-seven participants were involved in the development phase, where their reflective entries were manually annotated according to a classification framework for critical reflection on L2 learning, and the annotated entries were then used to develop the ACTIVE system. The remaining participants were asked to generate automated feedback reports on their reflective entries for improvement by using the system. To solicit their views towards the system, the participants were administered an online questionnaire and some of them were also invited to attend a semi-structured interview. The overall results indicate that the classification accuracy of the system is comparable to that of human annotators. They also suggest that both teacher and machine feedback types have strengths and limitations, highlighting the need to further explore the use of multi-channel, multi-layer feedback in improving students' reflective skills in L2 learning.