The purpose of this study was to investigate the effectiveness of an online automated evaluation and feedback system that assessed students' word processing assignments prepared with Microsoft Office Word. The participants of the study were 119 undergraduate teacher education students, 86 of whom were female and 32 were male, enrolled in different sections of Computer-I course taught at one of the major public universities in Istanbul, Turkey. A total of 52 and 67 participants were assigned to the control and experimental group, respectively. No statistically significant difference was found between the experimental and control group students' post-tests performance, self-efficacy perception and technology acceptance scores after the implementation in which the experimental group students used the online automated evaluation and feedback system to get feedback on their assignments, and the control group students didn't receive any feedback. However, the interview results showed that the experimental group students had positive experiences with the system such as contributions to their learning performance, high perceptions, easy use of the system and saving time for the assignments.