This descriptive study aimed at finding the impact of Tell Me More (TMM), an online language-learning program, on English as a foreign language (EFL) undergraduate learners' achievement in a University in Thailand. The study also looked at whether the time of use of TMM had an effect on learners' achievement. Data was collected from the scores of students at four proficiency levels who did the placement, progress and achievement tests in the TMM program for the 2015 academic year. The analysis of the data indicated an improvement in English language achievement for the beginner and advanced proficiency levels after the use of the TMM program. However, TMM did not have any effect on students of intermediate+ and intermediate proficiency levels. The ANOVA and pairwise comparison analysis revealed a significant difference between the proficiency levels. The analysis of the time on task was striking. It raises concerns about the use of time as the sole indicator for assessment. The findings suggest that learning goals and assessment have the capacity to influence the use of computer-assisted language learning technologies. The study therefore will guide instructors on how to design curriculums for autonomous online learning and improve ways of assessment.