Analyzing students' growth remains an important topic in educational research. Most recently, Diagnostic Classification Models (DCMs) have been used to track skill acquisition in a longitudinal fashion, with the purpose to provide an estimate of students' learning trajectories in terms of the change of fine-grained skills overtime. Response time (RT), the amount of time the test taker spends considering and answering each item, has been extensively studied and used in testing environments as a useful source of information to reflect individual response behavior and item characteristics. In this study, we consider using RTs in a learning environment to model students' learning progress. This could provide additional diagnostic information on students' fluency of applying the mastered skills. We propose a framework to model changes in RTs with a higher-order hidden Markov DCM. The proposed models are evaluated through a computer-based learning system that is designed to improve students' spatial skills. Results indicate that the proposed model can demonstrate both within and between group differences in learning through the predicted growth of latent speed on different items.