A major aim of computer-assisted language learning (CALL) is to create computer environments that facilitate students' second language (L2) acquisition. To achieve this aim, CALL employs technological innovations to create novel types of language practice. Evaluations of the new practice types serve the important role of distinguishing effective practice environments from less effective environments, while simultaneously informing educational practices and second language acquisition (SLA) theory. Accordingly, evaluations of CALL systems necessarily deal with multiple criteria. Most researchers would probably agree that motivation is an important criterion in CALL evaluations: a system can provide sufficient L2 input and opportunities for L2 output, yet fail to be pedagogically effective if learners are unwilling to participate. Furthermore, knowledge of the motivational impact of practice can provide valuable context linking individual language learners, practice effort and learning outcomes. From the perspective of recent theoretical developments in L2 motivation theory, this paper surveys a representative sample of CALL system evaluations that include motivational impact. Our analysis suggests not only that CALL needs to do more to align its treatment of motivation with recent L2 motivation theories, but also that it is well positioned to do so. We find that: (1) few CALL studies treat motivation as it relates to practice as a dynamic variable; (2) behavioural practice logs are underexploited; and (3) very few evaluations take into account learners' individual interests and goals. Drawing on these and other findings, we suggest four new directions for developing the motivation dimension in CALL evaluations.