Digital tools are not always used in the manner their designers had in mind. Therefore, it is not enough to assume that learning through CALL tools occurs in intended ways, if at all. We have studied the use of an enhanced word processor for writing essays in Swedish as a second language. The word processor contained natural language processing technology to display linguistic information such as part of speech while students wrote their texts. We investigated how students' real classroom activity when using the tool corresponded to that which was planned by their teacher. Our contribution is on two levels: (1) a real-world analysis of occasionally problematic technology and (2) an elaboration on research methodology for conducting classroom studies within the sociocultural perspective. Our findings show that use of the tool varied greatly between sessions and that paying close attention to the near-future proficiencies of students is paramount. It is therefore important to account for novelty effects in CALL and to provide guidance to teachers concerning the use of such tools in their pedagogical context. We also suggest further elaborations of sociocultural theory and method in order to capture learners' classroom activities.