Electronic tutors able to respond appropriately to a user's input have been shown to be effective in improving learning in a number of contexts. This study extends this research into the context of conceptual change during in-service science teacher workshops. Quantitative data were collected from 1,049 South African grade 12 physical sciences teachers who attended 54 in-service teacher workshops held by the first author across all South African provinces and hosted by volunteer schools. These teachers completed pre- and post-tests about the force concept. The teachers from each workshop were assigned, by convenience, to one of three treatment groups. The individual group (n = 296) engaged with the software between answering the tests. The plenary group (559) was taught using the tutoring software as an aid. The control group (194) received no electronic tutoring intervention. Participants also answered questions about themselves and their schools. The findings suggest that individual use of the software can be effective in promoting conceptual change in the context of the in-service teacher workshop, especially for those who already have relatively high levels of prior knowledge. However, plenary use of this software, in this context, appears to have limited effectiveness in promoting conceptual change. The patterns of engagement indicate an evolutionary, slow and iterative process of conceptual change, with limited transfer of learning across contexts. The findings suggest that multiple short modules of varying levels of advancement are more advisable than fewer, longer, less varied modules.