The development of the constraint-referenced analytics tool for monitoring algebra learning activities presented here came from the desire to firstly, take a more quantitative look at student responses in collaborative algebra activities, and secondly, to situate those activities in a more traditional introductory algebra setting focusing on procedural understanding. Procedural skill was analyzed by modeling the complexity of attempts to make equivalent transformations of algebraic expressions. The constraint-referenced analytics system uses log files of student inputs on a classroom network of handheld devices to measure success rate as students make attempts to replace one algebraic expression with another equivalent expression. The analytics engine produced psychometrically verifiable results. Moving averages of student performance revealed that when students experienced a period of struggle and persisted in attempting similar transformations, an apparent conceptual shift led to subsequent success. Students also responded to periods of struggle by switching to familiar tasks or choosing non-participation.