As the use of web-based homework delivery and checking systems expands, we have greater need to evaluate how students engage with these systems and how changes in the systems influence student behavior. Reported here is the assessment of WeBWorK, an open-source web-based homework program used largely in postsecondary math and science courses, with recent extension to high school courses. WeBWorK draws from a large library of problems to generate individual assignments and provides immediate feedback about correctness of students' answers. WeBWorK allows entries in multiple choice, formula and numerical formats. The entries recorded by the system create an extensive database of student interactions with the system. In our two-part study, we (1) review 2387 student surveys about WeBWorK in three semesters across a range of mathematics courses and (2) test a model designed to classify student responses to system feedback, permitting a fine-grained analysis of the system's records at the keystroke level for a cohort of 96 Calculus I students. Based on this data, our goals are, first, to understand how students use the system and respond to its feedback; second, to analyze the impact of upgrades to the system interface on student users, and third, to evaluate the efficiency of training raters to classify student records. We find that the introduction of new features to the WeBWorK interface (improvements generated by our initial assessment) were welcomed by most students and reduced the overall number of submitted errors by half by the third year of the study. Importantly, the interface improvements reduced the number of errors that were not pedagogically useful. We also find that our error classification model, called here the Student Response Model, was a good fit for our student sample, as reflected by inter-rater reliability of three raters who coded the student records. In sum, our error classification model appears to be useful for analyzing formula entries. To make this assessment model widely available, we designed a system of exercises to train raters to classify student response files for WeBWorK and similar web-based systems.