We share results from a quasi-experimental study in which we compared achievement between traditional face-to-face and computer-based sections of Intermediate Algebra on a common multiple choice exam as well as performance on open-response tasks. Students in the computer-based group performed better on the final exam and were also more likely to correctly complete the open-response tasks than students in the traditional sections. However, the computer-based group showed limited ability to interpret an equation and relate it to a contextual situation as compared with the face-to-face group. Students in both groups showed limited ability to write algebraic equations to represent contextual situations.