This study examines the differences in students' critical thinking ability through the Watson-Glaser Online Assessment software in a pretest-posttest setting. There is a noticeable increase in the categories of Recognize Assumptions and Evaluate Arguments and a relatively large decrease in the category of Draw Conclusions. In general, students from Non-STEM majors benefits more from teaching with technology than those in STEM majors.