With only minimal changes to the course structure, classroom clickers were introduced in introductory chemistry to allow students to regularly compare their perceived abilities with their actual abilities, a measurement also known as calibration. Students used the clickers to provide knowledge judgments, an indication of their confidence in correctly answering questions, prior to actually answering the same assessment item. The clickers allowed responses from this exercise to be viewed by the entire class and permitted additional instruction when the perceived knowledge of the class did not match the actual performance. Overall, introducing this practice resulted in gains in student calibration accuracy from the beginning to the end of the semester. The improvements, however, were not equally distributed across all student groups. Students who performed in the top third of the class appeared to become more confident in their abilities, resulting in greater accuracy in their knowledge judgments. No significant gains were made in knowledge judgment accuracy by students in the bottom third of the class. This study indicates that practice alone does not support improved calibration for students at all ability levels.