In the past couple of decades, the geoscience education community has made great strides toward investigating how to provide effective student learning experiences in the college setting. While experiences such as student-centered teaching strategies and course design elements are useful for the instructor, they may not make important elements of the learning process itself explicit to the student. As a result, students may navigate a course without timely and necessary feedback related to their learning progress. To help remedy this issue, we designed and developed the Confidence-based Learning Accuracy Support System (CLASS) to provide students explicit feedback related to their mastery of geology content and the accuracy of their perceptions of their abilities. CLASS leverages robust evidence from education psychology regarding student metacognition and self-regulated learning (SRL). We investigated the relationship between students' judgments of their performance and their actual performance during summative exams in an introductory physical geology course. This study collected student confidence data for every question of students' midterm exams and compared this confidence to performance via multiple empirically-derived measures of the disparity between students' perceptions of their performance and their actual performance. In addition to exam-based data, we developed and provided CLASS quizzes (with varying requirements) to provide students with feedback regarding their learning and accuracy during the target courses. Results indicated that students utilizing CLASS performed better than their predecessors for the first two exams and were generally more accurate in their approximations. Overall, results provide support for CLASS's potential to serve as a tool for increasing student metacognitive awareness, self-regulation and performance in undergraduate geoscience courses.