Graduate student instructors (henceforth, instructors) play a crucial role in teaching early STEM courses. Thus, professional development for these instructors addresses an urgent need to improve STEM student success. This paper focuses on a semester-long professional learning community in which six mathematics graduate student instructors engaged in regular cycles of peer observation, feedback, and reflection. In contrast to other professional development work, this approach emphasized that instructors give, not just receive, peer feedback. Analyses of post-semester interviews indicated that all instructors enhanced their noticing of students. The interviews also highlighted the challenges of providing critical, supportive feedback.