Stereotype threat is frequently purported to be an important determinant of gender gaps in math. Unlike prior studies, which mostly occur in lab settings, I use data from the National Assessment of Educational Progress (NAEP)--a large, representative assessment of U.S. children--where through a design quirk, students are randomly assigned test blocks, some of which include gender prime questions while others do not. I exploit this natural field experiment by comparing the gender gap in math scores of students receiving primes to those who do not. I find that girls actually perform better relative to boys for some primes (stereotype reactance) and no worse for others. These findings suggest that stereotype priming effects are relevant outside of lab settings, and that consistent with findings from a companion lab experiment and other lab studies from the stereotype literature, the effects appear to depend on the exact phrasing of the primes. (Contains 10 notes, 6 figures, and 6 tables.)