The Many-Facet Rasch (MFR) Model is traditionally used to evaluate the quality of ratings on constructed response assessments; however, it can also be used to evaluate the quality of judgments from panel-based standard setting procedures. The current study illustrates the use of the MFR Model by examining the quality of ratings obtained from a standard setting employing the Multiple Yes-No (MYN) modified Angoff standard setting procedure to set the cuts on the Advanced Placement® (AP®) Environmental Science exam. The locations of the panelists, judged item difficulties, rounds, and performance standards using the MFR Model were examined. Results revealed evidence of acceptable quality of the ratings obtained from the panelists. Thus, the current study provides an important contribution of validity evidence for the AP Environmental Science standard setting, as well as an illustration of the utility of the MFR Model to evaluate standard setting judgments.