Media literacy interventions offer promising avenues for the prevention of risky health behaviors among children and adolescents, but current literature remains largely equivocal about their efficacy. The primary objective of this study was to develop and test theoretically-grounded measures of audiences' degree of engagement with the content of media literacy programs based on the recognition that engagement (and not participation per se) can better explain and predict individual variations in the effects of these programs. We tested the validity and reliability of a measure of engagement with two different samples of 10th grade high school students who participated in a pilot and actual test of a brief media literacy curriculum. Four message evaluation factors (involvement, perceived novelty, critical thinking, personal reflection) emerged and demonstrate acceptable reliability.