With the advent of Web 2.0 media, there is a greater prevalence of science misinformation available to the public. This issue is particularly problematic for novices who often believe that science in the media is factual and objective, even though an expected outcome of secondary education is to develop students' abilities to critically evaluate information. By conducting cognitive Think Alouds and retrospective interviews with high school students, this study examined (1) their stances towards flawed claims in a Web media article about vaccination, (2) types of background knowledge used to make their assessments, and (3) responses when asked directly to critique the claims. Results of qualitative coding indicated that students who were most critical of the claims based their evaluation on knowledge of appropriate scientific reasoning and literacy skills. In contrast, students who accepted the claims relied on novice-level content knowledge, or a flawed understanding of scientific reasoning. Lastly, some students initially accepted the claims became critical in retrospect when explicitly asked to critique the article during the interview phase. Findings from this investigation suggest a need for more opportunities for students to critique science information, and a greater curricular emphasis on teaching evaluation skills and knowledge of valid scientific reasoning.