It is widely accepted that to be scientifically literate one needs to have the ability to make thoughtful decisions about socioscientific issues (SSI). This includes critical assessment of scientific claims and arguments involved. In this study we asked 89 science education students with substantial academic qualifications in science, working in groups of two and three, to assess the reliability of scientific claims in an article of their own choice, but related to a socioscientific issue, and to present their evaluation in a short text. In analyzing the students' texts, we focused on the criteria they had explicitly and implicitly used in their evaluations. Through a qualitative analysis, we identified 13 different criteria focusing on empirical and theoretical adequacy, completeness of presented information, social aspects, and manipulative strategies. An inspection of the students' evaluations revealed that they drew upon knowledge of possible institutional interests, different signs of competence and an appreciation of concurrent expert views, but also methodological norms in science, specialized content knowledge, and an appreciation of evidence and disclosure of sources. The number of criteria used and the quality of their application varied, indicating that critical examination of texts with a science dimension needs to be emphasized in science teacher education.