Science literacy involves knowledge of both science content and science process skills. In this study, we describe the Assessment of Critical Thinking Ability survey and its preliminary application to assess the critical thinking skills of undergraduate students, graduate students, and postdoctoral fellows. This survey is based on a complex and partially conflicting data set drawn from the medical literature of the early 20th century. Several open-response questions ask subjects to synthesize the data to a single conclusion, propose studies to increase confidence in the conclusion, and ask if other conclusions are possible. Their responses to each question are scored on a 4-level scale in terms of their ability to deal with the complexity and conflicts in the data. We found a significant increasing trend in these skills with increased academic level as well as significant room for improvement. (Contains 2 tables and 2 figures.)