The Evidence-Based Reasoning Assessment System (EBRAS) brings together advances in modeling scientific reasoning and assessment design to guide the development of written assessment items that target, disentangle, and elicit evidence of the multiple proficiencies underlying scientific argumentation. In this study, the EBRAS was used to assess the conceptual sophistication, specificity, and validity of middle and high school students' reasoning on the topic of buoyancy. Results imply items designed using the EBRAS are more reliable in formative contexts and can better reveal misconceptions, rhetorical strategies, and specific logical errors than traditional items that focus on accuracy. The EBRAS is proposed as a more valid, reliable, and useful means for analyzing and assessing students' use of evidence in scientific reasoning. (Contains 10 figures and 12 tables.)