The central goal of this study was to create a new diagnostic tool to identify organic chemistry students' alternative conceptions related to acid strength. Twenty years of research on secondary and college students' conceptions about acids and bases has shown that these important concepts are difficult for students to apply to qualitative problem solving. Yet, few published studies document how students' prior knowledge of acids influences their understanding of acid strength in organic chemistry contexts. We developed a nine-item multiple-tier, multiple-choice concept inventory to identify alternative conceptions that organic chemistry students hold about acid strength, to determine the prevalence of these conceptions, and to determine how strongly these conceptions bias student reasoning. We identified two significant alternative conceptions that organic chemistry students hold about acid strength. Students who answered items incorrectly were more confident about their answers than peers who answered items correctly, suggesting that after one semester of organic chemistry, students do not know what they do not know. Implications for the teaching of acid strength are discussed. (Contains 3 tables and 1 figure.)