This article describes the rationale for and development of a computer-based instrument that helps identify commonly held science misconceptions. The instrument, known as the Science Beliefs Test, is a 47-item instrument that targets topics in chemistry, physics, biology, earth science, and astronomy. The use of an online data collection system aided in developing this instrument and in ascertaining its validity and reliability. Validity was also established through use of expert panels, previously published items, and feedback from pilot tests. Using KR-21, internal consistency was established at 0.77. A test-retest reliability coefficient was established at 0.776, or moderate. As of December 2005, 1,071 respondents participated in this study, including 17 college and university educators, 40 members of the general public, and 41 K-12 educators. Eighty-five graduate students, 254 K-12 students, and 634 undergraduates also took the survey. This instrument continues to be revised to clarify items and add others to further its usefulness.