Research suggests that difficulty in making connections among three levels of chemical representations--macroscopic, submicroscopic, and symbolic--is a primary reason for student alternative conceptions of chemistry concepts, and computer modeling is promising to help students make the connections. However, no computer modeling-based assessment tools are currently available. This article introduces an approach to using Rasch measurement to develop a computer modeling-based instrument to assess students' conceptual understanding of matter. The instrument contained 15 multiple-choice and 3 constructed-response questions. Two pilot tests were carried out in high schools both in the United States and China. In pilot test I, the instrument was given to 112 students. The partial-credit Rasch model was applied to the data to examine item and test properties. The instrument was then revised and given to 403 students in pilot test II. The partial-credit Rasch model was applied again to the data, and noticeable improvements were achieved in technical qualities of both items and the instrument as a whole. This study demonstrates that using Rasch measurement to develop computer modeling-based measurement instruments in chemistry is promising. (Contains 10 tables and 5 figures.)