This study intends to develop a standardized instrument for measuring classroom teaching and learning in secondary chemistry lessons. Based on previous studies and interviews with expert teachers, the progression of five quality levels was constructed hypothetically to represent the quality of chemistry lessons in Chinese secondary schools. The measurement instrument was revised from the Evaluation Scale of Effectiveness of Primitive System of Classroom Teaching (ESEPrSCT). 90 videotaped chemistry lessons were collected and measured to validate the instrument in the pilot and field stage. By means of Rasch modeling, the instrument consisting of 18 items with five response categories was finally validated in this study. The results provide the validity and reliability evidence for using this measurement instrument to assess the quality of chemistry lessons.