When collecting data, science education researchers frequently have multiple respondents evaluate multiple artifacts using multiple criteria. Herein, the authors introduce Multifaceted Rasch Measurement (MFRM) analysis and explain why MFRM must be used when "judges'" data are collected. The authors use data from elementary science methods students enrolled in a senior-level, semester-length course. Acting as judges, students used the 23-item Elementary Science Teaching Analysis Matrix (ESTAM) instrument to evaluate five separate science lessons taught by five different teachers. Our MFRM analysis documents that multiple judges need not act as identical robots. Rather, differences in judges' severity can be computed and corrections made in an analysis to take into consideration differing judge severity. This paper extends the current literature in "Science Education" regarding the application of Rasch measurement.