The Saudi National Assessment Centre administers the Computer Science Teacher Test for teacher certification. The aim of this study is to explore gender differences in candidates' scores, and investigate dimensionality, reliability, and differential item functioning using confirmatory factor analysis and item response theory. The confirmatory factor analysis results for 6 371 examinees' scores of 66 multiple-choice items when grouped into three content domains showed that the test data were unidimensional (ability, trait). The domains were highly correlated (0.883 to 0.949) within this dimension. Data reliability estimated through latent variable modelling was acceptable at 0.848. Gender results for DIF signalled 13 items, five cases against males and eight cases against females; a finding of some balance in DIF direction against males and females. The study results confirm the validity of the Computer Science Teacher Test and support further refinement of multiple forms of the test.