Student evaluations of teaching (SET) are used globally by higher education institutions for performance assessment of academic staff and evaluation of course quality. Higher education institutions commonly develop their own SETs to measure variables deemed relevant to them. However, "home-grown" SETs are rarely assessed psychometrically. One potential consequence of this limitation is that an invalidated instrument may not provide accurate information for the intended purposes. Moreover, in the absence of psychometric assessment, the students' voices collected by the SETs often fail to provide insight relative to their intended purpose. The present study evaluates a "home-grown" SET using a Rasch model and confirmatory factor analysis. Our results identified weaknesses in two areas: the rating categories and the number of items used to measure the intended constructs. Suggestions are provided to address these weaknesses. This work provides an additional tool set for critical analysis of SET that is generally applicable for a variety of institutions, including those in Asia.