Understanding secondary student attitudes towards science (SAS) is important if we are to increase the number and diversity of students who pursue STEM fields and address the gender imbalance in STEM careers. Existing studies that measured SAS tended to directly translate an ordinal scale (e.g., from 'strongly agree' to 'strongly disagree') into a set of linear scores (e.g. 5 for strongly agree and 1 for strongly disagree). We suggest that SAS constructs need to be calibrated on an interval scale in order to be meaningfully quantified. We propose the use of Rasch measurement where SAS constructs can be ranked according to their prevalence as SAS measurements. This paper implemented Rasch measurement to examine how Chinese boys and girls perceived different aspects of physics. A total of 495 boys and 490 girls from five secondary schools in Hong Kong participated in this study. An invariance analysis of SAS constructs important to boys and girls were found to have shown a similar hierarchy and that there are gender differences with respect to some but not all construct items. Main findings on how Asian boys and girls perceived physics, its implications and recommendations focusing on possible strategies to encourage girls to study physics are discussed.