Purpose: This paper aims to explore academics' perspectives on the quality of appraisal evidence at a Chinese university. Design/methodology/approach: An online survey with both closed items and open-ended questions was distributed among all academics at the university (n = 1,538). A total of 512 responded to the questionnaire. The closed items were initially analysed using EXCEL and SPSS; the open-ended questions were thematically analysed. Findings: The academics believed that the quality of student survey and peer observation of teaching were affected by subjectivity and the lack of understanding of appraisal. Academics also suggested that appraisals should be contextualised and the approach standardised. The study suggests the need for training that informs and engages relevant stakeholders to ensure the rigour of appraisal. Originality/value: The study raises the issue of quality assurance regarding appraisal data from the perspective of academics. It is based on the collaborative effort of academics in Australia, China and New Zealand, with the support of the management staff at the case study university. The study informs both appraisers and academics of quality assurance issues in appraisal. It also contributes to the literature, in that it initiates dialogues between communities of practices through collective questioning on the quality and mechanisms of appraisal in tertiary education.