Test-retest reliability assessments rarely investigate whether reliability itself is stable or change in reliability affects findings from substantive models. Research across the social sciences often recognizes that measurement error could influence results, yet it rarely applies established error correction methods. Focusing on gender wage inequality, we address two questions. First, to what extent does reliability vary over time, across genders and across measurement protocols? Second, does correcting for measurement error influence substantive conclusions about gender wage inequality? Comparing British and Australian panel data, we find little temporal variability in reliability; however, measurement error effects are variable and sometimes substantial.