In the field of educational assessment, the "validity of assessment" can be defined as the degree to which evidence and theory support the interpretations of test scores that are entailed by proposed uses of the test. Many concerns have arisen over what is termed the "consequential basis" of validity, the intended and unintended social consequences of testing. In this article, the authors review the long history of the Keju exam system, that became central in a government-orchestrated system of high-stakes employment testing, education testing, and test-driven education. The authors find that problems such as memorizing and reproducing model performances, focusing on test-taking skills, cheating, and psychopathological effects, are the same problems recurred throughout the ages. Rater effects were largely controlled through a number of procedural designs. However, repeated attempts to control other sources of irrelevance led to these problems mutating to different forms. The authors suggest that test theorists and psychometricians need to develop appropriate and realistic models of scoring or of the use of assessment results in which such chronic consequential factors are accepted as inevitable components of the score or of the system. From the perspective of policy decisions, it does not appear that the solution will be found in designing procedures to stop such chronic problems. Rather, it will most likely come from taking measures to minimize the stakes of tests. (Contains 49 footnotes.)