The implications of data from a review of ten years of the American Educational Research Journal (AERJ) indicating that random sampling is rare and that there is increased use of quasi-experimental designs lacking in random assignment are considered. It is suggested that tests of significance could be abandoned or at least placed in a subsidiary role as a test of one rival internal validity hypothesis (i.e., that chance caused the results) rather than being the center of data analysis. The use of a priori effect size criteria for evaluating the importance of research results, the greater use of measures of association, and more replications are advocated. A shift in emphasis from teaching graduate students in education how to do tests of significance to the examination of the theory behind inferential statistics could have a salutory effect on educational research. (RL)