Steven Ross's (2005) recent paper empirically measuring the efficacy of formative assessment compared to the summative variety found that the former produced positive effects on some aspects of language learning. In this critique, it is contended that Ross's study is flawed because his two groups of learners, one receiving summative assessment and the other formative, are neither distinct enough to have statistical measures applied to them, nor can he exclude other factors that may have influenced the findings. In addition, his brief description of formative assessment when contrasted with his lengthy and complex statistical analysis presents a disturbing imbalance to readers. In an extension of this notion, broader comments are made about similar studies in applied linguistics that employ complex statistical tools in a rigorous fashion, yet have a less than rigorous regard for the assumptions and definitions upon which the foundations of these studies rest.