This article employs the Common European Framework Reference for Language Acquisition (CEFR) as a basis for evaluating writing in the context of machine scoring. The CEFR was designed as a framework for evaluating proficiency levels of speaking for the 49 languages comprising the European Union. The intent was to impact language instruction so that "mastery" of one language has the same meaning as it does in another. A second objective is to provide a crosswalk for what one automated writing evaluation (AWE) system does in attending to the dimensions of the framework. The CEFR Framework is divided into five traits and different proficiency levels. The question then becomes: Does the AWE system attempt to measure these dimensions of writing? And, if so, how is this operationalized? Is it measuring aspects of communication that are not specified? The goal here is to create a common vocabulary between the writing community and those interested in AWE systems as to what is actually being measured by their software, and mapping that to a developmental scale of writing performance.