A growing body of literature in second-language writing suggests that the writing ability of international second language (L2) learners, who attend post-secondary education abroad after having completed high school in their home countries, and the so-called Generation 1.5 population, that is, L2 learners who enter post-secondary education after…

We compared three assessor types (self-assessors, peer-assessors, and teacher assessors) to determine whether they differed in the levels of severity they exercised when rating essays. We analyzed the ratings of 194 assessors who evaluated 188 essays that students enrolled in two state-run universities in Iran wrote. The assessors employed a…

In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for…

Automated Essay Scoring (AES) has garnered a great deal of attention from the rhetoric and composition/writing studies community since the Educational Testing Service began using e-rater[R] and the "Criterion"[R] Online Writing Evaluation Service as products in scoring writing tests, and most of the responses have been negative. While the…

This article presents considerations for using automated scoring systems to evaluate second language writing. A distinction is made between English language learners in English-medium educational systems and those studying English in their own countries for a variety of purposes, and between learning-to-write and writing-to-learn in a second…

This paper examines the construct measured by automated essay scoring (AES) systems. AES systems measure features of the text structure, linguistic structure, and conventional print form of essays; as such, the systems primarily measure text production skills. In the current state-of-the-art, AES provide little direct evidence about such matters…

This article offers one potential response to Yancey's (1999) call for a fourth wave of writing assessment able to capture writing development in all of its complexity. Based on an ecological perspective of literate development that situates students' growth as writers across multiple engagements with writing, including those outside of school,…

This article reviews key developmental theories that have been adopted by writing development researchers over the last fifty years. It describes how researchers have translated these theories into definitions of writing development capable of influencing curricular design and interpretations of student writing and explores the implications for…

Supporting undergraduate students with their academic literacies has recently been a major focus in higher education in the UK. This paper explores the value of tutor mediation in the context of academic writing development among undergraduate business studies students in open and distance learning, following the dynamic assessment (DA) approach…

This paper examines the use of portfolios as pedagogical tools for developing academic writing. In particular, it considers the value of multi-drafting, where learners reflect on the learning of a text type as well as focusing on micro and macro aspects. The paper outlines a situated pedagogical approach, where students come to understand their…

Following Messick's definition of validity as a multi-faceted construct that includes contextual, substantive, structural, generalizable, external, and consequential dimensions, this study examined an established directed self-placement (DSP) system that had been functioning for ten years at a large university. The goal was to determine the extent…

Measuring the Academic Skills of University Students is a procedure developed in the 1990s at the University of Sydney's Language Centre to identify students in need of academic writing development by assessing examples of their written work against five criteria. This paper reviews the literature relating to the development of the procedure with…

This article considers the claim that machine scoring of writing test responses agrees with human readers as much as humans agree with other humans. These claims about the reliability of machine scoring of writing are usually based on specific and constrained writing tasks, and there is reason for asking whether machine scoring of writing requires…

Existing comparative studies between peer and teacher feedback in English writing classes have predominantly used frequency measures of peer and teacher feedback in learners' revisions to suggest their relative values for developing learners' writing proficiency. However, learners do not necessarily understand the feedback that is used in their…

College and university instructors who assess student writing, including but not limited to teachers of first-year composition, sometimes keep the gate. Gatekeeping by writing instructors is an activity directly related to grading practices and, in many cases, to grading practices which emphasize formal, mechanical, usually sentence-level choices…