We compared three assessor types (self-assessors, peer-assessors, and teacher assessors) to determine whether they differed in the levels of severity they exercised when rating essays. We analyzed the ratings of 194 assessors who evaluated 188 essays that students enrolled in two state-run universities in Iran wrote. The assessors employed a…

This study investigated the use of automated essay scoring (AES) to identify at-risk students enrolled in a first-year university writing course. An application of AES, the "Criterion"[R] Online Writing Evaluation Service was evaluated through a methodology focusing on construct modelling, response processes, disaggregation, extrapolation,…

In this paper, we provide an overview of psychometric procedures and guidelines Educational Testing Service (ETS) uses to evaluate automated essay scoring for operational use. We briefly describe the e-rater system, the procedures and criteria used to evaluate e-rater, implications for a range of potential uses of e-rater, and directions for…

Automated Essay Scoring (AES) has garnered a great deal of attention from the rhetoric and composition/writing studies community since the Educational Testing Service began using e-rater[R] and the "Criterion"[R] Online Writing Evaluation Service as products in scoring writing tests, and most of the responses have been negative. While the…

This paper examines the construct measured by automated essay scoring (AES) systems. AES systems measure features of the text structure, linguistic structure, and conventional print form of essays; as such, the systems primarily measure text production skills. In the current state-of-the-art, AES provide little direct evidence about such matters…

In this paper, I describe the design and evaluation of automated essay scoring (AES) models for an institution's writing placement program. Information was gathered on admitted student writing performance at a science and technology research university in the northeastern United States. Under timed conditions, first-year students (N = 879) were…

This article offers one potential response to Yancey's (1999) call for a fourth wave of writing assessment able to capture writing development in all of its complexity. Based on an ecological perspective of literate development that situates students' growth as writers across multiple engagements with writing, including those outside of school,…

An increasing number of writing programs and assessments are employing writing-from-sources tasks in which reading and writing are integrated. The integration of reading and writing in such contexts raises a number of questions with regard to writers' use of sources in their writing, the functions these sources serve, and how proficiency affects…

Raters of Georgia's (USA) state-mandated college-level writing exam, which is intended to ensure a minimal university-level writing competency, are trained to grade holistically when assessing these exams. A guiding principle in holistic grading is to not focus exclusively on any one aspect of writing but rather to give equal weight to style,…

Supporting undergraduate students with their academic literacies has recently been a major focus in higher education in the UK. This paper explores the value of tutor mediation in the context of academic writing development among undergraduate business studies students in open and distance learning, following the dynamic assessment (DA) approach…

Writing is complex, and assessment of writing is equally complex, particularly when considering the need to measure outcomes at the institutional level while providing meaningful data that informs curriculum reform and supports learning at the discipline-level. Using a multi-layered assessment that incorporates standardized measures of writing…

This action research demonstrates the answer to this question: How can literacy professors provide effective training in evaluating writing to preservice graduate education students? The study examines writing assessment instruction in the context of a literacy course required of preservice teachers seeking secondary (7-12) certification in…

Mixed quantitative and qualitative methods were used to assess student attitudes toward the assessment criteria for higher-order critical and analytical thinking skills in writing-intensive curricula in first-year, general education courses. The courses varied in emphasis on critical thinking in the criteria used to assess writing. The analysis is…

Teachers' feedback on students' written work is an important aspect of pedagogy. However, theoretical views differ on what constitutes "good" feedback, both among applied linguists and academics in other disciplines. In-depth research needs to be carried out into the contextual difficulties of evaluating and assessing academic assignments, and the…

This paper examines the use of portfolios as pedagogical tools for developing academic writing. In particular, it considers the value of multi-drafting, where learners reflect on the learning of a text type as well as focusing on micro and macro aspects. The paper outlines a situated pedagogical approach, where students come to understand their…