ERIC Number: EJ1124143
Record Type: Journal
Publication Date: 2017
Abstractor: As Provided
Diagnosing University Students' Academic Writing in English: Is Cognitive Diagnostic Modelling the Way Forward?
Educational Psychology, v37 n1 p26-47 2017
The study utilised a fine-grained diagnostic checklist to assess first-year undergraduates in Hong Kong and evaluated its validity and usefulness for diagnosing academic writing in English. Ten English language instructors marked 472 academic essays with the checklist. They also agreed on a Q-matrix, which specified the relationships among the checklist items and five writing subskills. This conceptual Q-matrix was refined iteratively via fitting a psychometric model (i.e. the reduced reparameterised unified model) to empirical data (i.e. the checklist marks) through the computer program "Arpeggio Suite." The final Q-matrix was found to be valid and useful; it had far fewer parameters but greater power to discriminate masters and non-masters of academic writing skills. The study found that the cognitive diagnostic model (CDM)-based skill diagnosis could identify the strengths and weaknesses in the five writing subskills for students across three proficiency levels and could provide richer and finer information than the traditional raw-score approach. However, limitations and caveats of the CDM approach were also observed, which warrant future investigations of its application in assessing writing.
Descriptors: Academic Discourse, College Students, College English, Foreign Countries, Language Teachers, Essays, Language Tests, Freshman Composition, Check Lists, College Freshmen, Writing Skills, Diagnostic Tests, Matrices, English (Second Language), Second Language Learning, Statistical Analysis, Comparative Analysis, Markov Processes, Monte Carlo Methods, Item Response Theory, Q Methodology, College Faculty
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A
Identifiers - Location: Hong Kong
Identifiers - Assessments and Surveys: Test of English as a Foreign Language