NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1101829
Record Type: Journal
Publication Date: 2016-Apr
Pages: 22
Abstractor: As Provided
ISSN: ISSN-0265-5322
Extending the Scope of Speaking Assessment Criteria in a Specific-Purpose Language Test: Operationalizing a Health Professional Perspective
O'Hagan, Sally; Pill, John; Zhang, Ying
Language Testing, v33 n2 p195-216 Apr 2016
Criticism of specific-purpose language (LSP) tests is often directed at their limited ability to represent fully the demands of the target language use situation. Such criticisms extend to the criteria used to assess test performance, which may fail to capture what matters to participants in the domain of interest. This paper reports on the outcomes of an attempt to expand the construct of a specific-purpose test through the inclusion of two new professionally relevant criteria designed to reflect the values of domain experts. The test in question was the speaking component of the Occupational English Test (OET), designed to assess the language proficiency of overseas-trained health professionals applying to practise their profession in Australia. The criteria were developed from analysis of health professionals' feedback to trainees, a source that reflected what the professionals value, that is, their indigenous assessment criteria. The criteria considered amenable to inclusion in the OET were as follows: (1) Clinician Engagement with the patient, and (2) Management of Interaction in the consultation. Seven OET assessors were trained to apply these professionally relevant criteria at a workshop that introduced a checklist derived from the original data analysis as a tool to aid understanding of the new criteria. Following the workshop, assessors rated a total of 300 pre-recorded OET speaking test performances using both new and existing criteria. Statistical analyses of the ratings indicate the extent to which (a) the judgements of the language-trained assessors using the new criteria were consistent, and (b) the new and existing criteria aligned in terms of the construct(s) they represent. Furthermore, feedback from the assessors in the process shows how comfortable and confident they are to represent a health professional perspective.
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research; Tests/Questionnaires
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Australia
Grant or Contract Numbers: N/A