NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ787478
Record Type: Journal
Publication Date: 2008-Mar
Pages: 21
Abstractor: Author
Reference Count: 41
ISBN: N/A
ISSN: ISSN-0013-1881
Tales of the Expected: The Influence of Students' Expectations on Question Validity and Implications for Writing Exam Questions
Crisp, Victoria; Sweiry, Ezekiel; Ahmed, Ayesha; Pollitt, Alastair
Educational Research, v50 n1 p95-115 Mar 2008
Background: Through classroom preparation and exposure to past papers, textbooks and practice tests students develop expectations about examinations: what will be asked, how it will be asked and how they will be judged. Expectations are also involved in the automatic process of understanding questions. Where a question and a student's expectations do not match, the student may get the answer wrong for the wrong reasons. Purpose: This research investigates the effects of students' expectations in exams and how changes to questions may reduce threats to validity (i.e., help ensure that the assessment really measures what it is claimed to). Sample: The study involved 339 15- to 16-year-old students from four schools in England. Design and methods: Students sat a science test constructed from adapted versions of past GCSE (General Certificate of Secondary Education) examination questions. (GCSEs are UK national qualifications taken by most students at age 16 years.) Four of the test questions were included because students' expectations may adversely influence response behaviours. For some question sub-parts, a manipulated version that might reduce the influence of pre-existing expectations was constructed. Question versions were assembled into different test versions in a fully counterbalanced design. Ten students were subsequently interviewed. Statistical analyses, including Rasch analysis, were used to consider the impact of manipulations on student marks, question difficulty and the extent to which the questions differentiated appropriately between students of different ability. Results: For three of the questions the manipulation resulted in improvements to the mean scores (which were sometimes statistically significant) and Rasch analysis indicated that the manipulated versions were easier than the originals. For two of these questions there were also improvements in the correlation of scores with total test scores, indicating improved validity. Additionally, forone of these questions a problem with lack of discrimination was removed. With the fourth question there was only a small effect suggesting that, in some cases, students' expectations are too strong to be suppressed successfully during an exam. Conclusions: The manipulations of the questions often reduced potential threats to validity relating to students' expectations. The results of this and other research have led to a proposed framework of levels at which students' expectations can affect the examining process: at the level of the subject, the question and the sentence. Ways in which question writers can reduce negative effects of expectations are discussed. (Contains 10 figures and 3 notes.)
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals/default.html
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: United Kingdom (England)