ERIC Number: EJ946836
Record Type: Journal
Publication Date: 2011
Pages: 25
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0895-7347
EISSN: N/A
Available Date: N/A
An Experimental Test of Student Verbal Reports and Teacher Evaluations as a Source of Validity Evidence for Test Development
Leighton, Jacqueline P.; Heffernan, Colleen; Cor, M. Kenneth; Gokiert, Rebecca J.; Cui, Ying
Applied Measurement in Education, v24 n4 p324-348 2011
The "Standards for Educational and Psychological Testing" indicate that test instructions, and by extension item objectives, presented to examinees should be sufficiently clear and detailed to help ensure that they respond as developers intend them to respond (Standard 3.20; AERA, APA, & NCME, 1999). The present study investigates the use of verbal reports, one of many sources of evidence for validity arguments, as a way to evaluate the content clarity of 30 items from a large-scale science assessment. Student reports were used to edit items and create a "student-modified" test form. Evaluations from expert preservice teachers were used to edit the items and create an "expert-modified" test form. Both experimental forms, along with the original set of 30 items, were then randomly assigned to a sample of 264 examinees. Hierarchical regression analyses indicated that examinee performance on the student-modified and expert-modified forms was similar relative to performance on the original test items. Item statistics indicated that student-modified test items were equally difficult and discriminating as expert-modified test items. The implications of using student and teacher evaluations are discussed for informing test development. (Contains 3 footnotes and 8 tables.)
Descriptors: Test Construction, Validity, Evidence, Science Tests, Test Items, High School Students, Junior High School Students, Preservice Teachers, Foreign Countries
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Secondary Education; Grade 11; Grade 8; High Schools; Junior High Schools; Middle Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Canada
Grant or Contract Numbers: N/A
Author Affiliations: N/A

Peer reviewed
Direct link
