ERIC Number: EJ1124711
Record Type: Journal
Publication Date: 2016-Aug
Pages: 118
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2330-8516
EISSN: N/A
Who's on First? Gender Differences in Performance on the "SAT"® Test on Critical Reading Items with Sports and Science Content. Research Report. ETS RR-16-26
Chubbuck, Kay; Curley, W. Edward; King, Teresa C.
ETS Research Report Series, Aug 2016
This study gathered quantitative and qualitative evidence concerning gender differences in performance by using critical reading material on the "SAT"® test with sports and science content. The fundamental research questions guiding the study were: If sports and science are to be included in a skills test, what kinds of material are appropriate for fair and valid assessments? Does item type matter with regard to differential item functioning (DIF)? What factors can provide explanations for the different levels of difficulty, discrimination, and DIF exhibited? Do differences in students' interest in and familiarity with the material affect their performance? From data gathered during the two phases of our research, it appears that most sports and science material of the sort used in this study should be permissible in a skills test. Particularly in the context of a long passage, in which most information needed to answer an item is provided, very little material appears to be too "specialized." From participants' statements during cognitive interviews, it seems that interest in and/or familiarity with a subject has little impact on performance--despite previous research to the contrary. On the basis of our observations, we recommend the following: (a) use more than one type of statistic to evaluate differential item functioning; (b) consider the amount of context provided with an item rather than just the particular content of an item; and (c) when possible, use data rather than assumptions about gender bias to make decisions about test content that may generate differential item functioning.
Descriptors: College Entrance Examinations, Gender Differences, Critical Reading, Reading Tests, Test Items, Athletics, Sciences, Test Bias, Difficulty Level, Student Interests, Familiarity, Performance, Statistical Analysis, Qualitative Research, Interviews, Pretests Posttests
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research; Tests/Questionnaires
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: SAT (College Admission Test)
Grant or Contract Numbers: N/A