NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1112477
Record Type: Journal
Publication Date: 2015
Pages: 20
Abstractor: As Provided
ISSN: ISSN-1302-597X
A Study on Detecting of Differential Item Functioning of PISA 2006 Science Literacy Items in Turkish and American Samples
Çikirikçi Demirtasli, Nükhet; Ulutas, Seher
Eurasian Journal of Educational Research, n58 p41-60 2015
Problem Statement: Item bias occurs when individuals from different groups (different gender, cultural background, etc.) have different probabilities of responding correctly to a test item despite having the same skill levels. It is important that tests or items do not have bias in order to ensure the accuracy of decisions taken according to test scores. Thus, items should be tested for bias during the process of test development and adaptation. Items used in testing programs, such as the Program for International Student Assessment (PISA) study, whose results are inform educational policies throughout the participating countries, should be reviewed for bias. The study examines whether items of the 2006 PISA science literacy test, applied in Turkey, show bias. Purpose of the Study: The aim of this study is to analyze the measurement equality of the PISA science literacy test of 2006 in Turkish and American groups in terms of structural invariance and also determined whether the science literacy items show inter-cultural bias. Methods: The study included data for 15 year-old 757 Turkish and 856 American students. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) was performed to determine whether the PISA science literacy test was equivalent in measurement construct in both groups; multi group confirmatory factor analysis (MCFA) was used to identify differences in the factor structure according to cultures. Item bias was detected via the Mantel-Haenszel (MH), Simultaneous Item Bias Test (SIBTEST) and Item Response Theory Likelihood- Ratio Analysis (IRT-LR) procedures. Findings and Results: : According to the MCFA results PISA 2006 science literacy test for both Turkish and American groups showed equivalent measurement construct. Moreover, the three analyses methods agreed at B and C levels for 15 items in the Turkish sample and 25 items in the American sample in terms of DIF. According to expert opinions, common sources for item bias were: familiarity with item content and differing skill levels between cultures. Conclusions and Recommendations: The 38 items that showed DIF by each of the three methods were accepted as having DIF. The findings of the present study, possible source of bias in the items will not change the average level of student performance in participating countries. However, it will be beneficial that the review of item content before test administration, in order to reduce the errors items with DIF across different language and cultural groups in international comparative studies.
ANI Publishing Company. Kizilirmak Sokak 10/A Bakanliklar, Ankara, Turkey 06440. Tel: +90-312-425-8150; Fax: +90-312-425-8111; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Turkey; United States
Identifiers - Assessments and Surveys: Program for International Student Assessment
Grant or Contract Numbers: N/A