ERIC Number: EJ974754
Record Type: Journal
Publication Date: 2012
Pages: 21
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1530-5058
EISSN: N/A
Methodologies for Investigating Item- and Test-Level Measurement Equivalence in International Large-Scale Assessments
Oliveri, Maria Elena; Olson, Brent F.; Ercikan, Kadriye; Zumbo, Bruno D.
International Journal of Testing, v12 n3 p203-223 2012
In this study, the Canadian English and French versions of the Problem-Solving Measure of the Programme for International Student Assessment 2003 were examined to investigate their degree of measurement comparability at the item- and test-levels. Three methods of differential item functioning (DIF) were compared: parametric and nonparametric item response theory and ordinal logistic regression. Corresponding derivations of these three DIF methods were investigated at the test-level to examine both differential test functioning (DTF) and the correspondence between findings at the item-level with those at the test-level. Item-level findings suggested consistency in DIF detection across methods; however, differences in effect sizes of DIF were found by each method. Test-level results revealed a high degree of consistency across DTF methods. Discrepancies were found between item- and test-level comparability analyses. Item-level analyses suggested moderate to low degrees of comparability, whereas test-level findings suggested a higher degree of comparability. Findings also indicated the direction of DIF was mixed as some DIF items favored English-speaking students and others favored French-speaking students, suggesting that DIF cancellation may explain why item-level incomparability was not detected at the test-level. (Contains 2 tables and 4 figures.)
Descriptors: Foreign Students, Test Bias, Speech Communication, Effect Size, French, Item Response Theory, Regression (Statistics), Reliability, Student Evaluation, Educational Assessment, Problem Solving, English, Comparative Analysis, Measurement, Statistical Significance, Scores
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Program for International Student Assessment
Grant or Contract Numbers: N/A