ERIC Number: EJ1111572
Record Type: Journal
Publication Date: 2007-Mar
Pages: 43
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2330-8516
EISSN: N/A
Small-Sample DIF Estimation Using Log-Linear Smoothing: A SIBTEST Application. Research Report. ETS RR-07-10
Puhan, Gautam; Moses, Tim P.; Yu, Lei; Dorans, Neil J.
ETS Research Report Series, Mar 2007
The purpose of the current study was to examine whether log-linear smoothing of observed score distributions in small samples results in more accurate differential item functioning (DIF) estimates under the simultaneous item bias test (SIBTEST) framework. Data from a teacher certification test were analyzed using White candidates in the reference group and African American candidates in the focal group. Smoothed and raw DIF estimates from 100 replications under seven different sample-size conditions were compared to a criterion to determine the effect of smoothing on small-sample DIF estimation. Root-mean-squared deviation and bias were used to evaluate the accuracy of DIF detection in the smoothed versus raw data conditions. Results indicate that, for most studied items, smoothing the raw score distributions reduced variability and bias of the DIF estimates especially in the small-sample-size conditions. Implications of these results for actual testing programs and future directions for research are discussed.
Descriptors: Test Bias, Computation, Sample Size, Accuracy, Teacher Certification, Licensing Examinations (Professions), Whites, African Americans, Comparative Analysis
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A