NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1110981
Record Type: Journal
Publication Date: 2009-Dec
Pages: 45
Abstractor: As Provided
ISSN: EISSN-2330-8516
Improved Reliability Estimates for Small Samples Using Empirical Bayes Techniques. Research Report. ETS RR-09-46
Oh, Hyeonjoo J.; Guo, Hongwen; Walker, Michael E.
ETS Research Report Series, Dec 2009
Issues of equity and fairness across subgroups of the population (e.g., gender or ethnicity) must be seriously considered in any standardized testing program. For this reason, many testing programs require some means for assessing test characteristics, such as reliability, for subgroups of the population. However, often only small sample sizes are available for the subgroups of interest. Traditionally used reliability estimates (e.g., Cronbach's alpha) can have low precision for small samples. This study investigated whether an empirical Bayes (EB) technique could produce more precise reliability estimates than traditional methods in the presence of small samples. Several Bayesian estimates were compared to estimates obtained by other methods (e.g., the traditionally and currently used Cronbach's alpha coefficient), in terms of both bias and variance. A secondary purpose of this study was to compare the various EB approaches across different sample sizes. This paper also discusses EB estimates of standard error of measurement (SEM), their accuracy and precision, and how they compare with SEM estimates derived from the alpha.
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: High Schools; Secondary Education; Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: National Merit Scholarship Qualifying Test; Preliminary Scholastic Aptitude Test
Grant or Contract Numbers: N/A