NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1105233
Record Type: Journal
Publication Date: 2016
Pages: 16
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0026-7902
EISSN: N/A
A Meta-Analysis of Reliability Coefficients in Second Language Research
Plonsky, Luke; Derrick, Deirdre J.
Modern Language Journal, v100 n2 p538-553 Sum 2016
Ensuring internal validity in quantitative research requires, among other conditions, reliable instrumentation. Unfortunately, however, second language (L2) researchers often fail to report and even more often fail to interpret reliability estimates beyond generic benchmarks for acceptability. As a means to guide interpretations of such estimates, this article meta-analyzes reliability coefficients (internal consistency, interrater, and intrarater) as reported in published L2 research. We recorded 2,244 reliability estimates in 537 individual articles along with study (e.g., sample size) and instrument features (e.g., item formats) proposed to influence reliability. We also coded for the indices employed (e.g., alpha, KR20). The coefficients were then aggregated (i.e., meta-analyzed). The three types of reliability varied, with internal consistency as the lowest: median = 0.82. Interrater and intrarater estimates were substantially higher (0.92 and 0.95, respectively). Overall estimates were also found to vary according to study and instrument features such as proficiency (low = 0.79, intermediate = 0.84, advanced = 0.89) and target skill (e.g., writing = 0.88 vs. listening = 0.77). We use our results to inform and encourage interpretations of reliability estimates relative to the larger field as well as to the substantive and methodological features particular to individual studies and subdomains.
Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Publication Type: Journal Articles; Information Analyses; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A