NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1255414
Record Type: Journal
Publication Date: 2015-Apr
Pages: 31
Abstractor: As Provided
ISSN: ISSN-1069-4730
What Is the State of the Art of Systematic Review in Engineering Education?
Borrego, Maura; Foster, Margaret J.; Froyd, Jeffrey E.
Journal of Engineering Education, v104 n2 p212-242 Apr 2015
Background: Systematic review refers to a growing set of research methodologies intended to critically appraise and synthesize research to inform policy and practice. Systematic review methods, widely used in fields such as medicine, psychology, and education, are being applied in engineering education. Analysis of current systematic review practices in engineering education can contribute to continuous improvement of an important set of synthesis methodologies. Purpose: To promote continuous improvement in systematic reviews, researchers are currently developing criteria to evaluate the quality of systematic review methodologies. The purpose of this article is to identify prior systematic reviews in areas of interest to the engineering education community, use these criteria to critique these systematic reviews, and make recommendations for improving use and quality of systematic review methods in engineering education research. Scope/Method: Using systematic review search methodologies, we identified 12 conference papers and 37 journal articles that are systematic reviews published since 1990. Two reviewers coded each by topic, education level, quantitative or qualitative sources and synthesis methods, and 18 quantitative measures of review quality. Interrater reliability was calculated using kappa as described by Landis and Koch, which ranged from 0.43 (moderate) to 0.80 (almost perfect) with overall agreement of 0.64 (substantial). Conclusions: We found that the 49 identified systematic reviews in engineering education are particularly strong in stating their purposes, rationale, and inclusion criteria. They can be improved by explaining reasons for excluding studies, appraising quality of included studies, engaging multiple coders in evaluation procedures, stating their limitations, and, in the case of meta-analyses, describing statistical methods in greater detail.
Wiley Periodicals, Inc. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF)
Authoring Institution: N/A
Grant or Contract Numbers: 1446369; 1446319