ERIC Number: EJ1138953
Record Type: Journal
Publication Date: 2017
Abstractor: As Provided
Reference Count: 17
Power of Statistical Tests Used to Address Nonresponse Error in the "Journal of Agricultural Education"
Johnson, Donald M.; Shoulders, Catherine W.
Journal of Agricultural Education, v58 n1 p300-312 2017
As members of a profession committed to the dissemination of rigorous research pertaining to agricultural education, authors publishing in the Journal of Agricultural Education (JAE) must seek methods to evaluate and, when necessary, improve their research methods. The purpose of this study was to describe how authors of manuscripts published in JAE between 2006 and 2015 tested for nonresponse error. Results indicated that none of the studies' tests had acceptable power to detect small effect sizes, 14.3% had acceptable power to detect medium effect sizes, and 43% of the studies' tests did not have acceptable power to detect large effect sizes. These findings suggest that while authors frequently find no difference between respondents and others, the tests used to detect these differences are often not powerful enough to do so, leading to higher than acceptable risk for Type II error. Using the theory of planned behavior as a framework, we highlight these findings to spur change within the profession's expectations of reporting statistical power when testing for nonresponse error and offer a primer to improve researchers' perceived behavioral control over reporting power. We also offer specific suggestions for conducting and reporting the results of tests for nonresponse bias.
Descriptors: Statistical Analysis, Agricultural Education, Effect Size, Risk, Journal Articles, Periodicals, Research Methodology, Guidelines, Statistical Bias, Error of Measurement, Authors
American Association for Agricultural Education. P.O. Box 7607, Department of Agricultural and Extension Education, Raleigh, NC 27695. Web site: http://www.aaaeonline.org
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Authoring Institution: N/A