ERIC Number: EJ1104781
Record Type: Journal
Publication Date: 2016
Abstractor: As Provided
Researchers' Perceptions of Statistical Significance Contribute to Bias in Health and Exercise Science
Buchanan, Taylor L.; Lohse, Keith R.
Measurement in Physical Education and Exercise Science, v20 n3 p131-139 2016
We surveyed researchers in the health and exercise sciences to explore different areas and magnitudes of bias in researchers' decision making. Participants were presented with scenarios (testing a central hypothesis with p = 0.06 or p = 0.04) in a random order and surveyed about what they would do in each scenario. Participants showed significant bias in deciding to randomize additional participants when p = 0.06 (24.5%) compared to when p = 0.04 (6.5%), ?[superscript 2] = 19.86, odds ratio (OR) = 13.5. Based on this bias to randomize additional participants, we present statistical simulations showing the cumulative negative effect this bias has on the false positive rate for a field. While it is important for researchers to have flexibility in their data-processing decisions, there is a cost/benefit trade-off to experimenter degrees of freedom and it is important that these decisions be unbiased. These data highlight the importance of a priori analysis plans in research.
Descriptors: Researchers, Attitudes, Statistical Significance, Bias, Research Methodology, Surveys, Vignettes, Randomized Controlled Trials, Comparative Analysis, Data Processing, Data, Hypothesis Testing, Sampling, Research Universities, Medical Schools, Foreign Countries, College Faculty, Higher Education, Department Heads, Online Surveys, Statistical Inference, Statistics, Statistical Analysis, Exercise Physiology, Health Sciences, Experiments, Simulation, Questionnaires, Error Correction
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A
Identifiers - Location: Canada; United States