ERIC Number: EJ1165874
Record Type: Journal
Publication Date: 2018-Jan
Abstractor: As Provided
Using Contribution Analysis to Evaluate the Impacts of Research on Policy: Getting to "Good Enough"
Riley, Barbara L.; Kernoghan, Alison; Stockton, Lisa; Montague, Steve; Yessis, Jennifer; Willis, Cameron D.
Research Evaluation, v27 n1 p16-27 Jan 2018
Assessing societal impacts of research is more difficult than assessing advances in knowledge. Methods to evaluate research impact on policy processes and outcomes are especially underdeveloped, and are needed to optimize the influence of research on policy for addressing complex issues such as chronic diseases. Contribution analysis (CA), a theory-based approach to evaluation, holds promise under these conditions of complexity. Yet applications of CA for this purpose are limited, and methods are needed to strengthen contribution claims and ensure CA is practical to implement. This article reports the experience of a public health research center in Canada that applied CA to evaluate the impacts of its research on policy changes. The main goal was to experiment with methods that were relevant to CA objectives, sufficiently rigorous for making credible claims, and feasible. Methods were "good enough" if they achieved all three attributes. Three cases on government policy in tobacco control were examined: creation of smoke-free multiunit dwellings, creation of smoke-free outdoor spaces, and regulation of flavored tobacco products. Getting to "good enough" required careful selection of nested theories of change; strategic use of social science theories, as well as quantitative and qualitative data from diverse sources; and complementary methods to assemble and analyze evidence for testing the nested theories of change. Some methods reinforced existing good practice standards for CA, and others were adaptations or extensions of them. Our experience may inform efforts to influence policy with research, evaluate research impacts on policy using CA, and apply CA more broadly.
Descriptors: Foreign Countries, Evaluation Methods, Public Health, Research and Development Centers, Public Policy, Smoking, Theories, Statistical Analysis, Qualitative Research, Research Utilization
Oxford University Press. Great Clarendon Street, Oxford, OX2 6DP, UK. Tel: +44-1865-353907; Fax: +44-1865-353485; e-mail: firstname.lastname@example.org; Web site: http://applij.oxfordjournals.org/
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Authoring Institution: N/A
Identifiers - Location: Canada
Grant or Contract Numbers: N/A