Deputy Director, Centre for the Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer), School of Social Sciences, Cardiff University, Cardiff CF10 3BD, UK
Dr. Graham Moore
Deputy Director, Centre for the Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer), School of Social Sciences, Cardiff University, Cardiff CF10 3BD, UK
Process evaluation is an essential part of designing and testing complex interventions. New MRC guidance provides a framework for conducting and reporting process evaluation studies Attempts to tackle problems such as smoking and obesity increasingly use complex interventions. These are commonly defined as interventions that comprise multiple interacting components, although additional dimensions of complexity include the difficulty of their implementation and the number of organisational levels they target.1 Randomised controlled trials are regarded as the gold standard for establishing the effectiveness of interventions, when randomisation is feasible. However, effect sizes do not provide policy makers with information on how an intervention might be replicated in their specific context, or whether trial outcomes will be reproduced. Earlier MRC guidance for evaluating complex interventions focused on randomised trials, making no mention of process evaluation.2 Updated guidance recognised the value of process evaluation within trials, stating that it “can be used to assess fidelity and quality of implementation, clarify causal mechanisms and identify contextual factors associated with variation in outcomes.”3 However, it did not provide guidance for carrying out process evaluation. #### Summary points In 2010, a workshop funded by the MRC Population Health Science Research Network discussed the need for guidance on process evaluation.4 There was consensus that researchers, funders, and …
Implementing interventions with a previous evidence base in new contexts might be more efficient than developing new interventions for each context. Although some interventions transfer well, effectiveness and implementation often depend on the context. Achieving a good fit between intervention and context then requires careful and systematic adaptation. This paper presents new evidence and consensus informed guidance for adapting and transferring interventions to new contexts.
Guideline developers are increasingly dealing with more difficult decisions concerning whether to recommend complex interventions in complex and highly variable health systems. There is greater recognition that both quantitative and qualitative evidence can be combined in a mixed-method synthesis and that this can be helpful in understanding how complexity impacts on interventions in specific contexts. This paper aims to clarify the different purposes, review designs, questions, synthesis methods and opportunities to combine quantitative and qualitative evidence to explore the complexity of complex interventions and health systems. Three case studies of guidelines developed by WHO, which incorporated quantitative and qualitative evidence, are used to illustrate possible uses of mixed-method reviews and evidence. Additional examples of methods that can be used or may have potential for use in a guideline process are outlined. Consideration is given to the opportunities for potential integration of quantitative and qualitative evidence at different stages of the review and guideline process. Encouragement is given to guideline commissioners and developers and review authors to consider including quantitative and qualitative evidence. Recommendations are made concerning the future development of methods to better address questions in systematic reviews and guidelines that adopt a complexity perspective.
Feasibility studies are increasingly undertaken in preparation for randomised controlled trials in order to explore uncertainties and enable trialists to optimise the intervention or the conduct of the trial. Qualitative research can be used to examine and address key uncertainties prior to a full trial. We present guidance that researchers, research funders and reviewers may wish to consider when assessing or undertaking qualitative research within feasibility studies for randomised controlled trials. The guidance consists of 16 items within five domains: research questions, data collection, analysis, teamwork and reporting. Appropriate and well conducted qualitative research can make an important contribution to feasibility studies for randomised controlled trials. This guidance may help researchers to consider the full range of contributions that qualitative research can make in relation to their particular trial. The guidance may also help researchers and others to reflect on the utility of such qualitative research in practice, so that trial teams can decide when and how best to use these approaches in future studies.
Complex systems approaches to social intervention research are increasingly advocated. However, there have been few attempts to consider how models of intervention science, such as the UK’s Medical Research Council complex interventions framework, might be reframed through a complex systems lens. This article identifies some key areas in which this framework might be reconceptualized, and a number of priority areas where further development is needed if alignment with a systems perspective is to be achieved. We argue that a complex systems perspective broadens the parameters of ‘relevant’ evidence and theory for intervention development, before discussing challenges in defining feasibility in dynamic terms. We argue that whole systems evaluations may be neither attainable, nor necessary; acknowledgment of complexity does not mean that evaluations must be complex, or investigate all facets of complexity. However, a systems lens may add value to evaluation design through guiding identification of key uncertainties, and informing decisions such as timings of follow-up assessments.
10 September 2013 in Journal of Epidemiology and Community Health
Public health interventions aim to improve the health of populations or at-risk subgroups. Problems targeted by such interventions, such as diet and smoking, involve complex multifactorial aetiology. Interventions will often aim to address more than one cause simultaneously, targeting factors at multiple levels (eg, individual, interpersonal, organisational), and comprising several components which interact to affect more than one outcome.1 They will often be delivered in systems which respond in unpredictable ways to the new intervention.2 Recognition is growing that evaluations need to understand this complexity if they are to inform future intervention development, or efforts to apply the same intervention in another setting or population.1 Achieving this will require evaluators to move beyond a ‘does it work?’ focus, towards combining outcomes and process evaluation. There is no such thing as a typical process evaluation, with the term applied to studies which range from a few simple quantitative items on satisfaction, to complex mixed-method studies exploring issues such as the process of implementation, or contextual influences on implementation and outcomes. As recognised within MRC guidance for evaluating complex interventions, process evaluation may be used to ‘assess fidelity and quality of implementation, clarify causal mechanisms and identify contextual factors associated with variation in outcomes’.1 This paper briefly discusses each of these core aims for process evaluation, before describing current Medical Research Council (MRC) Population Health Sciences Research Network (PHSRN) funded work to develop guidance for process evaluations of complex public health interventions.