Background: Promising health interventions tested in pilot studies will only achieve population-wide impact if they are implemented at scale across communities and health systems. Scaling up effective health interventions is vital as not doing so denies the community the most effective services and programmes. However, there remains a paucity of practical tools to assess the suitability of health interventions for scale-up. The Intervention Scalability Assessment Tool (ISAT) was developed to support policy-makers and practitioners to make systematic assessments of the suitability of health interventions for scale-up. Methods: The ISAT was developed over three stages; the first stage involved a literature review to identify similar tools and frameworks that could be used to guide scalability assessments, and expert input to develop draft ISAT content. In the second stage, the draft ISAT tool was tested with end users. The third stage involved revising and re-testing the ISAT with end users to further refine the language and structure of the final ISAT. Results: A variety of information and sources of evidence should be used to complete the ISAT. The ISAT consists of three parts. Part A: ‘setting the scene’ requires consideration of the context in which the intervention is being considered for scale-up and consists of five domains, as follows: (1) the problem; (2) the intervention; (3) strategic/political context; (4) evidence of effectiveness; and (5) intervention costs and benefits. Part B asks users to assess the potential implementation and scale-up requirements within five domains, namely (1) fidelity and adaptation; (2) reach and acceptability; (3) delivery setting and workforce; (4) implementation infrastructure; and (5) sustainability. Part C generates a graphical representation of the strengths and weaknesses of the readiness of the proposed intervention for scale-up. Users are also prompted for a recommendation as to whether the intervention (1) is recommended for scale-up, (2) is promising but needs further information before scaling up, or (3) does not yet merit scale-up. Conclusion: The ISAT fills an important gap in applied scalability assessment and can become a critical decision support tool for policy-makers and practitioners when selecting health interventions for scale-up. Although the ISAT is designed to be a health policy and practitioner tool, it can also be used by researchers in the design of research to fill important evidence gaps.
9 October 2020 in Health Research Policy and Systems
Background: While known efficacious preventive health interventions exist, the current capacity to scale up these interventions is limited. In recent years, much attention has focussed on developing frameworks and methods for scale-up yet, in practice, the pathway for scale-up is seldom linear and may be highly dependent on contextual circumstances. Few studies have examined the process of scaling up from decision to implementation nor examined the sustainability of scaled-up interventions. This study explores decision-makers’ perceptions from real-world scaled-up case studies to examine how scale-up decisions were made and describe enablers of successful scale-up and sustainability. Methods: This qualitative study included 29 interviews conducted with purposively sampled key Australian policy-makers, practitioners and researchers experienced in scale-up. Semi-structured interview questions obtained information regarding case studies of scaled-up interventions. The Framework Analysis method was used as the primary method of analysis of the interview data to inductively generate common and divergent themes within qualitative data across cases. Results: A total of 31 case studies of public health interventions were described by interview respondents based on their experiences. According to the interviewees’ perceptions, decisions to scale up commonly occurred either opportunistically, when funding became available, or when a deliberate decision was made and funding allocated. The latter scenario was more common when the intervention aligned with specific political or strategic goals. Decisions to scale up were driven by a variety of key actors such as politicians, senior policy-makers and practitioners in the health system. Drivers of a successful scale-up process included good governance, clear leadership, and adequate resourcing and expertise. Establishing accountability structures and appropriate engagement mechanisms to encourage the uptake of interventions were also key enablers. Sustainability was influenced by evidence of impact as well as good acceptability among the general or target population. Conclusions: Much like Kingdon’s Multiple Streams Theory of ‘policy windows’, there is a conceptually similar ‘window for scale-up’, driven by a complex interplay of factors such as political need, strategic context, funding and key actors. Researchers and policy-makers need to consider scalability from the outset and prepare for when the window for scale-up opens. Decision-makers need to provide longer term funding for scale-up to facilitate longer term sustainability and build on the resources already invested for the scale-up process.
31 January 2020 in Health Research Policy and Systems
Background: Cross-sector collaborative partnerships are a vital strategy in efforts to strengthen research-informed policy and practice and may be particularly effective at addressing the complex problems associated with chronic disease prevention. However, there is still a limited understanding of how such partnerships are implemented in practice and how their implementation contributes to outcomes. This paper explores the operationalisation and outcomes of knowledge mobilisation strategies within the Australian Prevention Partnership Centre — a research collaboration between policy-makers, practitioners and researchers. Methods: The Centre’s programme model identifies six knowledge mobilisation strategies that are hypothesised to be essential for achieving its objectives. Using a mixed methods approach combining stakeholder interviews, surveys, participant feedback forms and routine process data over a 5-year period, we describe the structures, resources and activities used to operationalise these strategies and explore if and how they have contributed to proximal outcomes. Results: Results showed that Centre-produced research, resources, tools and methods were impacting policy formation and funding. Policy-makers reported using new practical methodologies that were helping them to design, implement, evaluate and obtain funding for scaled-up policies and programmes, and co-creating compelling prevention narratives. Some strategies were better implemented and more impactful than others in supporting these outcomes, with variation in who they worked for. The activities used to effect engagement, capacity-building and partnership formation were mostly generating positive results, but co-production could be enhanced by greater shared decision-making. Considerably more work is needed to successfully operationalise knowledge integration and adaptive learning. Conclusions: Describing how collaborative cross-sector research partnerships are operationalised in practice, and with what effects, can provide important insights into practical strategies for establishing and growing such partnerships and for maximising their contributions to policy. Findings suggest that the Centre has many strengths but could benefit from more inclusive and transparent governance and internal processes that facilitate dialogue about roles, expectations and co-production practices.
6 August 2021 in Translational Journal of the American College of Sports Medicine
Scientific publications on physical activity (PA) have proliferated in recent years. Previous work has shown that descriptive studies dominate the literature with disappointingly fewer studies reporting on interventions or dissemination. This study updates trends in PA research to determine changes, if any, in publications that are more likely to contribute to facilitating research translation to scale-up interventions that might influence population levels of PA. A bibliometric review of a random sample of published PA studies (n = 4069) across three periods (2008–2009, 2013–2014, and 2017–2018) was conducted using the SCOPUS database. A four-stage coding strategy was used, including 1) identifying PA as exposure or primary outcome, 2) classifying PA studies into data-based and non–data-based studies, 3) classifying data-based PA studies into intervention and nonintervention studies, and 4) classifying intervention studies into six research translation categories. Descriptive PA studies dominated in all three time periods (52.1%–61.9%). Intervention studies increased only slightly from 14.8% to 19.2%. Within intervention studies, most focused on testing efficacy or effectiveness, and these increased over time across the time periods from 38% to 50.5%. Research relevant for scale-up such as replication and dissemination studies declined from the first to the most recent period (28.1% to 16.7%). The prevailing trend of descriptive studies and efficacy and/or effectiveness studies on a small scale continues in published PA literature, contrary to the increasing recognition of the need for scale-up and dissemination research. More dissemination research is required to translate known efficacious interventions to population-level implementation. There is a need for cultural shifts in research where real-world impact on population health should be valued more than the volume of publications.
30 June 2020 in Public Health Research and Practice
Objectives: This study tested the utility of the Intervention Scalability Assessment Tool (ISAT) with real-world interventions considered for scale-up in the Australian context. We explored user perceptions of the usefulness of the ISAT in making scalability decisions and conducted an analysis of completed ISATs. Type of program/service: The ISAT was designed to assist policy makers/practitioners to make structured and reflective assessments of the potential scalability of their interventions. Methods: Participants (n = 26) were invited via email and needed to be considering a population intervention for scale-up, to be able to pilot the ISAT, and to be willing to complete an online survey to provide feedback. Survey data and completed ISATs were collated and common responses were identified, with exceptions noted. Results: Thirteen participants accepted the invitation to pilot the ISAT; six completed ISATs and returned surveys and five of each were included in the final analysis. The process by which participants completed the ISAT varied, some involving up to six stakeholders in the process and others only one. The time taken to complete the ISAT ranged from 2 hours to 3 days. Interventions considered for scalability were public health interventions and all but one involved a potential statewide scale-up. In terms of outcome, scalability scores across interventions varied, however, all but one intervention was recommended for scale-up. The remaining intervention was deemed to be promising but further information was required before assessing scalability. Lessons learnt: The ISAT was perceived as a useful tool to assess the scalability of real-world health programs, with participants noting only minor limitations and challenges. The ISAT can be applied to a variety of population health interventions and identifies strengths and weaknesses in the evidence base for scale-up. This study revealed that decisions to scale up are often made despite potential weaknesses highlighted through the ISAT. This is likely a reflection of the critically influential role of contextual factors in decisions to scale up, whereby decision makers may balance a variety of considerations, of which evidence of scalability is only one factor.