1 January 2020 in Policy & Politics
The article explores the potential benefits to public policy of combining traditional evaluative inquiry with insights developed dynamically in policy labs. Twenty leading labs from five continents are critically analysed through a literature review as well as policy and programme evaluation practices, assessing the extent to which the purpose, structures and processes used in policy labs address three challenges: (1) establishing the causality and value of public interventions, (2) explaining mechanisms of change, and (3) utilising research findings in public policy. The article concludes that creating synergies between evaluation inquiry and policy labs can improve the design and implementation of public policy and programmes.
6 October 2020 in Journal of Comparative Policy Analysis: Research and Practice
The article provides a comparative framework for articulating assumptions made during the policy design process. That includes the framing of the policy issue in terms of the behavior of addressees, identifying problems that obstruct compliance, and choosing a tool with a distinctive change mechanism. Based on this discussion, a spectrum of six generic policy tools have been provided and illustrated with examples: (1) equipping policy subjects to perform behavior, (2) banning misbehavior, (3) dis/incentivizing addressees to behave in a certain way, (4) informing to raise the awareness about need for compliance, (5) boosting, and (6) nudging towards desired behavior.
22 April 2020 in American Journal of Evaluation
Evaluation professionals need to be nimble and innovative in their approaches in order to be relevant and provide useful evidence to decision-makers, stakeholders, and society in the crowded public policy landscape. In this article, we offer serious games as a method that can be employed by evaluators to address three persisting challenges in current evaluation practice: inclusion of stakeholders, understanding of causal mechanisms, and utilization of evaluation findings. We provide a framework that distinguishes among games along two crucial aspects of evaluation inquiry - its function and the nature of the evaluand. We offer examples of successfully implemented games in each set of the four arenas we delineate: teaching knowables, testing retention, crash-testing mechanisms, and exploring systems. We explain how games can be employed to promote learning about and among stakeholders, and to collect valuable intelligence about the operations of programs and policies.
20 April 2016 in Evaluation
Evaluation units, located within public institutions, are important actors responsible for the production and dissemination of evaluative knowledge in complex programming and institutional settings. The current evaluation literature does not adequately explain their role in fostering better evaluation use. The article offers an empirically tested framework for the analysis of the role of evaluation units as knowledge brokers. It is based on a systematic, interdisciplinary literature review and empirical research on evaluation units in Poland within the context of the European Union Cohesion Policy, with complementary evidence from the US federal government and international organizations. In the proposed framework, evaluation units are to perform six types of brokering activities: identifying knowledge users’ needs, acquiring credible knowledge, feeding it to users, building networks between producers and users, accumulating knowledge over time and promoting an evidence-based culture. This framework transforms evaluation units from mere buyers of expertise and producers of isolated reports into animators of reflexive social learning that steer streams of knowledge to decision makers.
14 September 2017 in American Journal of Evaluation
Knowledge brokering is a promising practice for addressing the challenge of using research evidence, including evaluation findings, in policy implementation. For public policy practitioners, it means playing the role of an intermediary who steers the flow of knowledge between producers (researchers) and users (decision makers). It requires a set of specific skills that can be learnt most effectively by experience. The article explains how to develop knowledge brokering skills through experiential learning in a risk-free environment. It reports on the application of an innovative learning method—a game-based workshop. The article introduces the conceptual framework for designing game-based learning. Then, it demonstrates how this framework was applied in practice of teaching knowledge brokering. In conclusion, article discusses the first lessons of game application in training public policy practitioners. It concludes that game-based workshop is a promising method for learning about knowledge brokering strategies that increase evaluation use.
3 July 2018 in Policy Design and Practice
Successful policy solutions rely on policy addressees responding in certain ways. Policy designers need an analytical method that allows them to anticipate impact of a new intervention, while taking into account bounded rationality of policy actors and sociopolitical complexity. The article proposes using serious games at the stage of policy formulation to test the architecture of a new regulation in a safe environment. It provides a blueprint for using games in policy design, consisting of conceptual framework, design procedure, and techniques for strengthening game validity. The application is illustrated with an example of a draft regulation on rural transport in Poland. The case study points out three advantages of game method: (1) revealing mechanisms triggered by the architecture of regulation, meaning actors’ initial assumptions, decisions, and feedback loops created by actors’ responses, (2) demonstrating the consequences of mechanisms over time, that in real life would occur with a long delay, and (3) creating a risk-free environment where policy actors can verify their assumptions and experiment with ways of interpreting and responding to new regulation. The article concludes that serious games are a promising method for anticipating impact of complex policy regulation.