For the last several decades and recently amidst the COVID‐19 pandemic, many in the global evaluation communities call for shifts from linear, reductionist ways of thinking and working to approaches that embrace systems and complexity. In this introductory chapter, we orient readers to key systems and complexity traditions and terms and how these have been put to use in the evaluation field. Doing so provides a foundation from which to engage with the subsequent chapters. We close this chapter with highlights from the case examples featured in this issue.
In Ecuador, the participatory political design of the political party forming the government from 2007 to 2017, along with the constitution of 2018, created opportunities for citizen participation. Participatory budgeting (PB) is the most commonly used citizen participation mechanism. The direct participation of citizens is reflected in improving the governance by democratizing decision processes. The contribution of PB to the local management and governance of seven rural communities of the Ecuadorian Amazon was analyzed using a case study. Based on (1) the level of compliance with municipal planning through management indicators and, the amounts allocated to PB, (2) along with the level of citizen satisfaction, complementary perspectives (acquired through a survey) on the implementation of PB are provided. These sources of evidence allowed us to critically assess the effects of PB in the improvement of local management and governance. We found low levels of municipal planning compliance, i.e., a 20% (2017) and 43% (2018), high levels of citizen dissatisfaction (around 91%) and also a “disagreement” with the PB implementation process. Finally, it is observed that the implementation of the participatory budget in rural communities presents deficiencies that limit the obtaining of representative benefits and that imply an improvement in the governance and quality of life of the citizenry. This is mainly caused by the low interest of citizens to participate in the phases of execution and monitoring of projects, due to a low culture and participatory education.
With the increasing maturity of systems thinking and complexity science (STCS) within evaluation, this issue of NDE provides case examples of contemporary application. This article identifies themes across case examples to identify emergent patterns and opportunities for the continued development of evaluation practices that draw upon STCS. Each article describes STCS and its applications within specific evaluation contexts. Our review across cases identified three themes: (1) the importance of setting boundaries around what is included and excluded from within the STCS field in a specific evaluation; (2) using STCS concepts and theories to enhance program and evaluation theory, and (3) broadly drawing upon STCS to support changes in systems and develop capacity within organizations. While these implications show some promise for systems and complexity informed evaluation, they, more importantly, underscore the need for deeper engagement with STCS theories and methods while simultaneously remaining accessible for evaluators.
1 June 2015 in Procedia - Social and Behavioral Sciences
Rural communities in Cuenca (Spain) are characterized by a great social dislocation, mostly due to the low population density in these areas. In this way, the existence of groups of citizens able to be active agents of their development process is a critical aspect for any community-based development process in this Spanish region. The Institute of Community Development of Cuenca (IDC) has been working with this type of groups for the last 30 years focusing on the organizational empowerment of the rural communities. Main tools in this process have been the empowerment evaluation approach and the critical friend role when helping the groups to achieve their objectives and reinforcing them. This chapter analyses the empowerment process and how the critical friend role is nourished by the facilitator figure
Despite the increasing literature, applications, and discussions over the last 15 years, there remain questions about how to bring systems thinking and complexity science (STCS) into evaluation in ways that meaningfully enhance its practice, use, and results. One pathway forward is bridging the gap between the large STCS body of knowledge and evaluation practice by synthesizing STCS insights into actionable frameworks to support the work and role of those engaged in evaluation activities. This chapter aims to contribute to this challenge through two main goals. First, the authors develop an actionable STCS framework comprising ten guiding principles, identified through a review of STCS literature to support evaluation practitioners, researchers, and commissioners to bring STCS into evaluation. Secondly—and to test this STCS principles-based framework—this chapter assesses the extent to which TEEBAgriFood, an STCS-informed food systems evaluation framework, advances STCS into evaluation. Results provide learnings and insights about the use of STCS in evaluation design and practice.
Background: Ex ante impact assessment has become a fundamental tool for effective program management, and thus, a compulsory task when establishing a new program in the European Union (EU). Objective: This article aims to analyze benefits from ex ante impact assessment, methodologies followed, and difficulties encountered. This is done through the case study on the rural development programs (RDPs) in the EU. Results regarding methodologies are then contrasted with the international context in order to provide solid insights to evaluators and program managing authorities facing ex ante impact assessment. Research design: All European RDPs from the period 2007 through 2013 (a total of 88) and their corresponding available ex ante evaluations (a total of 70) were analyzed focusing on the socioeconomic impact assessment. Results: Only 46.6% of the regions provide quantified impact estimations on socioeconomic impacts in spite of it being a compulsory task demanded by the European Commission (EC). Recommended methods by the EC are mostly used, but there is a lack of mixed method approaches since qualitative methods are used in substitution of quantitative ones. Two main difficulties argued were the complexity of program impacts and the lack of needed program information. Conclusions: Qualitative approaches on their own have been found as not suitable for ex ante impact assessment, while quantitative approaches—such as microsimulation models—provide a good approximation to actual impacts. However, time and budgetary constraints make that quantitative and mixed methods should be mainly applied on the most relevant impacts for the program success.