Table of Contents for Tackling Wicked Problems in Complex Ecologies
1.Tackling Wicked Problems in Complex Evaluation Ecologies
Hopson and Cram
This introductory chapter has three goals. First, the chapter will expand on the definition of wicked problems in complex ecologies of evaluation. Second, this chapter will situate responsibilities, relationships, and relevance as core evaluation themes central to the study and practice of evaluation, building off of both knowns and unknowns in our field. Finally, the chapter provides a forecast of the chapters in the book, including setting the context for each chapter and contributor. As such, each chapter will be grounded in real case examples that illustrate how evaluators in complex ecologies of practice, policy, and praxis address wicked problems and reflect on the core themes of the book.
2.Ecological Thinking as a Route to Sustainability in Evaluation
Rowe
This chapter takes concepts of ecological thinking from contemporary ecology and natural science to sketch the directions needed to achieve sustainability-ready evaluation. Connectivity is taken as the primary mechanism of change for ecology and for the application of ecological thinking to evaluation. The first section in this chapter examines the Scottish national government as an example of connectivity at a governance level. Three evaluation scenarios are posited for a school siting and design evaluation, with the third scenario demonstrating what a connected, sustainability-ready evaluation might look like. The Wolong Nature Reserve provides a second evaluation context where ecosystem thinking is applied for sustainability-ready evaluation. Ecological thinking brings connectivity to the evaluation table and provides a starting point for adapting evaluation to a truly transformative role connecting programs and projects to public interest goals and developing sustainability-ready evaluation.
3.Indigenous Insight on Valuing Complexity, Sustaining Relationships, Being Accountable
Tuhiwai Smith
This chapter is an invitation to evaluators to negotiate the uncomfortable interface between indigenous and non-indigenous paradigms and be part of the movement to decolonize evaluation. The Chapter draws from my work as a Māori scholar in the fields of decolonizing methodologies, Kaupapa Māori research and indigenous studies more broadly. The challenge of Indigenous Peoples to a field such as evaluation brings to the fore the importance of the three 'Rs' of central importance to this book. Firstly, the importance of relationships, of being in relation to the world and of sustaining relationships between evaluators and Indigenous stakeholders. Secondly, the significance of relevance in terms of ensuring that the making of meaning is co-produced and meaningful at a deep level. Thirdly, the consequence of responsibility, in terms of establishing responsive and ethical processes that are understood by all participants and are available to be challenged and reviewed
4.Evaluating HIV Practices and Evidence-Supported Programs in AIDS Community-Based Organizations
Lin Miller
This chapter begins with a brief overview of the AIDS epidemic globally. Next, it characterizes the community-based AIDS environment in the United States and highlight its implications for crafting evaluations and evaluation practices that are relevant to these settings. Next the chapter turns to the issue of establishing and maintaining relationships within the community-based activist environment. Finally, the chapter considers the evaluator's moral responsibility in the face of the structural and social stigma. The author positions evaluation as a tool to enable communities to participate in decisions that affect them. Throughout the chapter, mini-case examples are provided by the author based on her recent evaluation work with small community-based organizations and AIDS-affected communities in the state of Michigan.
5.Complex Ecology in International Development Evaluation, Focusing on Women and People with Disabilities
Mertens and Boland
This chapter examines how evaluators can contribute to social transformation with women and girls and people with disabilities (PWD) in developing countries. The authors present strategies that evaluators can use to work with communities to enhance the relevance of their work, build relationships that connect with and transform the lives of women and PWD. Furthermore, the authors explore how evaluators can fulfill their responsibilities to meet the needs of the diverse constituencies involved in transformation for women and PWD in international settings. Advances in the evaluation community that integrate complexity theory with transformative frameworks provide the theoretical framing for addressing these issues. The chapter begins with a section that lays out the conceptual framework based on the transformative paradigm and the social justice branch of evaluation. The chapter provides a summary of complexity theory as a theoretical framework that is useful for informing evaluation work within the transformative paradigm.
6.Creating Collaborative Community Practices through Restorative Justice Principles in Evaluation
Chiuinard and Boyce
The focus in this chapter is to propose a restorative justice approach to evaluation, intended to create new knowledge and new ways of looking at complex ecologies by bringing together primary program stakeholders to restore voice and generate understanding between those previously incarcerated. The authors begin with an overview of the current evaluation practices in the criminal justice milieu, juxtaposing the ideological and epistemological differences between the 'gold standard' in criminal justice with alternative or innovative ideas. The authors provide a description of restorative justice, highlighting key values and principles with a foreshadowing of potential methodological implications. The authors provide a brief account of a sex offender program, highlighting the myriad challenges sex offenders (and communities) confront with current release practices. This description sets the stage for a discussion of restorative justice, using the Four Rs (relationships, responsibilities, relevance and restorative).
7.Creating a Sustainable and Equitable Food System
Hesterman and Millett
The purpose of this chapter is present ideas about how evaluation and evaluators can play a role in fixing the food system, and the chapter present an example of how this is working in the real world and in real time. First, the chapter dives into the complex ecology of the food system leads us to consider what a fair and sustainable food system looks like, and to profile how the Fair Food Network began to intervene to steer the system in this direction. Next, the chapter will look at the role of evaluation in understanding whether system change occurs, focusing on evaluator responsibility for collaborative evaluation design, the relevance of the evaluation methodology, and the relationships that were built during the evaluation that facilitated mutual thinking among stakeholders. The final section of this chapter contemplates the evaluation findings, particularly the shifts in local economies and the impacts on policy.
8.Developing Relevant and Responsible Recommendations in Health Policy
Barksdale, HOpson, Green, Schantz, Kenyon, Rodick, Kaul, Jacobs
This chapter describes a process informed by the 3Rs for developing actionable recommendations in a complex ecology of HHCOs in the United States. The evaluation project examined one strategy for reducing health disparities: the provision of culturally and linguistically appropriate services (CLAS) and, relatedly, the implementation of a national health policy titled the National CLAS Standards. The authors conducted a multi-site, multi-phase, mixed-methods evaluation examining awareness, knowledge acquisition, adoption, and implementation of the National CLAS Standards in a small sample of HHCOs. This evaluation yielded rich descriptions of the participating HHCOs' activities related to CLAS and the National CLAS Standards, along with actionable recommendations intended to help each HHCO advance its CLAS-related efforts and ultimately work toward reducing health disparities. This chapter explains the need and rationale for describing an approach to develop actionable recommendations, and the application of this approach in developing recommendations for HHCOs.
9.Considering the Paris Declaration Principles on Aid Effectiveness as a Means to Drive Reform
Quinn Patton
The chapter presents the Paris Declaration Evaluation as an exemplary model in taking both internal and external metaevaluation seriously. Seriously and systematically ensuring evaluation of the evaluation, with diverse stakeholder involvement, was one of the strengths of the Paris Declaration Evaluation process. To justify that judgment, the chapter reviews the complex ecologies of the evaluation of the implementation of the Paris Declaration Principles on aid effectiveness. The chapter will then describe and review the relationships, responsibilities, and relevance of the external metaevaluation. Ultimately, the chapters two parts include a description and analysis of the Paris Declaration Evaluation and, second, discussion of the complex ecologies of the metaevaluation, including the author's experiences and reflections as the metaevaluator and for reflective practice.
10.Digging Deeper to Engage Wicked Problems through Evaluation
Cram and Hopson
This chapter first asks after what has been learned from authors about the nature of complex ecologies. Following this, themes emerging from the authors about the three pillars of evaluation in these complex ecologies - relationships, responsibilities and relevance – are described. The addition of two additional pillars – restoration from Chouinard and Boyce and resilience from Patton – is then posited. Ultimately, the chapter is both retrospective and prospective, identifying key elements of each chapter within the boundaries of the volume focus and conceptual beams and putting forth a future about how to address wicked problems in complex evaluation ecologies for now and the future.We encourage evaluators to examine the complex ecologies within which they live and work, looking after their tūrangawaewae or place where they belong and have a right to stand.