Does Your Implementation Fit Your Theory of Change?

Authors : Steve Montague

A brief review of evaluation findings in almost any given domain typically reveals that most and sometimes all major findings deal with the implementation of initiatives—also known as action theory. Moreover, the findings regarding implementation frequently allude to mismatches between the type or level of implementation occurring and the fundamental nature of the initiative. Case examples will illustrate that while all permutations and combinations of change and action theories cannot be summarily assessed, one can use case analysis to draw some lessons to suggest that some combinations are essentially toxic, while others provide at least a reasonable chance of success. The implication is that further systematic coding and analysis of change theories, action theories, and in particular their combinations in programs could produce useful insights for both evaluation and public-policy decision making.

Using Actor-Based Theories of Change to Conduct Robust Evaluation in Complex Settings

Authors : Andrew Koleros & John Mayne

The use of theories of change (ToCs) is a hallmark of sound evaluation practice. As interventions have become more complex, the development of ToCs that adequately unpack this complexity has become more challenging. Equally important is the development of evaluable ToCs, necessary for conducting robust theory-based evaluation approaches such as contribution analysis (CA). This article explores one approach to tackling these challenges through the use of nested actor-based ToCs using the case of an impact evaluation of a complex police-reform program in the Democratic Republic of Congo, describing how evaluable nested actor-based ToCs were built to structure the evaluation.

Can’t See the Wood for the Logframe: Integrating Logframes and Theories of Change in Development Evaluation

Authors : Gordon Freer & Sebastian Lemire

There are numerous ways in which to model the underlying theory of programs. In the context of international development evaluation, the most ubiquitous are likely “logframes” and to some extent “theories of change,” both of which may serve to guide program development and management, monitoring, and evaluation. While logframes and theories of change are often developed in parallel, they are rarely fully integrated in their practical application. Drawing on lessons from a recent theory-based evaluation, this article argues that fully integrating the program theory of change within the program logframe provides for a stronger and more holistic understanding of program progress.

Knitting Theory in STEM Performance Stories: Experiences in Developing a Performance Framework

Authors : Jane Whynot, Catherine Mavriplis, Annemieke Farenhorst, Eve Langelier, Tamara Franz-Odendaal, Lesley Shannon

Gender equality has made its way to the forefront of discussions across various sectors in the Canadian context. Yet the intentional inclusion of gender and other intersectional identity dimensions is just beginning to permeate the realities of performance measurement and evaluation practitioners, particularly those using program theory. There is a vast body of knowledge regarding the measurement of women’s empowerment, gradually declining availability of resources targeting the inclusion of gender in theory, and even less guidance on integrating gender in theory in the context of gendered programming. Similarly, coordinated efforts from multiple sectors have resulted in an abundance of theory regarding girls and women’s representation, recruitment, retention, and promotion within STEM (Science, Technology, Engineering, and Math) but less guidance on the measurement and evaluation in these areas. This article shares recent efforts to bridge the divide using theory knitting to develop a performance measurement framework addressing the decreasing representation of girls and women across the STEM “leaky pipeline” using the COM-B theory of change model.

Till Time (and Poor Planning) Do Us Part: Programs as Dynamic Systems—Incorporating Planning of Sustainability into Theories of Change

Authors : Sanjeev Sridharan & April Nakaima

This article describes the need for, and challenge of, representing the sustainability of a program as a dynamic process. Part of what enhances the complexity of programs is the challenge of dynamic complexity—the complexity of the program evolves over time through the interaction of actors and their environment. This paper describes the need for, and challenge of, representing the sustainability of a program as a dynamic process. The problem is not just one of representation but also of planning—specifically planning for sustainability. We argue that an essential part of any accountability regime is planning for sustainability. Using the concept of programs as dynamic process, we argue that planning for sustainability needs to be a critical aspect of the impact chains of all theories of change. Both the representation and testing aspects of such a formulation are discussed.

Meta-Modeling: A Theory-Based Synthesis Approach

Authors : Sebastian Lemire & Christina A. Christie

Research synthesis has become an increasingly popular approach for summarizing primary research. In the past two decades, interest in mixed methods reviews has steadily grown, followed, more recently, by an increased attention to theory-based syntheses. This article advances and illustrates a practical application of meta-modeling—a mixed methods, theory-based synthesis approach. The proposed methodology combines meta-analytic and qualitative comparative techniques in developing a program theory—a meta-model—of how and why a program works. As the article illustrates, meta-modeling provides for a structured and transparent synthesis approach for building program theories across existing studies.

How We Model Matters: A Manifesto for the Next Generation of Program Theorizing

Authors : Sebastian Lemire, Jane Whynot & Steve Montague

In this concluding article, grounded on the exemplary contributions contained in the preceding pages, the guest editors scale the proverbial soapbox and present a manifesto to guide the pursuit and advancement of the next generation of program theorizing. Formulating ten declarations for program theory development and examination, the modest hope of the authors is to motivate and inspire reflective evaluation practitioners to broaden their views, approaches, and techniques for future program theorizing.