There are certain pitfalls for Theory of Change practice. Below we list the most common pitfalls, several of which were adapted from Funnell & RogersS.C. Funnell & P.J. Rogers (2011: 42-52) Purposeful program theory: Effective use of theories of change and logic models. San Francisco: Jossey-Bass.. Our Theory of Change Quality Audit may provide further guidance in developing a useful, high-quality Theory of Change.
Pitfall: No Actual Theory
A Theory of Change falls short of having an actual theory when it simply displays boxes of activities and boxes of outcomes without demonstrating logical and defensible relationships between them and the various items listed in the boxes. This makes it difficult to understand the causal rationale.
Pitfall: Activity Traps
In this case there’s too much focus on what the organisation does instead of the outcomes and impact they achieve. This may result in well-intended activities that on closer inspection don’t address the conditions that underlie the problems. And it may result in measurement of busyness (outputs) rather than the outcomes the organisation wishes to achieve.
Pitfall: Formulation of the Measurable Instead of the Actual Intended Result
When people fear being held accountable and feel it’s expected from them to measure everything, they may include longer-term outcomes and impacts in too narrow ways that reflect what is readily measurable rather than what is actually the intended result. Proxy measures of larger constructs then themselves become the construct, or the construct may be removed entirely.
Measurability should not be the first or even a major consideration in deciding what outcomes (or attributes of outcomes) will be admissible in a ToC. We’re looking for a full representation of our rationale, and not only of that what we can easily measure. Measurement issues should be addressed separately and systematically from ToC development.
Pitfall: Failure to Undertake a Good Situation Analysis, or No Contextualisation
A good situation analysis provides information about the situation (needs, resources, problems, and opportunities) and the causes and consequences of that situation. Without a good situation analysis you may develop a solution to the wrong problem; i.e. to a problem that does not reflect the situation.
Every situation and organisation is unique. Failure to tailor the intervention to the specifics of the local context may result in interventions that don’t address the actual underlying causes of the problem, as well as are likely to fail to leverage the context to their advantage.
Pitfall: Outcome Attributes Not Made Explicit
Some jump straight from the broad statement of outcome to an indicator or measurement. However, it is important to first make an outcome’s desired attributes explicit. For example, ‘improved job skills’ could be specified further in terms of the exact skills to improve. Or a desirable attribute might be that it should be distributed equitably across a target group and that those who are least advantaged should not be further disadvantaged. It is after specifying these attributes that one starts to think about measurement.
Pitfall: Ignoring Unintended Results
Unintended results can be negative (such as side effects) or positive.
Pitfall: Not Accounting for a Complicated or Complex Context
Treating an intervention as if it were a simple, closed system, when in fact is has important aspects that are complicated or complex, can lead to insensitivity to the inherent unpredictability of the contexts within which it functions. Such an environment calls for recognition of the need for a more emerging and adaptive Theory of Change, and a ToC as living document. Bringing in outside expertise and co-create with stakeholders are important instruments to absorb the context’s complexity.
Boundaries and what is within and outside scope may change over time. As open system that interacts with elements that lie outside its boundaries, it’s necessary to scan the environment, look for warning signs, and pick up on opportunities to influence those out-of-scope conditions important to success.
Pitfall: Overly Linear Thinking
When a Theory of Change doesn’t include feedback loops or relationship attributes, it is likely the result of overly linear thinking. Indeed, it can be hard to unlearn habits of linear thinking. However, a completely linear Theory of Change of A –> B –> C is likely insufficient to understand the reality of change processes. In reality, it is likely that important feedback loops are at play, which create a certain dynamism that is important to understand change processes.
Not only feedback loops create non-linearity, also individual relationships can be non-linear. It often is challenging to specify relationship details such as an applicable threshold level (minimum value of something at which it starts to take effect), ceiling (beyond which it doesn’t have additional effects), delayed effect (takes time before an effect becomes visible), tipping points, interaction between different ToC elements and other types of non-linear causality.
Pitfall: Oversimplifying or Too Detailed
There’s always the risk of trying to either get your ToC ‘perfect’ or to make it overly simplistic. The challenge is to come to a relatively simple but still complete representation of how change happens and why the activities will make your vision of success a reality. The Expert Lens on ‘level of specification‘ speaks more about this.
Oversimplifying means the ToC does not represent reality its complexities sufficiently to be of value. There may be missing links and large gaps between the intermediate outcomes and the ultimate outcomes. This goes beyond your organisation’s sphere of control and sphere of influence. Leaving out necessary preconditions for change means ignoring other important processes and possibly even discouraging them.
But trying to make your ToC a ‘perfect’ representation of reality also isn’t a good idea. If you seek to include all possible elements and linkages, you lose yourself in details making the ToC inoperable and impossible for anyone to understand.
Pitfall: Process Diagram Instead of Causal Change Process
A Theory of Change represents a change process, and change can best be explained by making causality explicit. However, sometimes sequence (this happens after that) is confused for consequence (this happens at least partly because of that). Although the time dimension of sequence may be represented in a Theory of Change (e.g., by the boxes their position), the arrows/relationships should represent causal relationships.
Pitfall: Circular Reasoning
Circular reasoning presents a poor logical argument. It happens when the conclusion is included in the premise, i.e. the conclusion is used, albeit often in a disguised form, in the premises which support it.From https://www.thoughtco.com/circular-reasoning-petitio-principii-1689842
Example: The word of Zorbo the Great is flawless and perfect. We know this because it says so in The Great and Infallible Book of Zorbo’s Best and Most Truest Things that are Definitely True and Should Not Ever Be Questioned.
Check out https://yourlogicalfallacyis.com for other traps related to logical reasoning, including “begging the question” from which the above example is taken.
Pitfall: Not Using the Theory of Change
A ToC is only useful if used in monitoring and evaluation. It should fuel strategic learning processes aimed at societal value creation and be at the centre of such. An example of when a ToC is being ignored is when only final impacts are measured and intermediate outcomes aren’t, thereby still representing a black-box evaluation. Or when analysis does not go beyond simply reporting whether outcomes were achieved, leaving relationships unexplored.
Pitfall: Only a One-Time Exercise
An extension of the previous pitfall is when ToCs are not used for learning. Some approach a ToC as just an output; a product that needs to be developed for example for funders (who may in fact expect a fixed plan from which little deviation will be allowed). There thus is a tendency towards a one-time exercise rather than a ToC as a living, breathing document that facilitates an adaptive learning process. In other word, there’s a focus on planning and reporting (ToC as a product) instead of learning (ToC as a process).
Pitfall: Not Interactive Enough, Too Academic
Theories of Change need to engage and inspire different audiences (e.g., internal staff, beneficiaries, partners, funders, etc). To this end, you may need different versions for different audiences. But no matter the audience, a ToC needs to engage and inspire. Overly academic, complex ToCs are unlikely to do so, no matter the audience. Therefore, it’s important to present the more academic content in an accessible way. Changeroo seeks to achieve this by means of the strategy narratives.