Introduction

Undertaking a ToC quality audit should enable you to identify inconsistencies, gaps and weaknesses in order to sharpen the strategies, make implicit assumptions explicit, and improve the overall quality of thinking that guides operational decisions.

If undertaken with a group of stakeholdersAlternatives to undertaking the audit with a group of stakeholders include individual self-assessments as well as hiring external auditors., an audit will involve discussing the context and updating yourselves on critical changes, as well as sharing experiences of what has worked well and not so well (yet). This reflection can then feed into the actual revisions about how the ToC needs to be improved: additions, changes, and deletions.

Choose What is Relevant to You

This Expert Lens presents a full and shortened ToC audit. The audit presented in this Expert Lens can be experienced as quite extensive and time-consuming. It is up to you to decide which parts are most relevant to your organisation, and which parts you decide to skip.

Attached to this post you can find the Word document with the audit, so that you can easily select the questions most relevant to you.

Sources of Inspiration

We adopted and adapted dimensions and items from several existing audits. We especially build upon Funnel & Rogers and the Hivos ToC Guidelines. Whereas, both incorporate elements referring to the ToC development process as well as the ToC as product, Hivos focuses more on the process side, while Funnell & Rogers focus more on the product side.

Funnell & RogersS.C. Funnell & P.J. Rogers (2011) Purposeful program theory: Effective use of theories of change and logic models. San Francisco: Jossey-Bass.	Guidance to evaluate ToCs on four dimensions: the visualisation, the traps that many ToCs fall into, internal validity and external validity.
Hivos ToC GuidelinesHivos ToC Guidelines: Theory of Change Thinking in Practice	A comprehensive quality audit that asks reflective questions that well represent the ToC development steps. It offers different audits for first-time draft ToCs and ToCs already used in practice.
DaviesRick Davies, Planning evaluability assessments, DFID.	The premise of Rick Davies is that a ToC is only as good as it is evaluable. This audit provides the criteria to assess the extent to which the design of a ToC is evaluable.
BetterEvaluationBetterEvaluation, Theory of change – guidance on developing, representing, and using.	Distinguishes between good practice, adequate practice and inadequate practice in developing, representing and using ToCs.

Audit

(A) Development and analysis

This category is about a comprehensive analysis of context, actors, and strategic options. Is the focus and strategy described in the ToC based on comprehensive and up-to-date information about the context and stakeholders?

Weak – Superficial, uncritical, business as usual
Has potential – Some new thinking, with big gaps in critical thinking
Reasonable – Critical thought on most areas, unclear in some areas, mainly based on known strategies
Robust – Critical, clear, focused, considers wide range of perspectives, information and strategies

Vision of Success. What is the desired change, why and for whom?
	Weak	Has potential	Reasonable	Robust
1. Is the desired vision of success clear and detailed enough? Does it present the issue(s) to change in a clear way, making the organisation’s objective clear?
2. Is the vision of success statement (still) genuinely representing the vision?
3. Are the people whose lives are meant to improve identified and described clearly?
4. Is the vision of success (still) relevant to the needs of the people whose lives are meant to improve?
5. Are the people whose lives are meant to improve (still) a critical group to benefit from our Theory of Action?

Analysis of Current Situation. What is the current situation in relation to the issue(s) we wish to change?
	Weak	Has potential	Reasonable	Robust
6. Does the situation analysis provide rigorous support of the relevance of the desired change to the needs of the target group?
7. Is there clear evidence of a rigorous analysis of social, political, economic, cultural, ecological, and geographical factors in terms of how they influence the issue that the vision of success seeks to address?
8. Is there a thorough discussion of the roles and interests of the key actors, going beyond the usual suspects, and how they influence or could contribute to or hold back the desired change?
9. Has the ToC incorporated all changes in context and in stakeholders’ interests and capacities that require adapting the ToC? Is the ToC up to date to take advantage of new opportunities to achieve the desired change? Consider social, political, economic, technological, cultural, ecological, and geographical factors.

Analysis of Change Process. Who and what needs to change, where and in which way, for the desired change to become possible? Who needs to do what differently?
	Weak	Has potential	Reasonable	Robust
10. Does the ToC reflect a solid consideration of the range of different aspects that need to change in order to make the desired change possible: relationships, capabilities, values, attitudes, behaviours, formal and informal institutions?
11. Is there a clear and well-argued prioritisation with regard to the domain(s) to focus on?
12. Are the domains of change to focus on up to date with the present context and stakeholders?
13. Is the specification of who needs to do what differently to make the desired change possible, up to date with the present context and stakeholders?

Identification of Strategic Priorities. What changes can the organisation best influence, why and how, in the next 3 – 5 years?
	Weak	Has potential	Reasonable	Robust
14. Is the analysis underpinning the selection of the strategic priorities clear and convincing?
15. Is the selection of strategic priorities realistic in view of the identified time frame?
16. Are the strategic priorities and the strategies themselves still fitting given the experiences to date?

Scope
	Weak	Has potential	Reasonable	Robust
17. Does the scope clearly define the ToC’s unit of analysis, including the stakeholders to include in the analysis?
18. Does the ToC’s scope (unit of analysis) include all stakeholders (still) important to the change process?

Weak – No thought on power dynamics, gender dynamics or scalability
Has potential – Weak and/or partial power analysis, gender analysis or scalability
Reasonable – Power, gender and scalability lens used but some areas or implications still underdeveloped
Robust – Power, gender and scalability lenses clearly inform analysis and strategies

Inclusion of a Power Perspective. Clear power analysis about ‘how change happens’ and the forces at play that help/hinder.
	Weak	Has potential	Reasonable	Robust
19. Does the situation analysis present a convincing and (still) accurate understanding of underlying power inequalities?
20. Are the chosen strategies explicit about which power inequalities are to be shifted, and why?
21. Are the chosen strategies (still) relevant and politically well-informed about which power inequalities are to be shifted, and why?
22. Is the analysis justifying the choice of strategies sufficiently convincing that strategies are (still) realistic about how power inequalities are to be shifted?

Inclusion of a Gender Perspective. Clear gender analysis about ‘how change happens’ and the forces at play that help/hinder.
	Weak	Has potential	Reasonable	Robust
23. Does the situation analysis present a convincing and (still) accurate understanding of underlying gender dynamics and inequalities?
24. Are the chosen strategies explicit about which aspects of gender inequalities are to be shifted, and why?
25. Are the chosen strategies (still) relevant and politically well-informed about which aspects of gender inequalities are to be shifted, and why?
26. Is the analysis justifying the choice of strategies sufficiently convincing that strategies are (still) realistic about how gender inequalities are to be shifted?
27. Are the people directly and indirectly targeted by the initiative specified in terms of gender?

Inclusion of a Scalability Perspective
	Weak	Has potential	Reasonable	Robust
28. Is the ToC well prepared for sustainable scaling if the scale of activities and impact increase?For example, while at a small scale certain practices might not do any harm to the environment, the same practices may harm the environment at a larger scale. A ToC well prepared for these kinds of issues is aware of them and has prepared a strategy for dealing with changes in ToC pathways that may materialize as the organisation grows.

Weak – Very few people involved ad hoc in formulation or review
Has potential – Intentional inclusion of some players in formulation or review
Reasonable – Clear process for diverse input planned with wide participation in some aspects but not fully realised
Robust – Clear process implemented with critical input from diverse relevant players

Active Participation of Relevant Groups of People in ToC DevelopmentThis principle is about ensuring that the vision is not just developed by a few people behind their desks but has been informed with involvement of the people who are supposed to benefit and those who are supposed to help implement the initiative.
	Weak	Has potential	Reasonable	Robust
29. Does the ToC narrative make clear who was involved in which way to inform the ToC?
30. Is the choice of people to involve in the ToC development process sufficiently justified in terms of stake, expertise and generating ownership of the ToC?
31. Were all the relevant people involved meaningfully in the ToC development process?
32. Are the opinions and experiences of key relevant people meaningfully influencing the revision of the ToC?
33. Does this include people who are supposed to benefit as well as those who are supposed to help implement the initiative?

(B) Visualisation

ToC representation requires three essential features: it is a coherent causal model (a comprehensible explanation of the causal processes that are understood to lead to the outcomes), it is logical (subsequent outcomes are plausibly consequential), and it communicates clearly.

Weak – Incoherent, illogical, unclear
Has potential – Some coherent, logical and clear change pathways, but still with significant pathways being incoherent, illogical and unclear
Reasonable – Mostly coherent, logical and clear change pathways, but still some weak segments
Robust – A coherent, logical and clear representation

Produce a Coherent Causal Model
	Weak	Has potential	Reasonable	Robust
34. Does the ToC communicate a coherent message about how the activities undertaken contribute to the results of interest? (not a flowchartOne of the common mistakes in drawing ToCs is to draw them as a flowchart of activities rather than a causal model of what produces the impacts.)
35. Does every causal strand in the ToC lead to an identified longer-term impact of interest? (no dead endsAvoid dead ends: if it is important enough to include the strand in the ToC, then it must be because it will contribute to some important final impact, and this should be shown in the ToC.)

Be Logical
	Weak	Has potential	Reasonable	Robust
36 . Are all the arrows incorporated in the ToC meaningful, and are there no arrows for which there is no plausible or only an insignificant causal connection?Poorly drawn ToCs put arrows between all the boxes, even when there is no plausible causal connection. Even when there is a causal relationship between two variables, if it is small, it might be more helpful not to include it and instead to focus attention on the main causal relationships.
37. Are ToC elements labelled in a way that clearly specifies the direction of change?A good ToC can be read as a coherent and plausible story of cause and effect. This requires attention to how the components are labelled. They need to be clearly specified in terms of the direction of intended change, so that the effect a cause leads to is understood clearly when reading the visualization.
38. Does the visualisation make the sequential and consequential progression obvious?Poorly drawn ToCs fail to show the sequence and consequences of activities and intermediate results. The boxes need to be ordered in a way so that the sequence and consequences are immediately obvious. The final result should be placed at the end. In addition, irregularities in the arrows and alignment of boxes are visually distracting from the main messages.

Communicate Clearly
	Weak	Has potential	Reasonable	Robust
39. Does the ToC communicate a clear, focused message?Focus on the key elements: A ToC is meant to support communication, so decide on the main message, and draw it in a way that clearly gets it across. Use symmetry and alignment to reduce distraction. If there are multiple strands, try to lay them out symmetrically. You might group the elements of each strand together and place a large box around them, making clear these elements belong together.
40. Does the ToC avoid too many arrows and feedback loops?ToCs should be as simple as possible (but not simpler). Although in some ways, everything might be connected to everything else, that does not mean it is going to be helpful to show this. A ToC cannot show every causal factor, every causal connection, and every feedback loop and at the same time be coherent and useful.
41. Is the ToC well readable and, if necessary, does it use nested ToCs to avoid presenting too many details at once?If a ToC is very detailed, it may be better to do an overview of the main elements, and then unpack these separate pages as nested ToCs rather than trying to squash it all in onto one page.
42. Are ToC labels easily understandable?
43. Does the ToC avoid trigger words and mysterious acronyms?Think of the wider group of people who may read the ToC. Check carefully for acronyms. Check for sensitive issues and careful wording. For example, choose which terms to use to refer to participants and the causal processes involved.
44. Is the ToC presentation fit for presentation to the particular audiences it is intended for?

(C) Internal validity

Internal validity refers to whether the ToC hangs together in a way that makes sense and tells a clear, coherent, believable, and logical story about the outcomes the organisation is trying to achieve, why those outcomes are important, and how the organisation will contribute to the outcomes.

Weak – Unclear, incoherent, incomplete, imprecise, unconvincing, illogical
Has potential – Partially robust and convincing, but with important parts still unclear, incoherent, unconvincing or illogical
Reasonable – Clear, coherent, believable and logical for most parts, but with still some weak segments
Robust – The way it hangs together tells a clear, coherent, believable and logical story

Combines a Narrative with a Visualization
	Weak	Has potential	Reasonable	Robust
45. In addition to the visualisation, does the ToC include a change story that summarizes the ToC in a clear and engaging manner?
46. Does the ToC include a complete narrative that zooms in on the details of the issues that affect the organisation’s capacity to produce societal value?

Mapping Change Pathways: Complete & Selective
	Weak	Has potential	Reasonable	Robust
47. Are the pathways of change the central organising principle for the ToC?The pathway of change is the link between what the organisation is doing and the problems it is seeking to address (i.e., the vision of success). However, many ToCs fail on the basic test of being outcome focused. They concentrate excessively on what the organisation will do rather than the outcomes it will achieve.
48. Can all intended outcomes and impacts be linked, directly or indirectly, to the needs that gave rise to the problem the organisation seeks to address (i.e., the vision of success)?
49. Is there one or more pathway for each domain of change that we consider important to the realization of the vision of success?
50. Are all elements within pathways (still) relevant and important enough to include, or can certain elements be dropped?The expert lens “Level of specification” can help determine a ToC’s appropriate level of specification.
51. Are possible unintended results considered sufficiently in the pathways?

Mapping Change Pathways: Clear & Detailed
	Weak	Has potential	Reasonable	Robust
52. Is the ToC spelled out sufficiently clearly? That is, is there enough detail to see how the organisation will work (without going into all the daily operational detail) and to be able to raise questions about the logic of the argument and the feasibility of implementation?
53. Are actors named clearly and their expected behaviour change made explicit at different levels of the change pathways?Often it is forgotten to explicitly specify the stakeholder of concern for a ToC element, or are other properties of the element ill defined. This produces ambiguity and consequently hampers effective usage of the ToC.

Strength and Plausibility of the Logical ArgumentThe strength of the logical argument includes considerations relating to coherence, sequencing, completeness versus glaring gaps in the logic, and temporal arrangement of outcomes and their relationship to the timing of activities.It also includes the adequacy of the organisation to produce the desired outcomes, given the ToC. Unrealistic objectives given the role of the organisation and the resources it commits, is a common deficiency. Adequacy of resources to achieve the desired outcomes is thus a key consideration.Questions about whether all intended outcomes are matched by activities and/or prior outcomes should also be raised. Vice versa, activities that are not linked to outcomes may be unnecessary or important outcomes may be missing.But even a plausible ToC may still fail: it does not rule out equally plausible competing theories; unintended outcomes can turn out even more important than intended outcomes; and implementation may not adhere to the ToC, which may point to failure in implementation and/or in the ToC itself.
	Weak	Has potential	Reasonable	Robust
54. Do the pathways build upon a plausible rationale or argumentation that is coherent and complete? That is, is there a continuous causal chain, connecting actions with the final impact of concern, without incomprehensible gaps in the rationale?
55. Is there a convincing temporal arrangement of outcomes and their relationship to the timing of activities?
56. Is it believable that the activities and available inputs will lead to the envisaged results, within the envisioned timeframe (if a timeframe is defined)?

Articulation and Sufficiency of Underlying Mechanisms for ChangeA ToC should articulate its mechanisms for change. These mechanisms are the ways in which outcomes will occur. Statements about mechanisms are the “because” statements that underpin the “if-then” statements. They are the arguments behind relationships.
	Weak	Has potential	Reasonable	Robust
57. Are the arguments behind relationships sufficiently articulated? That is, are they sufficient to explain the relationship?
58. Are additional or alternative mechanisms incorporated in the ToC?
59. Are important feedback loops identified?
60. Is the dependency on other organisations and their mechanisms incorporated in the ToC?
61. Are inputs and activities sufficient to activate the mechanisms to the degree needed to achieve the desired outcomes?Whereas a previous item (57) looked at whether the types of activities and inputs are believable to produce the envisaged results, this items zooms in on the sufficiency of the inputs and activities to activate the mechanisms to the degree needed.

Up-to-Date
	Weak	Has potential	Reasonable	Robust
62. Are the change pathways fully up-to-date with how we now understand change to actually happen?

Weak – None except most basic/obvious
Has potential – Some but not systematic, clear or critical
Reasonable – Fairly complete but not all well formulated
Robust – Clear, comprehensive, critical ones identified

Explicit Underlying Assumptions and Values
	Weak	Has potential	Reasonable	Robust
63. Does the ToC clearly include a thoughtful and comprehensive specification of the key underlying assumptions?
64. Are all assumptions included useful in a way, be it for monitoring, evaluation or learning? In other words, are we sufficiently selective?
65. Are the key assumptions on which our work is based still up to date with our lessons learned, or do some assumptions need to be added, changed or simply dropped?
66. Are the values that inform the strategic choices in the ToC explicit?
67. Do we clearly uphold the values that inform our strategic choices in the ToC?

ToC: Consistency
	Weak	Has potential	Reasonable	Robust
68. Is there consistency in the way the ToC is described across documents (Design, Monitoring & Evaluation plans, work plans, progress reports, etc.)?

(D) External validity

External validity refers to how well a ToC stands up in relation to external sources of evidence, the context in which it operates, and intended uses of the ToC.

Weak – No external evidence, disconnect from the local content, or unsuitable for its intended use
Has potential – Weak and/or partial external evidence, integration of the local context, or suitability for its intended use
Reasonable – External evidence, local context and intended use inform the ToC, but some areas still underdeveloped
Robust – External evidence, local context and intended use clearly inform the ToC

External Sources of Evidence
	Weak	Has potential	Reasonable	Robust
69. Is the ToC consistent with available data and research evidence?
70. Is the relevant evidence base current? Does evidence still hold true or does the ToC need to be reconsidered in light of new evidence?
71. Is the ToC consistent with accepted theories, i.e. broader ToCs or ToC archetypes?For example, the structured generic ToC framework by Mayne. It’s not necessarily wrong to deviate from such theories, but it may raise critical issues to which managers may very well be able to give good reasons for departure and for trying something different.

Context
	Weak	Has potential	Reasonable	Robust
72. Does the ToC incorporate consideration of its context by making it context-specific? That is, does it anticipate and take account of contextual factors, their effects on outcomes, and how it will work within its context?Contextualising a ToC extends the “if-then-because” statements that reflect ToC mechanisms to include “as long as” statements about what else needs to occur and the conditions under which the if-then assumptions may and may not hold. Not only slippage could occur because of mechanisms beyond the organisation, but external mechanisms can also boost the power of ToC mechanisms by, for example, removing other barriers or competing mechanisms.
73. Does the ToC apply an ecosystem approach? That is, does it incorporate also how other actors affect the change process, and not only what the organisation itself does?
74. Have assumptions about the roles of other actors outside the organisation been made explicit (both enablers and constrainers)?
75. Is the level of complication or complexity of the ToC commensurate to how complicated and complex its context is?More information on the difference between complicated and complex can be found at http://www.managingforimpact.org/resource/video-cynefin-framework. Complication may for example translate into a ToC with coordination and links among various sets of intended outcomes and activities. And complexity may translate to open systems considerations, including feedback loops, emergent outcomes and fluid boundaries.

Intended Uses
	Weak	Has potential	Reasonable	Robust
76. Does the ToC developed fit its intended use and purposes for which it has been developed?Such purposes could for example include: to clearly explicate a policy position or espoused theory, to represent current practice, to communicate with external stakeholders, to provide the framework for impact evaluation, to test a theory or parts of a theory, and/or team building.

Differences of Perspectives
	Weak	Has potential	Reasonable	Robust
77. If different stakeholders hold different views about the organisation’s objectives or how they will be achieved, are the different views of stakeholders sufficiently reflected in the ToC?

Ethical Base for the ToC
	Weak	Has potential	Reasonable	Robust
78. Do the ends justify the means?
79. Are the principles and ideological underpinnings of the ToC accepted by various stakeholders?
80. Are there no inconsistent principles and ideologies operating in parallel for different stakeholders?

(E) How to use

To take full advantage of a ToC it needs to be used as a living product and guide the work you do.

Weak – Plans or processes on how to use the ToC are absent, ToC offers no practical foundation for monitoring, evaluation and learning
Has potential – Plans or processes on how to use the ToC are there but vague, ToC fits practical use cases poorly
Reasonable – Plans or processes on how to use the ToC are there and ToC offers a framework for monitoring, evaluation and learning, but some areas are still underdeveloped
Robust – Well detailed and clear plans and processes on how to use the ToC, ToC offers a strong framework for monitoring, evaluation and learning

Planning for Use: Testable
	Weak	Has potential	Reasonable	Robust
81. Is the ToC such that it potentially can be used to generate helpful evaluation questions or set priorities for evaluation?For example, is it constructed in a way that shows the most important outcomes to look at and the most important factors that are likely to affect outcomes so that they can be explored as part of an evaluation or a research project?
82. Is it possible to identify which linkages in the causal chain will be most critical to the success of the project, and thus should be the focus of evaluation questions?

Planning for Implementation
	Weak	Has potential	Reasonable	Robust
83. Does the ToC narrative make clear how the ToC will be used and who will use it to guide implementation?
84. Does the ToC narrative make clear how the information will be used and by who in order to improve implementation as well as the ToC, and to show progress?

Planning for MEL: Monitoring, Evaluation and Learning
	Weak	Has potential	Reasonable	Robust
85. Does the ToC include critical information to track and analyse in order to learn from implementation?

Planning for Use: Validity and Reliability
	Weak	Has potential	Reasonable	Robust
86. Are there valid indicators for each ToC element for which measurement is considered critical? That is, will they capture what is expected to happen?
87. Are there reliable indicators for each ToC element for which measurement is considered critical? That is, will observations by different observers find the same thing?

Weak – Collecting dust
Has potential – Used infrequently, on request
Reasonable – Some proactive use but not updated
Robust – Frequent use and updating

Active and Regular Use by Relevant People to Guide Implementation and MEL Processes
	Weak	Has potential	Reasonable	Robust
88. Is the ToC being used to guide implementation?
89. Are there processes in place that ensure the ToC is being adapted to reflect lessons learned and changing contexts and circumstances?
90. Is there clear evidence that we are using the ToC to reflect, learn and improve our work?
91. Are we obtaining, analysing and using the information that we considered critical as planned?
92. Are we monitoring the assumptions that are most critical and that we are uncertain of?

Active and Regular Use for External Communication and Stakeholder Engagement
	Weak	Has potential	Reasonable	Robust
93. Is the ToC being used for external communication?
94. Is the ToC being used for stakeholder engagement?

Audit: Shortened Version