Description

In ToC thinking the paths towards your vision of success are described through series of interlinked results. We can distinguish between results at various levels, according to our degree of control. As demonstrated in the figure below, we speak of outputs, outcomes and impact.

In ToC thinking we tend to focus on the level of outcomes, as this is where the change needs to happen. At the same time, it is the level for which an organisation can be held accountable, more than the impact level, which usually involves a great deal of other factors and actors.

What is Impact?

Impact is long-term sustainable change at a systems level, i.e. the effect on society. It is a fundamental systemic change that is expected to persist for the long-term. Impact is thus something that takes place for a community or population instead of individuals or families (people as well as environment). For example, improved employment rate, improved health of people, or even an organisation may help individuals to increase their income (outcome), which at a system level may help reduce poverty rates (impact).

What is an Outcome?

Outcomes are noticeable changes in individuals or families. They are preconditions to impact; i.e. they may add up and produce sustainable and structural systemic change (i.e. impact). Outcomes are also the effects of activities/outputs from the perspective of individual stakeholders (or families). An outcome of a refugee guidance program may for example be increased well-being of a refugee, while if the government is included as stakeholder within your ToC’s scope then ‘reduced benefit payments by the government’ may be an impact.

Pathways

Many outcomes are intermediate results that lead to one or more other outcomes. Such chains of interdependent outcomes eventually produce the final impact. Specifying such pathways of change is discussed as a separate step. In practice, you will go back and forth between this and other steps.

Difference between Impact and Outcomes

Impact and outcomes thus both are preconditions to your vision of success, but differ in their level of analysis. While outcomes are the more direct results/changes for individuals and families from activities; impact on the other hand, looks at the fundamental systemic long term change to society. For example, while ‘people employed after training’ could be an outcome, an ‘improved employment rate’ within a community would be a resulting impact.

Formulating Impacts and Outcomes
In describing an outcome or impact, be specific with the stakeholder(s) that it is related to. For both impact and outcomes we recommend you to formulate them using a combination of the direction of for whom + change + what has changed. For example: ‘refugees gain improved well-being’.
In addition, it is essential to specify the important desired attributes of each outcome and impact. Doing so is essential to understand what the outcome that you seek to develop looks like, providing direction for sound ToC design and implementation.

Identifying Impact and Outcomes

The steps to identify your impact and outcomes:

These attributes can be incorporated in your ToC in four ways:

• Develop them as Key Performance Indicators (KPI’s)
• As assumptions
• Including them in the title of your impact or outcome
• Including them as a description for the impact or outcome

Specifying the attributes of each outcome is to operationalise it. For example, you can operationalise an outcome by expressing it in terms of quality, quantity, timeliness, equity, different target audiences and other features. This provides the basis for the development of practical success criteria and KPI’s to which to judge performance. Such criteria provide direction for sound ToC design and implementation. ToC implementation is to focus on the desired outcomes and desired features or attributes of outcomes, and not just those that are (easily) measurable. Therefore, even if concrete, measurable KPI’s cannot be developed, it is essential to clearly detail the attributes of each outcome. After all, different factors will be relevant to achieving different attributes.

Desired attributes can be identified in many different ways:S.C. Funnell & P.J. Rogers (2011) Purposeful program theory: Effective use of theories of change and logic models. San Francisco: Jossey-Bass.

• Take each of the key words in the outcomes statement and expand on them.
• Consult with stakeholders, including those who may be in some way affected by your organisation or denied access to it. These people will want assurance that certain safeguards are built into your organisation’s success criteria.
• Reviewing program documentation and relevant literature.
• Identifying what is realistically achievable.
• Identify social justice considerations that might otherwise be overlooked. This includes:
  • Ensure overall relevance for disadvantaged groups if these groups are among the target group. Don’t prejudice service delivery to the most needy.
  • Have success criteria that draw attention to the distribution of outcomes to identify the winners and losers, and not just average results.
  • Include attributes that not only focus on access to service but actual levels of take-up of the service as well.
  • Ensure that outcomes are achieved and services delivered in a way that preserves the self-respect and dignity of disadvantaged groups.

Questions to Ask

Specifying outcome attributes: What, When, Where, How, Why, and Who

What features the outcome should have

• Description of the nature of the outcome—for example, what skills are to be acquired or behaviours to change
• The quality—for example, reliable, accurate, to the client’s satisfaction, comprehensive, equitable
• The quantity—for example, how much, what proportion, whether to increase or decrease

With whom each outcome should be achieved and who has an interest

• Numbers and types of clients
• Target groups identified by types of needs
• Attributes important to different stakeholders

Where the results should be achieved

• Intended geographical location of results
• Particular sites

When the outcome should be achieved

• Timeliness of response—for example, of services to client requests for assistance
• Time targets or milestones for achievement of each outcome to a certain level
• The time frame it takes before the outcome or impact can be expected to come about
• The amount of time it is expected to last—for example, will the effects be there for just a short time or perhaps for a person’s entire life
• Whether an inflection point applies, i.e., a certain level of performance has to accrue before the outcome or impact causes other outcomes and impacts

How results should be achieved, including addressing side effects, unintended outcomes both positive and negative, and other agendas (but not the process for achieving the outcome)

• At a cost of $X
• With minimal disruption to other programs and other stakeholders
• With equitable distribution of results
• In compliance with statutory requirements
• Preserving clients’ dignity and self-determination

Why the criteria are important

• Stakeholder interests and concerns
• Link to overall effectiveness of the program (research base)
• Necessary to achieve higher levels in the outcomes chain—features required of lower-level outcomes, such as percentage of target population reached, that are a prerequisite for achieving higher levels of outcomes
• Mandated by professional standards.

Tools

More information

Behaviour change

* Centre for Behaviour Change, University College London: Resources on Behaviour Change.

* World Development Report 2015: Mind, Society and Behavior.

Four Dimensions of Change Framework

* Hivos ToC Guidelines: Theory of Change Thinking in Practice, page 90-91.

* I. Retolaza Eguren (2011) A thinking and action approach to navigate in the complexity of social change processes, p.7-8.

Backcasting

* The State of Victoria Department of Environment and Primary Industries (2014). The Engagement Toolkit, Book 3, Version 4, page 6.

Context Immersion

* Multi-Stakeholder Processes Knowledge Co-Creation Portal on context immersion.

I2L2 Framework

* ORS Impact, I2L2: A Formula for Change.

* ORS Impact, Impact, Influence, Leverage, and Learning (I2L2) Outcomes Framework.

Types of Impact & Outcomes

Impact vs. Outcome

Both are preconditions to the vision of success. They differ in their level of analysis:

• Impact: The long-term sustainable change for a society.
• Outcome: Changes in individuals or families that result from someone’s activities and outputs.

Different Types of Impact

Impact takes the form of fundamental changes at a systems level. Such fundamental changes are for example improved conditions, capacity, changed norms, culture (collective behaviour), etc. Such fundamental changes could occur to any of the following systems:Based on McKinsey & Co’s work. Source: McKinsey’s social impact assessment portal: http://mckinseyonsociety.com/social-impact-assessment/ and http://trasi.foundationcenter.org/terms_defined.php.

• Social conditions: Social impact includes changes in physical and mental health, quality of life (including education, healthcare and safety), and attitudes or behaviours.
• Environmental conditions: Environmental impact is any impact related to natural resources and ecosystems, such as changes in carbon emissions related to climate change, water quality, or biodiversity.
• Economic conditions: Economic impact refers to financial impact such as changes in income or financial stability of community members and/or level of public expenditures.
• Organisations & economic markets: A better functioning market system such as increased transaction transparency, inclusive participation, commitment to contractual relationships, shared technological standards, or investment strategies.
• Political: Political impact describes changes in legal and governmental policies, rules, and norms.
• Cultural: Cultural impact is related to creating, disseminating, validating, and supporting arts, culture, and diversity as a dimension of everyday life in communities.
• Other

Different Types of Outcomes

For outcomes, we could distinguish between the following types:

• Fundamental change in individuals and families
  • Attitude such as perceptions, beliefs and motivation
  • Awareness
  • Knowledge
  • Skills
  • Behaviour
  • Health
  • Financial status
  • Conditions (e.g., family stability)
  • Other
• Fundamental change in relationships
  • Power relations / status
  • Trust
  • Tie strength
  • Other
• Fundamental change in environment / ecologyOften, fundamental environmental change will take the form of impact rather than outcome. But this is not necessarily always the case.

Short-term, Intermediary versus Long-Term Effects

• Short-term effects: generally expected immediately as an activity is completed or can occur within a year.
• Intermediary effects: result from and follow short-term outcomes.
• Long-term effects: the ultimate expected impact of a change process.

Impact is generally always considered a long-term effect.

Sphere of Influence

Furthermore, we distinguish between effects inside and outside the organisation’s sphere of influence:

• Inside: outcomes and impact the organisation has (at least some) influence on.
• Outside: outcomes and impact that do affect the realization of your organisation’s vision of success but that your organisation has little or no influence on.

More about Impact & Outcomes

ToC Development as a Process of Backward Modelling

The development of a ToC is a process of backward modelling. It starts with your vision of success. This is followed by the preconditions: the changes, in the form of impact and outcomes, necessary to make your vision of success a reality. Only then, you begin to look at what your organisation and other organisations (will) do to achieve these outcomes and impact. In effect, what actions or activities you do comes last in the analysis. Often the mistake is made to start the analysis with what the organisation does.

Difference between Outcomes and Output

Outcomes can sometimes easily be confused with outputs. For example, if a training programme aims to get people into jobs then completion of the training is an output, while getting the job is an outcome.

Different Meanings of Impact

For Changeroo, the way impact is defined involves a process of development, evolution and emergence. But this is not completely the same as how some of the impact measurement methods define impact. The term ‘impact’ within some of the impact measurement literature is reserved for that part of the realized outcomes that can be attributed to the organisation, often expressed as a percentage. Impact then thus equals outcomes minus the part of the outcomes that would also have taken place without the organisation’s efforts. For example, as part of an ongoing trend or an event that is external to the organisation’s influence.