Module 3: Intervention performance monitoring

Measuring changes in indicators

Once the indicators for the results chain have been defined, the following steps should be carried out and recorded in a results measurement plan:

• Establish a baseline position for each indicator
• Project the result expected at a specific moment in time
• Measure changes in each indicator through the course of implementing the intervention 

A final step is to assess what changes in all indicators suggest about the progress of the intervention towards its goal; this is covered in more detail in Module 5. 

Establish a baseline position

Information on the baseline position should be recorded for each indicator. Conducting baseline studies can be expensive, so it makes sense to use information gathered during the initial diagnostic process for the intervention or for the programme as a whole where possible. As the diagnosis often happens before actual implementation begins, it is important to check that data is still accurate.

The need for market systems programmes to be flexible and adaptive means that baseline data may become redundant if a decision is made at programme level to abandon a particular intervention or sector. It may therefore make sense to establish small baselines for pilot interventions, rather than producing a comprehensive baseline for all interventions. 

Project the result expected at a specific moment in time

Projections are not targets, but estimates of what is intended to be achieved by a given date. Change at systems level can be slow at the beginning of an intervention, and it is important to reflect this in order not to raise expectations. Even so, while it is important not to be bound by these projections, it is essential to have a benchmark that allows the programme team to assess whether change is occurring as expected. 

Projections are also useful for estimating Value for Money in advance of implementing an activity, and help to decide whether particular investments are worth their cost.

Interventions are necessarily based on assumptions about costs, the level of uptake of a new product, service or other innovation, and the level of response that the intervention will generate in the wider market. It is important to make a note of these assumptions in the results measurement plan, so they can be revisited and validated over time. 

The methods and sources of information that will be used to measure changes in the relevant indicators should be recorded in the measurement plan. It is also necessary to include details of who will measure what, and when.

Measure changes in each indicator through the intervention

Monitoring activities should be carried out at relevant intervals. The precise time period will depend on the intervention in question and how quickly it is expected to show results. This will also depend on the pace of change in the wider market system. Monitoring efforts will need to be stepped up in very dynamic environments for instance, as discussed in Module 4.

It is important to consider what indicator data suggests is happening at all levels of the results chain:

• If indicator data shows that progress is being made towards achieving the desired outcome for the intervention, the fact that indicator data is available for preceding levels will help to illustrate whether the intervention has contributed to this or not
• If the outcome is not achieved, results for indicators in the preceding levels of the results chain may help to identify where things went wrong.

Market systems are dynamic and constantly changing, and it is important for programme teams to keep informed on developments as they occur. It is therefore advisable to carry out monitoring activities in-house rather than delegating these to an external contractor, in order to help the programme team develop a better understanding of the market system.

M4P Operational Guide, Chapter 5: Measurement

Core principles and frameworks for measuring results.

Developing a program specific monitoring and results measurement manual

Comprehensive guidance on how to develop your own MRM manual, illustrated with examples from real-life programmes

Monitoring beneficiary numbers

While market systems programmes typically work with market actors to achieve a wider change in the market (e.g. piloting a new product or service) they often have no direct contact with the much wider range of beneficiaries whom the change is aimed to benefit. The Katalyst programme in Bangladesh, for example, has worked with input suppliers to distribute smaller, more marketable packets of fertiliser, in order to improve the productivity of smallholder farmers.

Research suggests that while market systems programmes have good systems in place to monitor numbers of direct beneficiaries, many do not have systems in place to assess indirect beneficiaries. Yet measuring these wider beneficiary numbers is important for assessing the scale of market systems change.

Scale of outreach in market systems development: building the evidence base

Approaches used by implementing organisations to collect evidence on outreach.