What evidence is there for the impact of market systems development?

The strength (or weakness) of evidence to support use of the market systems approach is a topic that always provokes heated debate.

As part of Itad’s work with BEAM we recently completed a second annual review of evidence in the BEAM Evidence Map. The Map is a database of around 100 carefully vetted documents that report results achieved by MSD programmes.  

Crucially the Review revealed that there are now sufficient evidenced examples of programmes promoting economic development, improving access to services and reducing poverty, to validate the market systems approach.

However, this headline masks a nuanced picture. Our review examined results from a wide variety of programmes, across different geographies and sectors. It allowed us to delve deeper into the BEAM Evidence Map and analyse the characteristics of the evidence base.

We also undertook a more detailed examination of a sample of six examples that have documented systemic and poverty reduction-level change.  

What does the synthesis tell us?

Our report is a narrative synthesis, not a systematic review.  It highlighted evidence of initiatives contributing to making market systems work in a more pro-poor manner in a number of different ways. These include examples of:

• crowding-in by other market actors
• improving regulations and government policies
• influencing the way buyers and sellers behave in a variety of ways

Market systems programmes have a hierarchy of results in their theories of change. They start with intervention-level results, then proceed through levels characterised by market system change, creating pro-poor growth or improved access, and finally sustained and large scale reductions in poverty.
 

Reset Fullscreen Strategic Framework for market systems development, M4P Operational Guide

We found that evidence was stronger for results at the lower levels of this hierarchy. This is not surprising given the expected time lag in the presentation of poverty reduction and also the difficulty in assessing this. 

MSD programmes can demonstrate results at the level of ‘growth and access’ more frequently, however more and different evidence (such as ex-post evaluations) is often needed to assess poverty reduction with high confidence.

Market systems development remains a highly contextual and complex approach. As an industry we still need to expand the evidence base to better answer in-depth questions about who benefits and how, and in what circumstances the approach can be most effective. 

Among the six example documents that we examined in detail, there was a prevalence of interventions in supporting functions to the core market in question. These included skills/ human capacity and access to information.

Using the Adopt-Adapt-Expand-Respond (AAER) framework showed that the earlier stage ‘Adoption’ of new behaviours was the most common systemic change evidenced by programmes.

The analysis of the poverty reduction results in these six examples also evidenced a logical chain of results For example, where increased productivity resulted from systemic change in supporting functions such as skills and technology transfer in four examples. 

What are the implications for market systems development practitioners?

For those involved in MSA, whether it be implementing programmes, funding programmes or researching/ evaluating, there are recommendations to allow us to develop a fuller, more nuanced account of impact and outcomes:

For funders and implementers

• Budget for impact evaluations. To increase the number of evidenced, high confidence examples.
• Conduct further research and undertake systematic analysis on where evidence is being generated across market systems. To increase the number of evidenced, high confidence examples.

Also, for implementers and evaluators

• Investigate and discuss unintended and negative outcomes. To build a fuller understanding of the impact of MSA programmes to the wider market system.
• Disaggregate data more fully and analyse differences in results – with a particular, but not exclusive, focus on gender.  To further build our understanding of whom MSA is impacting and how.
• Be explicit about methodology. To allow others to better interpret the results from evidence documents.

The further reasoning behind these recommendations is covered in the full report as well as some recommendations for the BEAM team around categorising evidence within the map.

What does the future hold for understanding MSD?'

The evidence base for using the market systems approach is growing. It describes results at different results levels and comprises a diverse typology of documents. Results are now being consistently described in different contexts and markets with varied insights into how the approach has been adapted to deliver results. 

However, it is clear that there are quality or confidence limitations to the existing evidence documents. And there are important ways that the evidence base could be strengthened.

Context is key. Fully understanding this and adapting to it is a vital aspect of success. As an industry we need to prioritise production of more evidence that takes into account the recommendations above.