Improving the evidence for market systems initiatives

We're hearing more calls for rigorous evidence, but what does it mean?

Just as markets do not exist in a vacuum, initiatives using market systems approaches are affected by wider issues in international development and social programming. Currently in this wider context we are hearing more calls for rigorous evidence. But what does "rigour" mean, particularly in the case of market systems initiatives? 

A recent thinkpiece commissioned by BEAM discusses concepts of "rigorous" evidence and examines the extent to which rigorous evidence exists for market systems initiatives. The piece draws on a series of consultations (interviews and a BEAM ediscussion) as well as wider literature about evidence and rigour from beyond the fields of market systems and of development. It aims to enable practitioners and donors to build on and improve existing monitoring and evaluation practices in order to facilitate better responsiveness to the social, economic and environmental realities of people living in poverty. 

Looking beyond methods

You may or may not be pleased to hear that the thinkpiece is not just another paper about methods. While methods are discussed, the piece largely focuses on other elements which are essential for generating rigorous evidence, including the scope of assessments and organisational cultures.  

One of the key arguments is that in order to be congruent with systems thinking and complexity theories, assessments need to go beyond the use of targets and indicators to assess the effects of programmes on both intended beneficiaries as well as other segments of the population, particularly those living in poverty. This should include not just the effects that we might anticipate (i.e. targets and risks), but should also capture the multidimensional effects that people find significant to their lives and environments.

Rather than advocating for a certain method, the piece presents principles for achieving this which can potentially be achieved using a broad range of methods.

Keeping it practical

You might be wondering whether expanding the scope of assessments is practical given the existing constraints on project budgets and staff time.  

It is important to recognise that not all studies need to be rigorous in order to be valuable and not all programmes are of a scope or significance to merit investment in rigorous assessments. For those programmes that do merit efforts to produce rigorous evidence, the paper presents suggestions of how this could be feasibly done within reasonable budget and time parameters. Some of this can be relatively simple, such as including a wider array of perspectives (i.e. going beyond beneficiaries and other people with a direct stake in the project).  

The full paper with analysis and recommendations can be read here.

Elise Wach is the Monitoring, Evaluation & Learning Advisor at the Institute of Development Studies.