Maxwell Stamp's research, funded by BEAM, is yielding fascinating insights into how market systems programmes are achieving impact at scale. Discover what the main challenges initially identified are, and how they could be addressed.
See the webinar resources, including video recording.
A complex and initially confusing picture emerged from our review of dozens of interventions across different market systems programmes throughout Africa and Asia. The good news is that there are some impressive interventions and convincing success stories. But at the same time, the research revealed some common (although not universally applicable) structural weaknesses in current approaches to project design, implementation and reporting.
Our findings also confirmed our individual experiences; we all know that expectations can be unrealistic, credible and committed partners can be few and far between, some interventions work better than others and commercial pressures tend to encourage us to focus on the positive. The recent BEAM Exchange conference in Lusaka also highlighted that donors and implementing partners alike were somewhat frustrated by inconsistent intervention performance and by a lack of balanced evidence.
So, based on our findings, the research team will suggest a number of refinements to general practice. We are not claiming to have any simple 'template' solutions. But with further work and dialogue with practitioners within this research initiative, we believe these suggestions could have important implications for the way that market systems programmes plan, manage and report on interventions. Our intention is to develop these suggestions as a toolkit to assist practitioners.
Our hope is that by refining the way we manage programme design and implementation, we could see a significant improvement to the chances of interventions breaking out of the pilot phase and achieving impact at scale.
At this preliminary stage, our research suggests that there are at least four areas where general practice of M4P/MSD programmes could be refined in order to improve scaling up success rates:
- Strategic planning: Setting realistic targets and having a workable strategy is an essential starting point to ultimately achieving impact at scale. We need more rigorous pre-investment analysis of the human and economic parameters, dynamics and potential at a sector level. We have observed good procedures for such analysis in some of the programmes we have visited. This contextual, high level analysis can determine intervention scale potential and may inform intervention strategies.
- Portfolio approach: But with the best strategic planning in the world, not all interventions will go according to plan. The question is not whether every intervention works, but whether a portfolio has enough successful interventions to generate results that justify the total programme investment. An explicit portfolio approach would recognise the inevitable variation in intervention results, help to manage expectations and allow greater transparency.
- Understanding the private sector perspective: In most interventions the reaction and commitment of private sector investors (both the original partner and others) is critical to achieving sustainable change and impact at scale. However, very few programmes have developed or applied a structured approach to critically assessing the potential return on investment of the proposed intervention (for example using financial modelling or calculating net present value of future cash-flows). A deeper understanding of the investment decision making process – which goes beyond looking at break even and profitability – would help programmes to engage with the private sector.
- Transparent and comprehensive reporting: Even in well-run and well-documented M4P/MSD programmes, reporting of results is not always transparent, consistent, coherent and comprehensive, nor is the reporting of the costs by which such results have been achieved. A M4P/MSD project needs to know how it assesses value (impact for large numbers of beneficiaries delivered by market system changes) for money comprehensively across the board in its portfolio, and to make sure that its reporting system delivers this assessment across the board.
We will be publishing our research in full before the end of the year, and we intend to address these areas, and perhaps others, in further depth in the practitioners’ toolkit. The toolkit will help programmes devise scale strategies, make portfolio management decisions, engage with the private sector and report impact and outcomes (e.g. partner investments) and market change results against costs, in sectoral terms and broken down by specific intervention.
While we think the guidelines will be helpful, our objective is not to promulgate any radical alternative approach or to supersede established operational guides or reporting standards. Rather, we want to provide practical advice on how to achieve impact at scale, and stimulate programmes to acknowledge the fundamentals of the value for money equation in M4P/MSD programmes. For that, we welcome your suggestions.
Webinar
We have met many dedicated and skilled practitioners (to whom we are enormously grateful) and we have witnessed some innovative and exciting interventions. Through the observations sketched out in this blog we hope to begin to repay some of the cooperation that we have experienced with an outline assessment of leading challenges.
The next step in our research project is to run a webinar (on 28 July) to present initial findings, exchange ideas with practitioners and to set out the structure and contents of the toolkit. We look forward to a lively and fruitful discussion. Join us!
Find out more about the research, or contact us at research@maxwellstamp.com.
2 comments
Feedback on Research
HI there James
this is really exciting the research you are conducting. Thanks for sharing an update. Just a couple of comments:
1. Under Strategic Planning you state: "We need more rigorous pre-investment analysis of the human and economic parameters, dynamics and potential at a sector level." I certainly agree with you here. I am curious as to whether in your recommendations or conclusions you will be able to provide some frameworks with formulas that will guide us in conducting a pre-investment analysis?
2. What a brilliant term-- portfolio approach-- to explain the necessity of several interventions to address the root cause of market failure that afflicts the markets that we often work in. Addressing market failure in complex markets involves several responses necessary to address the root cause of the market failure-- your portfolio approach is a great way to describe addressing market failure in a complex market
But I don’t really understand how a portfolio approach would ‘recognize the inevitable variation in intervention results’. It seems to me that in a complex market, market failure requires diverse interventions to be facilitated to respond to the runaway market failure. So then the various interventions within the portfolio of interventions are addressing distinct aspects of the market failure—thus of course there is going to be variation in intervention results.
Maybe I am missing something, so could you please explain better what you mean how a portfolio approach “would recognize the inevitable variation in intervention results, help to manage expectations and allow greater transparency.”
3. Under the section private sector perspective, is it really just a case of calculating net present value of future cash-flows for a private sector partner? That seems to be a bit limited. For sure it would make a business case for a private sector actor or firm to get on board, but what about a collaborative discussion about the value the participation would bring to both partners (NGO/donor & private sector actor/firm)? And to achieve this value what activities do the partners need to engage in (promotion, offering of TA, distribution, production) and then what value is attributed to these activities (financial, human resources, time)? The Partnership Canvas provides a nice framework for this https://valuechaingeneration.com/2014/10/17/the-partnership-canvas/
Cheers
Mary
Response to Mary's questions
Mary, thank you so much for your questions, and apologies for the delay in responding (am currently on holiday, but never so far from the computer!).
To answer your questions:
1. As well as the research findings, we will be separately publishing a toolkit for practitioners, which will include a suggested framework and guidance for a process of pre-intervention analysis. This is currently work in progress. Without being prescriptive, we will draw on our experience and the research findings in order to set out an approach (steps and tools) that we hope will help programmes to assess the scale potential of an intervention before identifying partners or investing in a pilot.
2. Our "unit of analysis" for the research is at the sector level, and we believe that this is a very useful way of looking at programme results in other contexts. Sometimes, programmes will undertake one intervention, but often multiple interventions, designed to achieve sector-wide change. By portfolio approach, what we mean is that programme funders and implementing partners should explicitly acknowledge that some sector interventions are likely to work better than others, and (more importantly) that some sectors will respond more than others to those interventions. Of course, we invest in sector interventions because we believe that they will achieve results, but at the programme level we should have in mind that there is likely to be a wide distribution of actual results around the intended results. With a portfolio approach, we hope that there would be no "shame" in reporting individual intervention "failure" in the context of the whole programme results, which should be judged as a whole rather than on an intervention by intervention basis. This clearly has implications for the way in which we assess and report programme results, with much more emphasis on indirect sector wide impacts.
3. On private sector engagement, our underlying observation is that MSD programmes are generally much better at articulating the "development case" for a sector intervention than they are at setting out the "business case". I suppose that this is to be expected considering that interventions are funded by donor agencies and implemented by development professionals? I absolutely agree that there are many components to the business case (and thank you very much for the link to the Canvas). But at the end of the day, private sector investment is driven by risk adjusted returns. This is especially relevant to scale, which crucially requires indirect effects arising from competitive/non-competitive response. If we do not have a clear view of what those likely returns might be, we will continue to shoot in the dark. DCF is a very useful tool, but certainly not the only one!
I hope that these responses go some way towards answering your questions? There is still a lot more thinking to go into our deliverables and your questions and challenges are most welcome - thank you!
Add your comment
Sign up or log in to comment and contribute.
Sign up