Advice on measuring improvements in market access

Reply on DGroups | 12 comments

18 Oct 2018, 8:05 p.m.

George Bete

Dear Jenny

The indicator looks a too broad for me. However, my assumption is that by including it in the program design there is some kind of hindrance that your beneficiaries face in accessing markets.
I am imagining bottlenecks related to:

a. Financial- i.e- purchasing power- for this normally an indicator related to this can be formed

b. Context related- security, gender/ cultural practices ( disaggregated to specific groupings of your beneficiaries)

c. Infrastructure related- potentially seasonally impassable roads- Cash for Work (CfW) normally addresses this smoothly,

Now, depending on your program objectives I would suggest that you play around and identify the constraints that beneficiaries will have to overcome ( perhaps the broad-topics above can help) in order to access the markets. If it is something that can be addressed by your program then formulate the indicators to measure the success. In general your data sources might be a mixture of general observations of your beneficiaries’ ( and other consumers) access to the markets/ improvements in context etc., beneficiary self-reports ( during Post Distribution Monitoring).

Pleased to learn how you would have worked your way through this discussion point!

Kind regards

George Bete |Economic Recovery and Development Coordinator (Roving)- Emergency Response Team
International Rescue Committee
122 E 42nd Street, New York, NY 10168-1289 |Rescue.org <http: www.rescue.org=""/>
Mobile: +16468753496 , Skype: betegeorge
[IRC eSignature Graphic]
Follow us [twitter-logo - 25] <http: twitter.com="" #!="" ircjobs=""> [facebook-logo] <http: www.facebook.com="" internationalrescuecommittee=""/> [LinkedIn_Logo - 25] <http: www.linkedin.com="" groups="" international-rescue-committee-irc-39119?gid="39119&trk=hb_side_g">

17 Oct 2018, 3:36 p.m.

James Omolo

Dear Jenny
Since you had a baseline situation before the response, do another survey somehow similar to the one done for the baseline situation (I believe one was done). Compare the figures (Volumes of trade, number of traders, variety of good, %tage of HH who source goods from the market) and see if and how it has changed.
James OMOLO
Cash Transfer and Social Protection Expert REOWA - UN FAO, Senegal
Tel.       +221 77 292 22 13 (Senegal)              +254 786 144240 (Kenya)                      +254 725 291507 (Kenya)
Skype: james.omollo1

Von: Jenny Swatton <[Hidden email]>
An: Markets in Crises <[Hidden email]>
Gesendet: 12:55 Dienstag, 16.Oktober 2018
Betreff: [mic] Advice on measuring improvements in market access

Dear colleagues,

I hope this finds you well.
We are looking for some advice and any guidance on measuring improvements in market access please. We have been asked to include the following indicator in a programme 'improved market access for the beneficiary community' but are struggling to decide how best to measure this.
Given that market access is multi-faceted we were thinking that we need a composite indicator that takes into account physical access, financial access, market functionality etc. but would love to hear from others.
Kind regards
Jenny

17 Oct 2018, 12:49 p.m.

Corrie Sissons

Hi Jenny,

Just to add a few concrete examples of specific indicators around market access :

* OFDA’s 2018 Guidelines have at least one for Physical Access (p.114) https://www.usaid.gov/sites/default/files/documents/1866/USAID-OFDA_Proposal_Guidelines_February_2018_0.pdf

‘ Amount of market infrastructure rehabilitated by type (kilometers of market feeder roads, kilometers of ditches, kilometers of irrigation canals; number of bridges; other)

* The IASC Key Humanitarian Indicators also have a physical access one : https://interagencystandingcommittee.org/needs-assessment/documents-public/key-humanitarian-indicators

‘% HHs without physical access to any market’

That said – I agree with Karri that the lack of access is often context specific and can be physical / social / financial - so something akin to the spider diagram (although less neat than the above) is likely more relevant. You also do need to have something where change in access can be measured.

The Global Food Security Cluster is in the process of updating our Indicator Handbook <http: fscluster.org="" sites="" default="" files="" 11._indicators_handbookv4_1.pdf=""> (Section9) and this is an indicator we are discussing so this is a timely request!

The old SPHERE has some detail but the new SPHERE should also have updated guidance on this given its focus on cash and markets I hope : http://www.spherehandbook.org/en/food-security-livelihoods-standard-3-access-to-markets/

Best,

Corrie

Corrie Sissons | Oxfam
Emergency Food Security & Vulnerable Livelihoods (EFSVL) HSP | skype: corriesissons | email: [Hidden email] <mailto:[Hidden email]> |www.oxfam.org.uk

Internal Resources: EFSVL Toolbox <https: oxfam.box.com="" s="" 8j9v4cfanhclxkfdjo2t9j4y2euirugn="">; EFSVL on Compass <https: compass.oxfam.org="" communities="" humanitarian="" groups="" efsvl-emergency-food-security-and="" wiki="">; EFSVL on Workplace <https: oxfam.facebook.com="" groups="" 1535559343415230=""/>; EFSVL Induction <https: prezi.com="" rtzcqd_w_cgr="" a-guide-to-efsvl-in-oxfam="" ?utm_campaign="share&utm_medium=copy">

17 Oct 2018, 12:39 p.m.

Laura

Hi Jenny – I think I can only but throw a couple of ideas. What market are we talking about?
If it is job market , that’s the easiest: work permits and formalization of contracts;
A market of good and services: maybe looking at the good/volume exchange increased?
Maybe looking at mobility and access – that could be somehow measured – people moving.

Bests
Laura

16 Oct 2018, 6:01 p.m.

Hassan Olow

Dear Jenny,
I think this could best be informed by looking at what the programme is targeting regarding improving market access in this particular case. This should then inform which aspects of market access needs to be measured and reported on in this case. In a situation where the programme supports multiple dimensions of market access, I would say, then the focus could be on the one with the biggest influence to be used for measuring and reporting against the indicator.
Best regards,Hassan Olow

On Tuesday, October 16, 2018, 3:55:51 PM GMT+3, Jenny Swatton <[Hidden email]<div class="original_message_link">Original message
> wrote:

Dear colleagues,

I hope this finds you well.
We are looking for some advice and any guidance on measuring improvements in market access please. We have been asked to include the following indicator in a programme 'improved market access for the beneficiary community' but are struggling to decide how best to measure this.
Given that market access is multi-faceted we were thinking that we need a composite indicator that takes into account physical access, financial access, market functionality etc. but would love to hear from others.
Kind regards
Jenny

16 Oct 2018, 5:04 p.m.

Simon Levine

Hi Jenny, hi all

You say you’re thinking of a composite indicator because 'market access' is multi-faceted. Indeed it is, but in general I would steer very clear of composite indicators of any kind. The minor problem is that they involve endless arguments on how to construct them - even if you can agree what goes in, there are then arguments about weighting, etc. That's the minor problem, though, The main one is that composite indicators rarely tell you anything helpful! If you find that market access has gone, what should you do? You won’t have a clue, unless you know why it’s gone down, in what way it’s got worse (which may or may not be something relevant to your intervention). So then you need to go into the disaggregated data to understand your composite indicator, all the while wondering why on earth you spent so much aggregating what you know have to deconstruct!
Worse still, if the indicator shows no change, you still haven't got a clue what is happening. It may be that some aspects of market access are getting better and other getting worse, but this will be hidden. (It’s more than somewhat relevant to you to know whether the aspects of market access that you are trying to address are getting worse or better.)

I'm not quite sure what the advantages of a composite indicator are supposed to be, but I can’t actually think of any (beyond having a nice number to put in a report that looks scientific, of course.)

So, what should you be looking to assess? (I prefer using the word 'assess' to 'measure' because in some cases it makes sense to have a scalar way of quantifying things, but not always.) I think this depends on your problem analysis. In the given situation that you are working in, what specific aspects of market access are a problem for the people you are trying to help? (Difficulty in getting there (perhaps for some people but not all?)? Costs? The price they get/have to pay? The reliability of suppliers of what they need/demand for what they are selling? Etc, etc.) Those would be good things to keep an eye on. You could then look at what you are trying to achieve, what changes you are trying to bring about and how (apologies for the jargon, but your theory of change). You'd want to keep your eye on all the relevant aspects there. In other words, the information that you would want to use in managing/redesigning an intervention, this would be a good start in your monitoring/assessment/measurements. I don’t want to offer any more specific suggestions - because the more generic an indicator is, the less likely it is to be relevant in any specific setting!

Good luck!

Simon

Simon Levine
Senior Research Fellow

Humanitarian Policy Group
Overseas Development Institute
203 Blackfriars Road
London SE1 8NJ
United Kingdom
Tel: +44 (0)20 7922 8224
E-mail: [Hidden email]
Web: http://www.odi.org.uk

16 Oct 2018, 5:04 p.m.

[Hidden email]

Hi Jenny,
As others have said there are lots of ways to interpret "improved market
access" and much of it will depend on what barriers the communities
are facing (with regards to market access now). A couple options I have
used or seen used:
1) A quick and dirty EMMA-like baseline is done early on in the project to
identify what the barriers are, design specific activities to address those
barriers, and use as a comparison point later (like the emergency overlay
of an EMMA, but without the emergency, just acknowledging the difference in
market access)

2) A spider diagram that used several data points to give a score on how
different communities are experiencing different aspects (i.e. road
quality, purchasing power, etc) of market access. Mid-line and endline
'surveys' also give scores and in this way progress over time is shown on
market access.

3) Outcome mapping/harvesting that focuses on market access. Here is a good
link <https: www.betterevaluation.org="" en="" plan="" approach="" outcome_harvesting="">
to that methodology. (Outcome harvesting was just mentioned at session at
the SEEP conference as a good way to measure systemic change.)

Good luck, and do let us know what you decide to use... this group is
obviously interested in the topic!
Karri

16 Oct 2018, 3 p.m.

Winnie

Dear Jenny,

Depending on where you project is located, one of the ways beneficiary communities have increased access to market has been on pooling together their business wares and or collectively bargaining for goods and services at affordable costs hence increasing market engagements. Secondly the formations of grouping s that enable them to engage while minimizing the costs of doing so. I therefore encourage you to look at indicators you have mentioned below but also formations of associations or groupings that encourage market engagements.

Regards,

Winnie
 
Winfred Mueni Mbusya
STREAM Coordinator
Somalia
 
ACTED I Act for change I Invest in potential
ACTED, Chania Road, Opposite LER Apartments, Kilimani | P.O. Box 21217 - 00505, Nairobi, Kenya
Cell: +254 725 530 336, +252 613 359 005 (Hormud) Skype: winfredmueni
Website www.acted.org I Facebook ngoACTED I Twitter @ACTEDKen_Som I Linkedin ACTED

16 Oct 2018, 3 p.m.

Marie-Pascale Verly

What’s the baseline? First I will say be very conscious about what you are trying to change and what is the current situation.
MPVerly

16 Oct 2018, 2:48 p.m.

Sarah J Ward

Hi Jenny (and Monty too…)

This is a question that has come up in some research and thinking I have been working on lately, and it seems to be related to an even wider one…when we say there is “market access”- there is little agreement across our field what that means. We use it when we say there is “sufficient market access” to indicate cash transfers, as well as when we claim we are having positive effects on markets with our work, or positive effects on consumers.

Even when we use a very simplistic measure – like how far away markets are (physical access), we have major qualifications and contextual issues (not necessarily a bad thing, but it has to be spelled out and thought out). A field staff we quoted below said it most clearly..

This is from a piece (still not final, but soon…) “The use of data needs to extend to the choice of modalities and the data points we use to make those choices. We need to acknowledge that many of these judgment calls are subjective and context specific. For example, if we talk about market access, how far is too far to walk? If we talk about a sufficient number of vendors, how many is enough? These definitions are fluid. ‘“Something we’ve been finding out in more recent market assessments and focus group discussions with beneficiaries and potential beneficiaries is that people are regularly traveling to other markets further away, and not because of vendors or a [humanitarian] program. Is it okay to have them travel 10km to use their vouchers if they’re already traveling to those markets anyway? I think it is OK. Some people might not.’”

Another issue raised after digging into to the ex-post of response and ‘when we did what and how’ was the idea of “most appropriate” response. We often say we will use cash, or another market-based response, when it is “most appropriate”, but in all the response analysis work there was not a real understanding or agreement of what makes it “most appropriate” over something else.

When we measure how we choose what responses to bring, and WHY that one, it feels like we need a stronger consensus on what types of metrics we are basing our choices on. We have this, somewhat, for the choice of modality if we have already decided to use cash, but not really if we need compare cash to another modality, like local/reginal procurement programs, hiring health workers or build back better trainings, hygiene campaigns or other options. As more and more sectors (like WASH, Health and Shelter) start making these comparison choices, I think we need to understand our measurements.

All this to say that this is a needed conversation and I would love to hear how other organizations are tackling it….

Looking forward to hearing from others!

Sarah

Sarah J Ward

Livelihoods and Economic Recovery in Crisis

<mailto:[Hidden email]> [Hidden email]

skype: sarahjward

+1 518 929 6975

LinkedIn: <https: www.linkedin.com="" in="" sarah-ward-5280196=""> https://www.linkedin.com/in/sarah-ward-5280196

16 Oct 2018, 2:01 p.m.

Monty Lynn

Jenny,

I don't have an immediate contribution but am working on collecting and
testing some related measures of pre- and post-intervention market
functioning. I'd be interested in learning more about what you develop/use.

Monty

Monty Lynn
Abilene Christian University

16 Oct 2018, 12:53 p.m.

Jenny Swatton

Dear colleagues,

I hope this finds you well.
We are looking for some advice and any guidance on measuring improvements in market access please. We have been asked to include the following indicator in a programme 'improved market access for the beneficiary community' but are struggling to decide how best to measure this.
Given that market access is multi-faceted we were thinking that we need a composite indicator that takes into account physical access, financial access, market functionality etc. but would love to hear from others.
Kind regards
Jenny