RCT and market systems: an opportunity to gain insights

Tom Harrison on using RCT methodology in a market-system change programme.

The Business Innovation Facility (BIF) programme in the Myanmar garments market is fortunate to be planning a Randomised Control Trial (RCT) as a significant component of our interventions. How are we fortunate? I think there are at least three ways:

Locally generated and rigorous evidence

Evidence for the link between decent jobs and the increased profitability of garment manufacturers is at the heart of BIF's theory of change.

This evidence is critical for garment factory owners and managers to change their practices for managing their employees and processes, and we believe that this change in practice will unlock hundreds of thousands of decent jobs that can empower women in Myanmar. What better evidence can we produce to support this than that from a well-designed and run RCT?

Re-humanisation as the change process

For this intervention, we work together with Tufts University and Impactt Ltd. A team of researchers at Tufts is designing the RCT and Impactt is delivering the consultancy input to the 12 garment factories in the trial. We have already had so many useful insights from our partners before we have even begun the RCT. Both organisations in their different ways are teaching us about the change process that can arise from a combination of productivity-based interventions and social psychology. We are beginning to understand how the de-humanisation of employees could be causing firms to make poor management decisions, which are both harming workers and damaging the firm’s commercial performance. The RCT will measure how addressing this de-humanisation, combined with relatively simple industrial engineering skills, can transform a company’s profitability and its employees' lives.  

How factory owners learn

We are already thinking in a new way about market facilitation. In a number of BIF market strategies we reached the conclusion that a change in business practice prompted by BIF must be taken up by players across a market not only by the organisations we work with. But how does that happen in practice? Tufts is helping us to understand the way that owners and managers of Myanmar garment factories learn, and in particular how their networks function in spreading innovations. This should enable us to design and implement a systematic approach to disseminating the evidence that will emerge from the RCT. 

The RCT in practice

The RCT will work on the basis of an internal control group. This will be created by phasing the support provided by Impactt such that a treatment group of 6 factories gets support around 9 months before the control group. 

Looking at what happens to the production line that pilots a new way of working in each factory, compared to the whole factory, will provide a further source of information. We will perform an extended baseline over 3 months on all 12 factories before any support is provided which will help Tufts to manage the relatively small sample sizes. Then the ‘control group’ of factories will be asked to continue collecting data for the period before they get support.  

Impactt's methodology, which has been honed in their Benefits for Business and Workers (BBW), programme in Bangladesh, involves a lot of data collection, which also greatly helps the RCT. However Tufts will also perform in depth surveys as part of the baseline, after the first 6 factories have been supported, and then after the full 12 have been supported. A further survey after another year will enable Tufts to discern what the 'spillover' effects have been from having the support staggered over 18 months, and also what happens after the Impactt support ends.

Why we think the RCT will work

We have high hopes of the RCT for a number of reasons. The Tufts team has done similar studies elsewhere. They have for example contributed to the ILO/IFC Better Work programmes in the region. The garment industry in Myanmar has a relatively small number of players (around 250 factories) and, while there are differences between these factories in terms of size and sophistication, it is a reasonably homogenous group compared to many other markets where these differences may be much larger.  

Tufts has also developed techniques for managing the more significant differences between factories whilst maintaining the ability to randomly assign them to either treatment or control group. This is critical to removing systemic effects that could make it impossible to discern the results of the Impactt support. To help avoid any possible conscious or unconscious bias in our results it is also important that our colleagues at Tufts are from an academic institution and will publish their results in peer-reviewed journals.

Follow our progress

The design document for the RCT begins with the following words:

'Producing scientific evidence establishing the business case for humane working conditions is the holy grail in the working conditions literature. The question before us is whether we can characterise, understand the genesis of, and expeditiously remediate systematic management errors that particularly result in harsh working conditions.'

Is it going too far to suggest that using RCT methodology in a market-system change programme is also a potential major step forward for such programmes in which a market transformation depends on the development and dissemination of a business practice in the market? The BIF team is very excited by the study, and in conjunction with our colleagues at BEAM Exchange we would like to know whether there is wider interest among market system practitioners in what we are doing. If there is, then I would like to report back regularly as we implement the RCT, reflecting on what we are learning by doing an RCT and discuss these findings with other practitioners.

As a first step in this process, we held a Google Hangout to discuss in detail the suggested set-up. View details.

Tom Harrison is a member of the international team that is delivering the Business Innovation Facility. Tom has 25-years’ experience in the development sector and his whole career has involved taking a business approach to achieving sustainable development outcomes. Tom's main interest and experience is the contribution that markets make to improving the lives of poor people, and in particular as an accredited broker of cross-sector partnerships, how collaboration between the private sector, government and NGOs can address poverty and disadvantage.