Four ways to increase experimentation in your theory of change

Forget static TOCs, these four evolving management practices can spark ideas for experimentation in your TOC and in turn make your analysis and strategies dynamic.

This blog is part of our series on adaptive management.

Experimentation is in our DNA here at iDE. It started decades ago with the vision of our founder Paul Polak and continues to this day in our innovative project portfolios in WASH, Agriculture and Finance. Maybe you have come across some of our successful design and business strategy experiments such as treadle pumps in Bangladesh or latrines in Cambodia.

What you may not know is that our project teams are also experimenting with how we action out our theories of change (TOC). We are proposing to see TOCs not as static, but as a dynamic and living perspectives of what we are trying to accomplish. While some aspects of the TOC remain unchanged - such as who we are trying to help or what we are ultimately trying to accomplish - a dynamic blueprint allows us to tailor approaches to an continuously evolving market. I’d like to share four evolving management practices from projects I work on here in Bangladesh - practices that we think bring the spirit of experimentation to our TOCs and help make our analysis and strategies dynamic. We seek to know our unknowns, actively pursue pilots, plan for “plumbing sessions”, and establish “micro-documentation”.

One: Know your unknowns

One of the greatest barriers to experimentation, is not realising what is known and what is still an assumption. Once I get absorbed in the day-to-day operations of the programming, it becomes difficult to untangle what are the assumptions that we have made about our TOC. For partners, donors and team-mates who come from more traditional development or from academic backgrounds, accounting for the string of assumptions in a TOC can be difficult. Acknowledging what you don’t know can be critical to establishing a dynamic team.  

We recommend that you actively acknowledge these assumptions and seek to review them on a regular basis. Perhaps even post the assumptions in a common location and have team-mates add evidence in printed photos or post-it notes about how the assumptions hold up. However, be sure to document these updates so you do not lose sight of from where you have come.

At iDE, in the SanMarkS (Sanitation Marketing Systems) project, the team has been actively engaged in reviewing these assumptions on a regular basis in two forums. Firstly, the full project team (about 40 people) discusses and debates our assumptions at annual planning workshops and quarterly strategy meetings at each field hub. Secondly, the Dhaka team dives into these assumptions in a workshop called a Plumbing Session which will be discussed later in this blog. The team has created a simplified TOC called the Market Report Card. The assumptions and building blocks on our Market Report Card are debated and revised yearly.

Two: Pursue pilots

We’ve found that embedding a culture of back-to-back pilots in projects is a strong way to experiment with a TOC. Often these small pilots are happening automatically within your operational team because of differences in geography, market context, facilitation style and past experiences. While it is difficult to document all these mini-pilots, it is an important part of understanding what experimentation is already happening and what experimentation should happen. By piloting different assumptions that already exist within your project or finding unique methods to engage with new stakeholders ,you can build confidence in innovating, while ensuring that you know what is actually happening on the ground. Both formal and informal pilots can be anything from branding options to implementation methods and are a catalyst for pivots.

We recommend that you have a brief concept note for each pilot with a defined timeline and objectives.  Also ensure that the team knows what will happen once the pilot has finished and how the results will be rolled-out. By putting a timeline on the mini-pilots you can move towards consistent implementation after the storming and forming phases of the project in the first couple of years.

At iDE, PROOFS (Profitable Opportunities for Food Security) project has been taking bi-monthly mini-pilots to directly test elements of our TOC. The mini-pilots run between each team meeting and the team is required to report on the learnings. Firstly the field team and then the intervention managers fill in a powerpoint template of trigger level information about what changes they are seeing in the market based on these pilots and if the the TOC needs to be revised. During bi-monthly meetings the team is encouraged to determine what to do with the mini-pilot results or even the difference in operational structures that have emerged as informal pilots.

Three: Plan "plumbing sessions"

In my experience projects have leaks and often quite a few of them. Leaks are places where project implementation continues while the strategy hasn’t been completely defined or areas where the team needs to pivot, but hasn’t figured out what direction to take. Plumbing sessions (go with me here, I work in sanitation) help identify these leaks and to prioritize what to address quickly and what can wait. This analysis helps to identify where the project is in the results chain and where things are going haywire.

We recommend that you take time on a regular basis to get out of your normal environment to review the results chain and/or TOC and find out where the leaks are with your core strategy team. This will prove critical in ensuring that you can explain your results, but also to create space to experiment in leak-filling methods. Tactics such as focusing on a specific challenge for a month (such as our PROOFS based Farm Business Advisor challenge) in focused research or setting very specific parameters for a six-month time frame (we are currently on a six-month latrine branding campaign) can help to fix critical leaks before bigger problems arise.

Quarterly in SanMarkS, the Dhaka based team reviews the TOC and discusses what is working and what isn’t working collaboratively. This information is done on post-it notes and then everyone has a chance to share the evidence for their statements. These are then prioritized and a strategy for how to fix the leak focusing is sketched out on a white board that remains up until the leak is fixed. The team checks off components as they are accomplished to create a visual reminder of progress.

Four: Establish "micro-documentation"

We find that documentation of experimentation doesn’t always keep up with the pace of change. A series of incremental changes can add up to big change that may seem off target for stakeholders and staff. For those stepping in and out of the project from management, partners or other country programs it can seem a bit chaotic and can lead to significant confusion.

We recommend that project teams create a strong method of documentation that fits the team’s writing and visual strengths. Graphical frameworks can be a great tool to ground your pivots in a strategic manner. We find that visual documentation is a stronger way to document these changes than text, as visuals translate more quickly across languages and cultures. Alternatively weekly blog-posts or intranet posts can support this documentation to the relevant project stakeholders

The SanMarkS team has an online Google Slides strategy presentation that is constantly revised.  Notes are made in comments boxes and in the space below the slide to record how these changes have been made. It is not the most glamorous document, but is shared easily with senior management and short-term consultants to establish a strong baseline. The team also tries to footnote as much as possible in formal documents to identify pivots in the strategy.

We hope that these four evolving management practices from Bangladesh can spark ideas for further experimentation in your TOC. Perhaps the first step is seeing your TOC as a living and dynamic document and not something that is just in a project proposal or an inception report. My final recommendation - print off a copy of your latest TOC, grab a pen and some post-it notes and see how experimentation can strengthen your management approaches and project strategy.

Jess MacArthur is the Team Lead for the WASH portfolio and is the Project Manager for Sanitation Marketing Systems (SanMarkS) with iDE Bangladesh. She began her time at iDE as a Market Research Lead and as a Technical Advisor in the WASH portfolio. Jess has trained as a Chemical Engineer and holds a MSc in Water Science, Policy and Development from the University of Oxford. Jess also holds six years of experience in education and WASH projects in sub-Saharan Africa, South Asia, Central Asia and Southeast Asia.

This blog is part of our series on adaptive management.