Module 6: Monitoring for adaptive management
Adaptive monitoring for adaptive management
Synchronising monitoring activities with the pace of implementation
It is important that a monitoring framework sets out a planned schedule of monitoring that indicates what will be investigated, and when. It is also important however to be prepared to adjust this plan if necessary, so that activities are synchronised with the pace at which interventions are implemented. For example:
- If the start of an intervention is delayed, monitoring activities will also need to be delayed
- In highly dynamic situations, monitoring may need to take place on a frequent, or even continuous basis
- In cases where significant effects can only be expected after considerable time lag, it makes sense to monitor for these results less often. This is also the case when reviewing changes in the wider market system, which should happen on a regular but not constant basis.
The intensity of monitoring efforts at particular points in an intervention can therefore be guided both by the expected trajectory of change as described in implementation plans, and by the observation of staff in the field. Increasing or reducing the intensity of monitoring efforts may be easier if the team members implementing the intervention are also the ones collecting monitoring data.
Which methods are appropriate for monitoring may also vary with the pace of change. Extensive surveys or narrative research studies may be appropriate where it is anticipated that monitoring will be undertaken at regular, fixed intervals. They will be less appropriate in periods of rapid change however, when informal observation by programme staff may have a better chance of identifying and understanding key trends and events.
Finally, while the majority of monitoring activities can be defined in advance, it is also advisable to allow some additional time and resources to answer specific questions or to undertake smaller studies commissioned to fill particular knowledge gaps. Where anecdotal evidence suggests that an important change is occurring for instance, additional ‘on-demand’ activities may be required to assess the relevance of these changes for the intervention, or for the programme as a whole.
Synchronizing monitoring with the pace of change in complexity: a complexity-aware monitoring principle
Guide on monitoring in complex situations and how to make sense of the data.
Identifying early signs of change
One of the more difficult issues to take for a team review is deciding if an intervention that is flagging just needs more time, or whether it is unlikely ever to produce the desired results.
An initial sign of a problem is likely to be provided by indicator data, showing that the results expected at a particular point in the implementation plan have not yet occurred. However, in interpreting these results it is important to understand how far implementation has progressed, in order to see what has been done at a particular moment in time compared to what was planned. For this to be possible, activities will also need to have been well documented, highlighting the importance of ingraining good documenting practices among the team.
To generate an understanding of the traction generated by an intervention, it is also important for staff to be very inquisitive, and to observe critically both the immediate effects of an intervention and the wider context. This highlights the need for staff to have a questioning mind in the field, as discussed below. Feedback from various stakeholders, based on narrative research methods or regular feedback sessions and interviews (Module 9), can be a good way of detecting and identifying early signs of change.
A further issue relates to the need to be aware of different possible trajectories of change. Linear relationships between inputs and results should not be expected in programmes that aim to achieve systemic change. Things may change quickly, or take a long time; results may also be disproportionally small or disproportionally large. Two very different trajectories of change illustrate the potential for variation:
- A J-shaped curve, where things become worse before they improve. For instance, initial setbacks occur in working with marginalised groups, with breakthroughs only occurring after some time
- An S-curve is typical for diffusion of innovations where, for instance, there is low initial take-up of a technology, followed by a rapid expansion as the majority adopt it, and then a slow down as uptake reaches saturation point.
The difficulty is that the exact shape of change cannot be predicted. Instead, it needs to be reviewed on an ongoing basis, using indicator data combined with a broader analysis of what is going on in the market system.