Why can’t we measure systemic change?

Why do we lack a precise definition for 'systemic change' despite over a decade of implementing market systems programmes with the explicit goal of systemic change?

Since starting to use market systems development concepts five years ago, I’ve been concerned with the disconnect between these MSD concepts and reality.

To do MSD well we have to understand the reality of systems we work in, so we need high quality information from diagnosis and monitoring and results measurement. Yet MSD concepts (‘core’, ‘supporting function’, ‘rule’, ‘system constraint’, ‘AAER’ etc) are too far abstracted from the actions and actors that make up social systems to be of use in gathering that information.

The connection from actor level to system level is not made explicit. As such we still lack a conceptual framing of systems or systems change that readily corresponds to what we encounter on the ground. And that is why we cannot measure systemic change - yet.

A new paper, building on the MOSC conceptual framework, sets out the requirements for a definition of systemic change that can more effectively support efforts to establish whether or not systemic change has happened.

There are three things we need to know.

1. How has the system changed?
2. Is the changed system adaptive and resilient?
3. What was the role of our intervention in causing the system changes?

Many readers will recognise these components of a definition. While their starting point is different, they aren’t so different from the findings of others who have sought conceptual clarity on this topic. See, for example, Taylor or Cunningham & Jenal. But these existing efforts build on MSD concepts and evolutionary economics concepts respectively, which lack explicit actor-system linkages and are, therefore, both difficult to operationalise for measurement purposes and inherently imprecise.

The definitions of the three components set out in What is systemic change? use instead the MOSC conception of systems. It has explicit aggregation links from actor to system built into the framework (see the figure above). This means that the definitions of systemic change described in terms of functions and systems can be translated into terms of actors and actions, and so may be readily employed to inform systemic change measurement and other information-gathering efforts.

Two notes of caution are necessary. First, the MOSC framework is nascent, and so definitions built on it are preliminary. Applied work is needed to test and extend our understanding of each component and how they might best be measured in practice.

Second, the paper does not address the challenge of figuring out where a line may be drawn beyond which change may be considered ‘systemic’.

The ambition, perhaps perverse, of being able to say, with an objective and measurable set of criteria, “that is systemic change, and that is not” is unlikely to be realised for a few years yet.