Integrating qualitative and quantitative information

Tags: challenge funds, data management, qualitative data

This is one of a series of stories that complement the BEAM Monitoring Guidance. It offers a practical example of how a market development programme has solved a typical monitoring or evaluation challenge.

In 2014, the United Nation's Capital Development Fund (UNCDF) launched its Shaping Inclusive Finance Transformations (SHIFT) programme. SHIFT is a regional market development and women's economic empowerment initiative in the Association of Southeast Asian Nations (ASEAN) region. It works to transition 6 million low-income individuals and small and growing businesses from informal financial services to formal, low cost, well-regulated financial services by 2020. SHIFT aims to achieve this goal through systemic interventions, including through a challenge fund initiative, data analysis and information sharing, learning and capacity building, and policy and advocacy activities.

The challenge

The SHIFT challenge fund to date has received over 100 expressions of interest proposals from private sector financial service providers. It is currently funding 10 innovative, high impact investments/interventions. As the programme expands, it is expected that there could be more than 30 active investments at any given time throughout the programme. We receive reports (qualitative and quantitative information) from investment partners on a quarterly basis, in addition to other data such as field mission data, meeting minutes, etc. This amounts to a lot of data (both qualitative and quantitative) which needs to be effectively managed, monitored and used for programme decision making and learning.

The solution

SHIFT uses three tools to effectively monitor and manage the flow of this information into programme management:

1. SHIFT has developed Excel-based MRM dashboards which are used for overall monitoring and results management at investment/intervention level. This includes: background on the investment, a results chain, MRM plan (indicators, responsible party, measurement tool), systemic change model, quarterly reports (qualitative and quantitative) and projections. Information is uploaded (manually) into the dashboards on a quarterly basis and as needed.
2. A quantitative aggregation dashboard was also created, to support the tracking of SHIFT's progress at the intervention and programme levels. Given that quantitative partner reporting is collected in separate Excel files from each partner, in order to better automate the process we have undertaken the following:

• We lock the reporting format so that partners can not alter the format;
• We then created a hidden page using the index and match functions. This allows data to be collected in a usable form for aggregation in the hidden page;
• We then import all of the reports into STATA, which allows us to quickly aggregate the data from the hidden pages;
• We then export the data back into Excel and are able to create pivot tables and slicers which allow for easy user operability. Simply speaking, slicers allow the user to click on a button/indicator i.e. women or men, location, etc. and the pivot table will change based on the data you have selected to be analysed. 

Whilst the initial set up of such systems was time intensive and needs further roll-out to the entire programme team for use, the initial results show that the programme is more effectively able to manage data/information and that the dashboards allow for a comparable analysis and analysis of projections vs. actual results for programme decision making, strategic planning and external stakeholder reporting purposes. Another important detail to note is that this approach was actually suggested and spearheaded by the programme data specialist and not the MRM expert. This is important, as it is recognisd that results measurement is the responsibility of the entire team, not just one person.  

1. With quarterly information coming in for each investment it is important to be able to effectively cross-reference such data for learning, identification of similar challenges/risks, and identification of best practices. This is especially true for the qualitative information. As a result, SHIFT uses the software Nvivo[1], which supports qualitative analysis, allowing all files/information on all investments (and other programme information from the data, policy and learning components of SHIFT) to be aggregated and effectively analysed.

The qualitative analysis allows for the programme to analyse much more than simply the investment reports that are coming in. For example, it allows all programme information to be stored, including reports, proposals, notes, even videos from SHIFT events/conferences. The team are then able to perform information searches through hundreds of documents and data sources in seconds. This allows for quick comparability and analysis of information based on the search criteria.

Additionally, this allows the programme to understand the market dynamics and where innovation is likely to appear or where there is a perceived need. Furthermore, this may allow SHIFT to clarify and confirm/deny initial programme theories or ideas and possibly identify key areas to work on. The benefits of this specific tool have yet to be fully harnessed as the system has only recently been put into place.

Do you have anything to add, or want to ask a question? Please comment below or contact the author.

To learn more, see the BEAM Monitoring Guidance on documenting and accessing monitoring data.