aes24 International Evaluation Conference

Lead Research Scientist, Local Innovation Group, Massachusetts Institute of Technology (MIT), USA

What does a wayfinding approach look like when seeking to learn from and evaluate interventions into complex systems? 

Many of the most intractable challenges facing communities around the world are system challenges requiring system-level responses. Development-focused donors and implementers at various levels are recognizing this and funding system-strengthening and systems-change work across a variety of systems. Monitoring, evaluation, and learning work, however, has traditionally been focused at the project level, not the level of the dynamic local systems in which projects operate. A new kind of evaluation is needed for this work and is in the early stages of being developed, tested, and improved through learning-by-doing.

In forums such as the UNDP’s M&E Sandbox and the BMGF-funded Systems Monitoring, Learning, and Evaluation initiative, development donors, implementers, and evaluators are asking questions such as: what evaluation designs and approaches are most suitable for learning from and evaluating system and portfolio-level interventions? And “how do we know if we are making progress, generating results, and contributing to positive change in a complex system?”
Drawing on experience implementing “complexity-aware” evaluations of system-change interventions in Northern India and Guatemala, this session develops and explores responses to these questions. The presentation shares an evaluation approach and six related tools that are being used to evaluate, learn, and implement adaptively in these two very different system contexts. The tools--while humble and likely familiar--can become powerful wayfinding instruments for navigating complexity when combined with a systems-informed evaluation design. This session introduces this approach through a keynote presentation and then further develops it through an interactive panel with systems-informed evaluators working both internationally and domestically in Australia.