Ensure that your profile is set to public (not private). If your profile is private you will be unable to fully participate and will miss important conference announcements. The online program is currently invite only. Over the coming weeks we will grant access to presenters and delegates so they can create a profile in the online app and choose sessions they’d like to attend. Please be patient, if you are a registered delegate or presenter we will send instructions when we are ready to invite you. Register at:https://conference2024.aes.asn.au
Authors: Ghulam Muhammad Shah (ICIMOD), Farid Ahmad (ICIMOD, NP)
Over the past few years, we have conducted rigorous research to identify 'resilience markers' in the context of environmental management in mountain areas in the Hindu Kush Himalayan region. Our findings suggest that evaluations focused on OECD criteria, as well as traditional experimental and non-experimental designs, have limitations in capturing the dynamic and evolving nature of environmental changes. In such scenarios, a complexity-aware, learning-oriented, and adaptive MEL approach becomes crucial. This approach involves adaptive monitoring and management of socio-ecological behavior, incorporating feedback loops to adjust interventions and strategies based on responses from the entire system.
In our quest for the most suitable and fit-for-purpose design, monitoring, evaluation, and learning approaches to effectively monitor and evaluate resilience outcomes, we have introduced three unique concepts: The Intervention Design Effect (IDE), External Reinforcing Factor's Trap (ERFT), and Type-III Error in Evaluations. We acknowledge that these terms are not only novel but also innovative concerning the design of monitoring and evaluating climate change and environmental programs. We believe that these critically important concepts have been overlooked by the field of 'evaluation science,' and we are introducing them for the first time.
These concepts are lined up for publication, starting with an AEA365 Blog.
The underlying principle is fundamental - it is crucial to identify, challenge, and explicitly define assumptions and risks from both the internal and external contexts of an intervention. By doing so, we ensure that the intervention is realistic and relevant to its implementation context, increasing the likelihood of success. Explicitly defining assumptions helps stakeholders better understand the rationale behind the intervention, fosters a shared understanding, and allows for constant testing of assumptions throughout implementation. This adaptive approach ensures the intervention remains responsive to changing needs and circumstances. Poorly designed interventions become susceptible to the Intervention IDE. Investigating and understanding IDE is critical in program design, planning, monitoring, evaluation, and learning, optimizing design and maximizing the likelihood of detecting potential distracting effects. Neglecting IDE may lead to falling into an External Reinforcing Factor's Trap (ERFT), causing unintended consequences that supersede the intended outcomes of an intervention.
Simultaneously, using standard experimental, quasi-experimental, or non-experimental evaluation designs to assess resilience outcomes carries the risk of not considering critical scenarios emerging between baseline, midline, and endline evaluations. In statistical terms, missing decisive information between standard evaluation stages may lead to Type-I or Type-II errors. Unintentionally missing this crucial information introduces a Type-III error into standard evaluation designs.
The widely used indicators-based approach to assess the resilience of project/program outcomes is a top-down method, relying on assumptions that lack anticipatory resilience management of the socio-ecological system as a whole. This approach fails to capture dynamic interactions and complex relationships between the structural elements of a socio-ecological system, which determine the system's behavior.
