aes24 International Evaluation Conference

Authors: Michael Limerick (Myuma Pty Ltd ),Melinda Mann (Myuma Pty Ltd, AU),Melissa Osborn (Myuma Pty Ltd, AU)

Emerging best practice principles for Indigenous evaluations encourage evaluators to find new ways of conducting evaluations of programs delivered in First Nations settings. The impetus for this work is a growing awareness that evaluation activity carries the risk of perpetuating colonising impacts on First Nations people, especially in relation to the sovereignty over knowledge and data, the level of consent and self-determination in the process, the level of appreciation of cultural insights and community strengths, and the sharing of the benefits of evaluation activity. For the evaluation of the Community Justice Group (CJG) Program in Queensland, the Department of Justice & Attorney General engaged our organisation, an Aboriginal social enterprise from Queensland, to deliver an evaluation guided by best practice Indigenous evaluation principles. Encouraged by the Department's evaluation brief, our organisation assembled a team of predominantly Indigenous people with deep community connections to facilitate a strengths-based and collaborative approach that would put First Nations voices and perspectives at the centre of the evaluation. Over three years, the team followed a process of working with CJG staff and members to co-design and deliver place-based 'local evaluations' in 25 locations, as the central feature of the Statewide program evaluation. The goal was to 'walk alongside' CJGs to respect their agency and afford them growth opportunities, and to seek out stories of success rather than evidence of deficit. Working in partnership, our organisation and the Department learned much on this journey. Fully implementing Indigenous ethical evaluation principles was not without its challenges - for example, meaningful participation can only occur by relationship-building that takes time and stretches evaluation budgets, and principles such as Indigenous data sovereignty can be difficult to implement in government contexts. However, the value of the approach is evident in firstly, the way that many CJGs embraced the local evaluations, and secondly, in the powerful qualitative evidence of program success yielded by the Appreciative Inquiry-inspired storytelling methods.

Authors: Ayesha Boyce (Arizona State University), Aileen Reid (UNC Greensboro, US)

Evaluation can be positioned as a social, cultural, and political force to address issues of inequity. We co-direct a 'lab' that provides new evaluators with hands-on applied research and evaluation experience to support their professional development. We are proud of our social justice commitments, and they show up in all aspects of our work. We believe the next generation of evaluators must be trained and mentored in high-quality technical, strengths-based, interpersonal, contextual, social justice-oriented, and values-engaged evaluation. We have found that novice evaluators are able to engage culturally responsive approaches to evaluation at the conceptual level, but have difficulty translating theoretical constructs into practice. This paper presentation builds upon our experiences and previous work of introducing a framework for teaching culturally responsive approaches to evaluation (Boyce & Chouinard, 2017) and a non-course-based, real-world-focused, adaptable training model (Reid, Boyce, et al., 2023). We will discuss how we have taught new evaluators three formal and informal methodologies that have helped them align their values with praxis. Drawing from our work across multiple United States National Science Foundation-funded projects we will overview how the incorporation of photovoice methodology, just-in-time feedback, and reflective practice have supported our commitments to meaningfully, and respectfully attend to issues of culture, race, diversity, power, inclusion, and equity in evaluation. We will also discuss our thoughts on the implications of globalization, Artificial Intelligence, and shifting politics on evaluation capacity building and training of new evaluators.