aes24 International Evaluation Conference

Authors: Eleanor Williams (Australian Department of Treasury ),Josephine Norman (Victorian Department of Health, AU),Melissa Kaltner (Lumenia, AU),Skye Trudgett (Kowa Collaboration, AU),George Argyrous (Paul Ramsay Foundation, AU),Luke Craven (National Centre for Place-Based Collaboration (Nexus), AU)

Delivering great evaluations requires a strong professional relationship between those commissioning and delivering the evaluation, as well as all relevant stakeholders.

Traditional evaluation commissioning approaches have tended to treat evaluation as a one-off exchange focusing on the completion of pre-defined tasks. However, the evolving landscape of policies and programs tackling complex issues demands a more nuanced and relational approach to get the most out of the journey of evaluation.

This big room panel session brings together speakers who are at the forefront of thinking around collaborative commissioning partnerships from the perspectives of government, not-for-profit and Indigenous-led organisations, and the private sector who can play the full suite of roles on the commissioning journey. The discussion will delve into the experiences of a range of organisations involved in commissioning who are seeking to build enduring relationships, and in some case partnerships, between the commissioners, the evaluators and the stakeholders to whom we are accountable.

Drawing on real-world case studies and empirical evidence, the discussion will highlight the challenges and rewards of transitioning from a transactional model to a relational model. It will explore how this paradigm shift can enhance collaboration and ultimately lead to a range of positive outcomes.

Attendees will be invited to respond to engage in dialogue with the panel to bring the collective wisdom of attendees together and consider how the destination of better commissioning relationships would look, the practical obstacle we face on our pathway, and how we can reach our destination. To facilitate this active discussion, attendees will have the opportunity to use Sli.do throughout the session to provide input on key questions, share experience in real-time and ask questions of the expert panel.

Authors: Moderator:  Prof Rick Cummings, Emeritus Professor Murdoch University, AES Fellow
Key Speaker: Eleanor Williams, Managing Director, Australian Centre for Evaluation
Panelists: Prof Lisa Cameron, Professional Research Fellow, Melbourne Institute of Applied Economic and Social Research, University of Melbourne
Dr Wendy Jarvie, Adjunct Professor, Public Service Research Group, University of NSW
Bruce Cunningham, Assistant Secretary, Employment Evaluation Branch, Commonwealth Department of Employment and Workplace Relations
Commentators: Prof Patricia Rogers, Former Professor of Public Sector Evaluation, RMIT University, AES Fellow
Scott Bayley, Principal, Scott Bayley Evaluation Service, AES Fellow

The Commonwealth Government has established the Australian Centre for Evaluation (ACE) to put evaluation evidence at the heart of policy design and decision-making by improving the volume, quality, and use of evaluation evidence to support better policy and programs that improve the lives of Australians. This aligns well with the aim of the AES to improve the theory, practice and use of evaluation for people involved in evaluation. The creation of ACE provides an excellent opportunity for the AES and its members to work with a government agency on our common purposes. This collaboration has already commenced through shared activities and, in particular, the involvement of the responsible Minister, Dr Andrew Leigh, as a keynote speaker at the 2023 AES Conference.

An area that has attracted considerable attention is the mandate for ACE to include randomised control trials (RCTs) in at least some of their evaluation studies of Commonwealth programs. This issue was the central topic of Minister Leigh’s keynote address and created considerable debate and discussion at the conference. This demonstrates that this is a topic of importance for the AES and its members.

The aim of the session is to explore the appropriate use of RCTs in evaluation studies of public policy in Australia. The strategy is to commence a communication process on this key topic between ACE and the evaluation community as represented by the AES. Ideally, this will lead to collaboration between ACE and the AES to improve evaluation practice in Australia.

The Fellows Forum session will commence with a prepared presentation by a senior staff member of ACE explaining its mandate and outlining its approach to including RCTs in evaluation studies. This will be followed by a panel of evaluators who have experience with RCTs to explain how they included RCTs in an evaluation study or where they chose not to include an RCT and the reasons why. They will also explore what they learned from this experience to inform their future evaluation practice. Finally, one or two Fellows will act as discussants, responding to the previous presentations with their thoughts on this issue. The session will be moderated by a Fellow and there will be time for audience members to ask questions of the panel members and discussants.

Moderator:  Prof Rick Cummings, Emeritus Professor Murdoch University, AES Fellow
Key Speaker: Eleanor Williams, Managing Director, Australian Centre for Evaluation
Panelists:  Prof Lisa Cameron, Professional Research Fellow, Melbourne Institute of Applied Economic and Social Research, University of Melbourne
                       Dr Wendy Jarvie, Adjunct Professor, Public Service Research Group, University of NSW
                       Bruce Cunningham, Assistant Secretary, Employment Evaluation Branch, Commonwealth Department of Employment and Workplace Relations
Commentators:  Prof Patricia Rogers, Former Professor of Public Sector Evaluation, RMIT University, AES Fellow
Scott Bayley, Principal, Scott Bayley Evaluation Service, AES Fellow