Implementing evidence for real-world impact:
How far have we come and where are we going?
In October, CEI co-hosted the biennial global event, the Evidence and Implementation Summit (EIS) in Melbourne, with Monash University and the National University of Singapore Centre for Behavioural and Implementation Science Interventions (BISI).
We invite you to share in the following reflections – across key themes that emerged at the event, from more than 150 presentations in person and more than a hundred pre-recorded.
Catch up on sessions you missed via the EIS portal (available to delegates until September 2024) or purchase your access today.
We are already looking forward to the next Summit in 2025!
“Who are we listening to?”: The importance of centring on community
Invited speaker Dr Vivian Tseng kicked off EIS2023 with this well-placed but challenging question, following a heartfelt Acknowledgement of Country from Aboriginal Elder and musician Ganga Giri. Vivian is a passionate and thoughtful advocate for democratising evidence, and she asked us to reflect deeply on what it means to truly “centre our work in the community.”
Vivian unpacked the three principles that drive her work as President and CEO of the Foundation for Child Development in the USA: 1) centre communities marginalised by oppression; 2) embrace historical perspectives; and 3) contend with power imbalances.
Later, invited speaker Professor Maureen Dobbins (McMaster University) cited trust and time as fundamental elements in the very human “messy reality of implementation”, no matter who we are working with. Equal partnership – being “100% committed to making [partners’] work more effective”, in Maureen’s words – is the glue that builds trust and success, with practitioners and community.
These themes – of involving community and being guided by lived experience – emerged across numerous presentations. These included a Co-design Compass tool developed by Queensland Children’s Health and QUT enabling an evidence-based, context-adaptive process in healthcare; the Victorian Aboriginal Childcare Agency’s co-designed initiative to identify and describe cultural practice elements to support work with Aboriginal children and families; a COVID health engagement project co-conducted with culturally diverse communities by the Australian Centre for Health Services Innovation; standards for community engagement in the infrastructure sector co-designed with professionals and the community, spearheaded by ANU; the work of Neami and ALIVE to evaluate five pilot Head to Health Centres alongside individuals with lived experience acting as co-researchers, mentors and guides roles; and work to improve social care practice through a focus on the ‘personal outcomes’ important to each individual, led by Scottish Social Services Council.
The time and space needed to sit, listen and work sustainably with diverse stakeholders needs to be acknowledged. As Maureen Dobbins reminded us: "This work is long, slow and really, really hard.”
This theme was particularly resonant because of the recent untimely passing of a giant in the field, Professor Richie Poulton, Director of the Dunedin Multidisciplinary Health & Development Research Unit. “Listen at the feet of the community,” Poulton had said. His ground-breaking work on the longitudinal Dunedin Study has offered unparalleled insight into human health, development and behaviour over five decades, producing more than 1,300 publications and reports, and influencing policymakers globally.
“The more sophisticated we get, the more siloed we become”: The importance of cross-disciplinary dialogue and learning
The breadth, depth and increased sophistication of the implementation work presented at EIS 2023 was enthusiastically welcomed, with projects presented ranging from kindergarten interventions to the construction sector.
But the need to keep pushing out of our silos and comfort zones was also raised. Invited speaker Dr Bianca Albers (Institute for Implementation Science in Health Care, University of Zurich) put on her self-described “grumpy pants” to critically assess some of the structural barriers that keep implementation science “treading water”, despite the fact “we know implementation matters.”
Bianca contended that implementation scientists – herself included– are too often the “clip-on sunglasses to research lenses that have already framed a worldview”; brought in too sparingly and too late to properly influence evidence uptake. A lack of “lighthouse institutes” is also an ongoing challenge to building the field, as is the necessary but often ignored need for slow, considered time for theory development. And “failure is not fatal”, Bianca cautioned, championing the important learning that comes from documenting missteps alongside success.
The benefits of working across disciplines and in collaboration and partnership were highlighted by many presenters, including: the Behavioural Insights Team sharing evaluation learnings across policy contexts (“When does an unpaid tax look like an unnecessary antibiotic prescription?”); the model of a three-year flexible learning partnership to support scaling between non-profit CARE and advisors IDInsight; 10 years of collaboration between 10 hospital, research and academic organisations to implement DNA sequencing into healthcare through the Melbourne Genomics Health Alliance; and a collaborative evaluation model in mental health through Orygen’s Individual Placement and Support services. In his dialogue with CEI’s CEO Dr Robyn Mildon, invited speaker Andy Feldman (host of the GovInnovator podcast) included mindfulness practice as part of the toolkit for leaders and policymakers.
Professor Bryce McLeod described the benefits of a Common Elements approach in providing more accessible and common language, in distilling what works for who under what conditions, and importantly, in working with community organisations to tailor these elements into application.
Invited speaker Dr Eric Barends (Centre for Evidence-Based Management) reflected on a surprising example of interdisciplinarity: how he and colleagues specifically drew on principles from evidence-based medicine to develop discipline of evidence-based management, now being taught in business schools worldwide. “Most people use too few sources and pay little or no attention to the strength of the evidence in decision-making,” Eric said, noting that we need to consider “critically evaluated evidence from multiple sources to increase the likelihood of a favourable outcome.”
“A shared language”: Building the field
What we perhaps didn’t expect would bring us together, according to Professor Nick Sevdalis from the National University of Singapore’s Centre for Behavioural and Implementation Science (BISI), is theory. With his own “grumpy pants” on, Nick took us on a journey through the use (and misuse) of implementation theory.
“Theory saves time”, because it “allows us to develop and organise incremental learning, and to share those learnings globally,” Nick argued. As a “set of principles that gives structure to our observations,” theory lends quality assurance in a fast-paced world and – perhaps most importantly – allows us “a shared language” to explain, communicate and debate.
The drive to build the field – and, in doing so, tackle growing global challenges like inequality – was a recurring theme among invited speakers. In concurrent presentations, reflective thinking and new tools were highlighted, including: tensions and paradoxes in scaling up (Dr Harriet Koorts); a model for calculating the financial cost of implementation (Dr Thomasina Donovan); models for implementation support practitioners (Dr Caryn Ward) and “embedded” implementers (Dr Sarah Hunter); evidence for fast-paced policy contexts (Eleanor Williams); why “early outcomes” might be the “sweet spot” of metrics (Dr Ben Tan); as well as reflections and learnings from 10 years of applying implementation in family services (Dr Jenna McWilliam and Jacquie Brown) and 25 years of observing training practices (A/Prof Brendan O'Hanlon).
Scaling continues to be an area of particular interest and focus – and a much-needed element in field-building. In her address on scaling, Jane Lewis, CEI’s Managing Director in the UK, cited the galvanising observation that not delivering useful interventions at scale potentially drives inequality. But scaling also “involves opportunity cost,” she said. “We need to make active, judicious choices about what we do and what we don’t scale. There are also inevitably tensions and trade-offs between reach and quality and cost,” particularly in reaching our most marginalised communities.
Case studies and ongoing studies in scaling were presented in concurrent sessions, including: the ENGAGE program promoting children’s self-regulation skills, expanding to almost 2,000 centres across New Zealand; Movember’s global work to scale 17 promising projects focused on improving mental health for men and boys; the Get Outside, Get Active program aiming to increase outdoor free playtime in Australian early childhood care centres, assessed for efficacy, scale-up and sustainment; reflections on the first two years of the Australian Education Research Organisation (AERO) guided by common reasons why evidence translation falters; and the roll-out of Victorian Early Parenting Centres working with fidelity to an evidence-based framework.
“Evidence is for everyone”: Moving forward with our shared mission
Eric Barends reminded us that probably our greatest shared issue is the growing distrust of experts and the rise of “alternative facts” and mis- and dis-information across all human debate: “That’s what we all need to fight.” In similar vein, Australian Government Assistant Minister Dr Andrew Leigh MP, opened with the observation “evidence is for everyone” in discussing the new Australian Centre for Evaluation. The global need for good evidence well used seems more urgent than ever in a “complex and increasingly turbulent world” – as Dr Bianca Albers put it.
Which returns us to Dr Vivian Tseng’s initial provocation: the need to democratise evidence, to build from community and to invest deeply in partnerships.
The notion of a shared global mission resonates deeply within the EIS community. Gracie Tredget from Kings College London wrote on LinkedIn: “Without a doubt, we are so proud to be part of a diverse, inclusive and global community of professionals promoting the use and dissemination of implementation science in their work. We’ve learned so much… and we’re excited to bring back with us to the UK new learning and insights from others working in this area around the world.”
Cricket Soong from Solution Design in The Philippines reflected: “It was such a boost to hear other experiences in health, policy and governance” at EIS 2023. And Matt Gardiner, CEO of 54 Reasons, applauded a “wonderful few days of learning and being challenged at …[the] best and most important professional development there is for leaders in the social services sector.”
Plaudits to the many excellent speakers, heartfelt gratitude to our Summit partners, and thanks to everyone who participated – in person and online.
Stay in touch with the Evidence and Implementation Summit via Twitter and by joining the EIS email list.
The EIS Virtual series will continue in 2024 and we look forward to seeing everyone for EIS2025!
