To drive real, lasting change, we need to understand not only what is effective but also how it works to create positive effects, as well as for whom it works, and whether it is being applied as intended. But assessing the ‘how’ part is much more complex than just investigating the ‘what’.
Youth Futures Foundation has commissioned CEI’s UK team and our partners at Dartington Service Design Lab to help with this challenge, through development of Theory-Informed Implementation and Process Evaluation guidance. The goal is to inform and guide more rigorous evaluation methodology, rooted in the latest theories, models and frameworks of implementation science and program theory.
“Implementation and process evaluation (IPE) is the term used to describe research that determines whether an activity has been implemented as intended and how it worked, if indeed it was successful,” explains CEI Principal Advisor, Dr Stephanie Smith, who is leading the project.
“Evaluating program implementation involves interrogating whether, how and how successfully the intended activities were brought into practical operation, how they aligned with program theory, as well as how they interact with the context in which they operate.”
Understandably, this kind of complex evaluation can be challenging to undertake. However, strengthening the rigour of IPEs leads to higher quality decision-making for policy and practice. Starting an evaluation by examining the theory behind the program and its implementation improves understanding of what is driving any positive change, as well as how the program can be best replicated and brought to scale.
“Robust evaluation evidence is foundational to informing the kind of decision-making that can lead to socially significant levels of change for marginalised young people,” says Jane Lewis, CEI’s Managing Director in the UK.
“If we can grow the number and quality of theory-informed IPEs taking place in the youth sector, and across the social sector more generally, this will have huge potential benefit.”
“We’re excited to be creating a practical, ‘how-to’ guide deeply rooted in program and implementation theory,” Stephanie comments. “What we want to do is strengthen evaluation practice by synthesising the latest and most relevant theories, and give guidance on how they can be best applied to the conduct of rigorous evaluation.”
The CEI and Dartington team will work closely with Youth Futures Foundation and its Evaluation Expert Advisory Group, as well as a user-group of evaluators, to create a user-focused toolkit of written and multimedia guidance.
“With our colleagues at Dartington, we’ll be considering options such as a modular learning program of video walkthroughs, carefully curated learning and reflection activities, and an interactive guidance tool,” says Stephanie.
“This project is an opportunity to develop the most rigorous and cutting-edge guidance, which can build capacity in an under-powered area of program evaluation. Our goal is to inform and improve evaluation practice across the sector.”
The Theory-Informed Implementation and Process Evaluation guidance (TIIPE) is expected to be released in 2025. See the Youth Futures Foundation website for updates.