Evidence has shown that positive family, school and community interaction during childhood is a protective factor for young people at risk of delinquency. But how might we evaluate a program that aims to change lives by preventing poor outcomes before they happen?
Singapore-based TOUCH Community Services (TOUCH) – a multi-service social service agency that serves people across ages, races and religions – engaged CEI to evaluate its multi-year pilot for 9- to 12-year-olds with emerging behavioural and emotional difficulties. This program, named SPARKX, aims to enhance children’s self-management, family relationships and school bonding outcomes. SPARKX’s focus is on building these protective factors early on, with the ultimate goal of reducing a child’s likelihood of delinquent behaviours in adolescence.
“It is challenging to evaluate the effectiveness of an initiative like SPARKX, where the aim is to intervene early to prevent oppositional disorders and juvenile crime,” says Dr Grace Chng, CEI Principal Advisor in Singapore. “We can’t follow individual young people and their families for long enough to see how things play out, but we can look to evidence-informed predictors of delinquency risk.”
“There is strong evidence that improved family functioning, higher school engagement and stronger social bonds with the community are protective factors for young people.”
The SPARKX team from TOUCH worked with 130 families over three years, providing individual counselling and groupwork interventions for the young person, family counselling, a parenting seminar based on Positive Parenting (Triple P), and a Family Day activity to boost parent-child interactions. Three tranches of the pilot ran for approximately 12 months each, over the period 2019 to 2023.
CEI assessed SPARKX’s reach, implementation and effectiveness. A developmental evaluation investigated the first two tranches, leading to program refinement and improvements in data collection, followed by outcome evaluation of tranche 3, alongside program-wide examination of reach and implementation.
“Overall, the findings suggest positive changes, mostly for the young people’s caregivers,” Grace explains. “They reported reductions in their children’s emotional dysregulation and negative behaviours, reduction in their own use of coercive and threatening parenting practices, and less parental stress and family conflict.”
Most caregivers found the program beneficial. Many attributed their positive experience to the quality of their caseworkers’ relationship with them, including the caseworkers’ ability to share practical and feasible strategies – a key implementation learning.
“The length of the program, frequency of sessions and potential for post-program follow-up are other implementation considerations going forward,” says Grace. “Families often didn’t attend all sessions, but many needed more than 12 months of support. There were also continuing caregiver concerns about their children’s major life transitions and development as they head into adolescence. And in terms of program content, excessive device use by young people was a recurring theme.”
“The TOUCH team was highly invested in SPARKX’s monitoring and evaluation process – the need for timely and high-quality data, and the important role this plays in understanding what works and what doesn’t. This has resulted in really useful, actionable evaluation insights.”
Head of Impact & Research for TOUCH, Ms Stella Teo, affirmed the evaluation process, saying: “We value the monitoring and evaluation process which has enabled TOUCH to better report the effectiveness of our programs. Apart from our in-house Impact & Research function to build the organisation’s capability and be accountable to our stakeholders, we also partner external stakeholders like CEI to measure impact.”