The home literacy environment has powerful effects on literacy and numeracy (Melhuish et al., 2008) and was particularly important at the height of the COVID pandemic (Andrew et al., 2020; Bayrakdar & Guveli, 2020). This is a propitious area for program innovation. This is a key area where effective interventions are much needed. By age seven children with a social worker are already experiencing a large educational attainment gap (Berridge et al., 2020; Sinclair et al., 2020).
CEI's evaluation of a family literacy programme, Our Skills, has now been published. This was carried out in partnership with Bryson Purdon Social Research for What Works for Children's Social Care in the UK. Our Skills aims to help parents and carers to support children's early reading, with the specific aims of improving parent/carers' confidence in and enjoyment of reading to children and understanding of phonics, increasing reading time, and improving children's learning attainment. It is a ten-session programme that uses play-based learning approaches and was delivered online for this evaluation because of COVID. The evaluation report is available here.
The programme was developed by Learning Unlimited working with the Campaign for Learning and an academic from the Institute of Education, UCL. It is an adaptation of an earlier programme, Family Skills. An evaluation funded by the Education Endowment Foundation (Hussain et al., 2018) had found no effects on literacy progress for eligible children, but further analysis by What Works for Children's Social Care indicated more positive impacts for children with a social worker (Sanders et al., 2020).
This pilot evaluation assessed implementation outcomes (feasibility, acceptability, and appropriateness) and evidence of promise. This involved program-specific measures of parent/carer attitudes and behaviour administered through an online survey in the first and final sessions and at 3-month follow-up, without controls. The evaluation was informed by the Consolidated Framework for Implementation Research (Damschroder et al., 2009) and involved observation, a bespoke system for tutor self-monitoring of delivery fidelity, registration, and enrolment data, and qualitative research with the program developers, tutors, and parent/carers who took part.
The evaluation found evidence of promise when the program is delivered at the intended dosage, with statistically significant changes in parent/carer reports in confidence, enjoyment, and understanding, and indicative (but not statistically significant) evidence of changes in parent/carer behaviours to support reading.
There were several challenges to full implementation. The eligibility criteria did not align with school data on children's social care involvement, particularly historical, nor did they identify what was seen as a coherent group of families with shared and distinctive characteristics and needs. For example, there was striking diversity in educational attainment among parents/carers, around a third having no or low educational qualifications and around a third having a higher education qualification. Participating online was challenging - while almost all families had Wifi only 15% had a laptop or tablet - and this inhibited some of the intended activities and dynamics. Only half of parents/carers attended five or more of the ten sessions, and sessions were generally shorter than intended.
Nevertheless, for the parents/carers who participated more fully, the program was very acceptable and largely appropriate, the main difficulty being the fit between the level of challenge and parent/carer needs. The program deliberately used the reading and phonics schemes employed by the participating schools and so was seen by schools as highly aligned with their approaches. Working online meant that school involvement was much lower than intended, which was disappointing given that the program centres the trilateral parent/carer-school-child relationship as key to effective support for children.
Our evaluation recommends testing hybrid or face-to-face delivery modes and a modular or shorter course, as well as reconsidering the definition of eligible families. The evaluation also advised on strategies to improve school and family take-up and leverage school engagement. We recommend further testing of an adapted and refined program through a pilot cluster randomised controlled trial using routinely administered school phonics and literacy measures, with an implementation evaluation, ideally as a hybrid II trial.
Suggested citation:
Lewis, Ott et al. (2022). Pilot evaluation of our skills. Centre for Evidence and Implementation.
References:
Andrew, A., Cattan, S., Costa Dias, M., Farquharson, C., Kraftman, L., Krutikova, S., ... & Sevilla, A. (2020). Inequalities in Children's Experiences of Home Learning during the
Bayrakdar, S., & Guveli, A. (2020). Inequalities in home learning and schools' provision of distance teaching during school closure of COVID-19 lockdown in the UK (No. 2020-09). ISER Working Paper Series. Retrieved from www.iser.essex.ac.uk
Berridge, D., Luke, N., Sebba, J., Strand, S., Cartwright, M., Staples, E., ... & O’Higgins, A. (2020). Children in need and children in care: Educational attainment and progress. University of Bristol, University of Oxford, 2020-05. Retrieved from www.nuffieldfoundation.org
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation science, 4(1), 1-15.
Husain, F., Wishart, R., Marshall, L., Frankenberg, S., Bussard, L., Chidley, S., ... & Morris, S. (2018). Family Skills Evaluation report and executive summary, Report to the Education Endowment Foundation. https://educationendowmentfoundation.org.uk
Melhuish, E. C., Phan, M. B., Sylva, K., Sammons, P., Siraj-Blatchford, I., & Taggart, B. (2008). Effects of the home learning environment and preschool center experience upon literacy and numeracy development in early primary school. Journal of Social Issues, 64(1), 95–114.
Sanders, M., Sholl, P., Leroy, A., Mitchell, C., Reid, L., & Gibbons, D. (2020). What Works In Education For Children Who Have Had Social Workers? Summary Report. What Works for Children’s Social Care. https://whatworks-csc.org.uk/wp-content/uploads/WWCSC_what_works_education_children_SWs_Feb20.pdf
Sinclair, I., Luke, N., Fletcher, J., O'Higgins, A., Strand, S., Berridge, D., ... & Thomas, S. (2020). The education of children in care and children in need: Who falls behind and when?. Child & Family Social Work, 25(3), 536-547.