This is a guest post by Kevin Wong, Associate Director – Policy Evaluation and Research Unit, Manchester Metropolitan University
The voluntary and community sector (VCS) should be measured on what the sector does well and does differently to probation and prison.
The current Probation review is the latest opportunity for Government and the sector to address the long-standing challenge of how best to measure the sector’s contribution to supporting offenders.
While backed up by sound foundations in desistance theory, adopting this approach requires both parties to be bold and visionary.
Why reoffending is flawed
While reoffending has become the dominant criminal justice measure of effectiveness, it shouldn’t be the only measure that is taken seriously. It benefits from being clear cut – whether an individual reoffends or doesn’t – if they do, how often? However, it has significant limitations. As a measure of desistance, it is flawed. Research by among others Shadd Maruna, Stephen Farrall, Anthony Bottoms, Fergus McNeill and Beth Weaver suggests that desistance is not a one-off event. People do not stop reoffending as a singular act, instead it is a back and forth process and like life more generally, sometimes a person makes progress and sometimes they have set-backs. Because there are many things going on in an offender’s life that influences desistance, whether or not they reoffend may not be solely due to the support that a single agency provides to that person. In addition, services have to wait a long time before they know if what they do makes any difference – at least 18 months for the reoffending measure used by Government.
Does the voluntary sector do things differently?
What does the sector say? Firstly, because most VCS offender services are voluntary – offenders choose whether or not be involved – they engage better. Secondly, the VCS offers holistic support generally dealing with all the offender’s needs – criminogenic and non-criminogenic. Probation and prison focus primarily on criminogenic needs. Thirdly, according to the sector’s advocates, Clinks and New Philanthropy Capital, the way the sector deliver their services contributes to desistance.
Is this borne out by the research evidence? At the Policy Evaluation and Research Unit (PERU), our work on VCS offender services for young adults, women and young people suggests that Yes, the sector do offer something distinctive. It seems to be about the way that they treat offenders – first and foremost as vulnerable people with needs; the way they deliver their services; and the type of services they offer. These findings seem to be confirmed by other researchers: Mike Maguire, Mary Corcoran, Anthea Hucklesby, Philippa Tomczak, Alice Mills and Rosie Meeks.
A fairer alternative
At PERU we think it is possible to measure these distinct benefits of voluntary services using the following measures:
- User engagement with the service;
- Changes in the needs that the service is directly helping the user with; and
- Changes in well–being, agency and relationships using standardised measurement tools
We recognise there are challenges in these measures, their precise contribution to reducing re-offending is not well established, however, there are precedents. Her Majesty’s Prison and Probation Service (HMPPS) previously commissioned researchers to develop what HMPPS called intermediate outcome measures (which contribute to reducing reoffending) to provide an effectiveness framework for voluntary agencies.
What we are proposing is not revolutionary, these measures have been regularly used by PERU researchers and others to evaluate the impact of voluntary services. However, what is different is using them in combination to also provide a proxy indicator of desistance. Engagement, non-engagement; dropping out from an agency then returning to the service; having their needs met but then encountering new needs and needing further help. This commonly reflects the journeys that users have with voluntary agencies. This particularly applies because the service is voluntary – the user has the choice to engage or not engage. Compared to probation and prison where engagement is required, offenders’ involvement with voluntary services most closely mirrors the non-linear processes of desistance.
Importantly, if the MoJ adopted this approach it would provide a large scale, longitudinal research dataset which could help us build better models of desistance. To date most studies have been small scale and/or qualitative.
To be clear, PERU researchers are not suggesting that all of these measures should be applied to all voluntary agencies. Measurement should be proportionate to the size and financial value of the service. Generally, voluntary agencies will record information on user engagement, user needs and occasionally well-being. This is not systematic but with adjustment to organisational processes, support and training, recording could be made more consistent and a regular part of what agencies do.
A basket of complementary indicators would be required to provide a nuanced assessment of effectiveness and mitigate against gaming. While engagement is important, keeping offenders ‘on the books’ without improving their circumstances would not count.
What should happen next?
The Probation Review is a golden opportunity for Government and the sector to re-think how best to assess its contribution to supporting offenders desist from crime. Reducing reoffending should not be the sole and/or main measure. Instead using engagement, needs and well-being as thematic outcomes gives voluntary agencies the opportunity to play to their strengths and provides complementary measures to reoffending.
Systematic collection of such data would provide Government with an unprecedented opportunity to quantitatively measure desistance with a robust dataset.
Because the VCS can access this data themselves, it gives them feedback about how well they are doing for their users and how they can improve what they do.
Recognising the holistic approach to service provision frequently adopted by voluntary agencies, acknowledges the sector’s contribution to promoting the social inclusion of offenders. Further, these thematic measures would represent a more profound shift, from viewing offenders primarily as risks that they pose to society but instead as people first – who yes have convictions but who also commonly have multiple vulnerabilities. Enabling individuals to address their vulnerabilities gives them the opportunity to be included as part of their communities, which benefits them and makes us a more socially just society.