Where is the evidence base for accredited programmes?

Penelope Gibbs of Transform Justice questions the evidence base of the probation accredited programmes currently being put out to tender.

Share This Post

Share on twitter
Share on facebook
Share on linkedin
Share on email

Who will be left holding the baby?

This is a guest post by Penelope Gibbs of Transform Justice.

Last week the government published a tender for the new probation services contract . The publication was controversial because of the timing – just before a potential change of government — and because it is not clear what is being tendered for. The reform of Transforming Rehabilitation was desperately needed but maybe it demanded more haste less speed.

The basic idea of the reforms is to keep management of probation in the public sector but to outsource rehabilitation programmes and unpaid work. Unfortunately we don’t know whether most of the outsourced programmes work. They are “accredited” by HMPPS but the accreditation system is swathed in a cloud of mystery. It is not clear who accredits courses, nor exactly what criteria are used.

The government says that all accredited programmes should “demonstrate sound evidence that there is a commitment to monitoring the quality of programme delivery and to evaluation”, but there is no explanation of what this means. And we know that accredited programmes don’t necessarily work. The sex offender treatment programme was found, after it had been running for over 20 years, to lead to an increase in offending

An unquantifiable risk

The problem with the new TR model is that private providers are being asked to take on an unquantifiable risk. The two main programmes which will be outsourced are the Thinking Skills Programme and Building Better Relationships (BBR). Neither of these programmes has an impact evaluation. Building Better Relationships is targeted at those who commit domestic abuse. It replaced a programme (IDAP) which had been run successfully for many years and had a positive evaluation. BBR started in 2012. 15,090 people have been placed on the programme in the community and 683 in custody but no impact evaluation is even underway, let alone published. HMPPS say they are doing “evaluability / feasibility work….to inform the best approach for impact evaluation”, and that the programme will be “subject to an impact evaluation” in 2020.

The Thinking Skills programme “supports participants to develop thinking (cognitive) skills to manage risk factors, develop protective factors, and achieve pro-social goals”.  It too has been accredited but we don’t know outcomes: “Impact evaluation data [is] not yet available for the community due to issues with CRC data” [From a letter from HMPPS to Transform Justice]. None of the other programmes HMPPS is outsourcing has an evaluation published by government.

It’s not clear why so little progress has been made on impact evaluations when it is well known that many a great sounding programme can fail on implementation. The thinking skills programme is based on standard CBT (cognitive behaviour therapy) theory and practice, but we need to know whether it is working in context – do the trainers have the right skills, are the clients the right ones, do they stay the course? Impact evaluations indicate whether courses make a difference to reoffending, but they also provide learning on how courses can be improved. I’ve no doubt that current participants fill in feedback forms, but these are not enough.

Unevaluated

It seems unfair that private and voluntary sector organisations should be expected to run these unevaluated programmes. Bidders will be asked to cost out and commit to running programmes which have no track record. With no impact evaluations, there is no data on the right way to run the programmes. This is a tickbox approach – a far cry from the ambition of TR to reward results.

What will happen if the evaluations suggest the programmes are not working? Tenders will need to be submitted next year. The evaluations of BBR and the Thinking Skills programme will be “subject to impact evaluation” in 2020, but there is no commitment to a publication date. The successful bidders are supposed to start work in 2021. What if an evaluation shows one of these programmes to be a disaster in December 2020 (or later), after each bidder has spent months working out how to deliver the programme and also recruited staff? Bidders would then be left high and dry.

A slightly ominous passage in the tender suggests what a provider might do if one of the accredited programmes fails: “The Authority may from time to time request that the Supplier provide optional services. These include …delivery of additional structured interventions in the areas of emotional management, attitude, thinking and behaviours (ATB) and domestic abuse”. “Structured interventions” is a vague term. Presumably it refers to unaccredited courses made up by the provider. Last year the probation inspectorate found that CRCs were designing their own domestic abuse interventions:

“Some responsible officers were delivering the domestic abuse RAR [rehabilitation activity requirement] on a one-to-one basis, borrowing resources from colleagues, browsing the internet for resources or devising their own one-to-one interventions. There was no system in place to make sure that interventions were evidence-based and delivered safely and effectively” 

These programmes were neither accredited nor evaluated. Once funding for offender behaviour programmes is transferred to independent organisations, there may be little to stop them continuing to improvise.

The scenario painted by the tender document sounds risky for all concerned. Companies will be bidding to run programmes which may or may not work, and probation clients will continue to be guinea pigs. If clients do not complete a programme, they may be breached, sent back to court and given a harsher sentence. Which seems doubly unfair given the lack of evidence for the programmes. 

Many prisoners and probation clients are anyway deeply sceptical about the ability of programmes to improve their lives. Evidence suggests courses can play a useful role but should never be seen as a panacea. We will only stop of tide of re-offending through creating supportive relationships, and by providing work, housing and mental health support. Meanwhile if I was a company bidding for one of the new TR contracts, I’d be asking some tough questions of HMPPS about the programmes they are promoting.

Share This Post

Share on twitter
Share on facebook
Share on linkedin
Share on email

Related posts

2 Responses

  1. Great post and important questions. It’s not just the probation clients who have offended who are ‘guinea pigs’, it is also their victims – partner/ex-partners and children – when it comes to the domestic abuse programmes. The ‘Partner Link’ element of the Building Better Relationships programme, which offers support to partner/ex-partners of men on the programme is critical but often overlooked, and faces similar issues with not having been evaluated. It is based on ‘version 1’ of a manual which has not been reviewed or updated since it was first introduced in 2015.

  2. This is a helpful summary which questions the role programmes play in effective rehabilitation. It echoes my own observations in recent years of accredited programmes becoming increasingly exclusive and process-driven; and a decrease in the quality of staff engagement skills, which led to my company Magistra designing a series of structured group work interventions focused on ‘early engagement’. Over the 35 years that I have worked in the CJS both as a Probation Officer, Prison Governor and Regional Senior Manager, I have witnessed interventions move away from meaningful engagement to ‘processing’ individuals. This process has been evidenced following recent high profile cases of IPP offenders reoffending despite completing numerous courses to ‘prove to the Parole Board’ that they are rehabilitated. The reality is that Rehabilitation takes place in the community however key to preparing for rehabilitation is building significant relationships to promote and support changes in attitudes and behaviour both in custody and the community; and effective risk management. Whilst Magistra’s programmes apply a recognised quality assurance and evaluation frameworks, they also apply restorative practice principles to provide a balance of challenge and support, and ‘engage‘ individuals successfully not only by ‘what’ is done but also ‘how’ it is done. In my view, the quality of engagement is the key to success not process. Improving staff engagement skills, not only with offenders but also other agencies, is key to reducing risk and supporting genuine rehabilitation.

Leave a Reply

Your email address will not be published.

All Probation Posts are sponsored by Unilink

With over 20 years’ experience in the criminal justice sector, Unilink is a world leader in probation and community corrections software applications, as well as prisoner self-service, prisoner/case management and prisoner communications. Unilink’s integrated suite of products provide a complete digital solution enabling efficient running of prisons and probation. Underpinned by biometrics it integrates seamlessly to deliver security, efficiency and value – while being proven to help rehabilitate prisoners.

Privacy Preference Center

keep informed

One email every day at noon