Keep up-to-date with drugs and crime

The latest research, policy, practice and opinion on our criminal justice and drug & alcohol treatment systems
Search
Paying for the wrong results?
The MoJ published their initial payment mechanism for the Transforming Rehabilitation contracts back on 3rd June and asked for feedback. They are currently developing a final version which should be published before the procurement process starts - scheduled for 23rd August. Last week, the Social Market Foundation published a paper by its director Ian Mulheirn which analysed the payment mechanism in forensic detail and came up with the devastating conclusion that : The payment mechanism encourages providers to cut spending on services and allow reoffending to drift marginally upwards.

This post is based on last week’s paper by Ian Mulheirn, outgoing director of the Social Market Foundation, which argued that the Ministry of Justice’s has got its proposed payment mechanism for the new reducing reoffending contracts wrong. You can follow Ian on Twitter: @ianmulheirn

Getting the contract right

The MoJ published their initial payment mechanism for the Transforming Rehabilitation contracts back on 3rd June and asked for feedback.

They are currently developing a final version which should be published before the procurement process starts – scheduled for 23rd August.

In the interim, various commentators have discussed that it appears to be a very severe contract with little incentive for providers to want to enter the probation market – essentially the MoJ want new providers to deliver services to more offenders (50,000 extra short term prisoners) for considerably less money within a contract which reduces in value every year while expecting higher performance.

I criticised the mechanism for the way in which it potentially penalised better performing providers who met the frequency but not the binary payment by results measures.

Last week, the Social Market Foundation published a paper by its director Ian Mulheirn which analysed the payment mechanism in forensic detail and came up with the devastating conclusion that :

The payment mechanism encourages providers to cut spending on services and allow reoffending to drift marginally upwards.

The paper is tightly argued but if, like me, you don’t find economic charts like this:

SMF1

this:

SMF2

or this:

SMF3the easiest to understand, I will try and summarise the main points, hopefully with reasonable accuracy.

The paper identifies four difficulties with the current payment mechanism.

1 Statistical risk and incentives

Quite rightly, the MoJ doesn’t want to pay providers for reducing reoffending rates if that would have happened anyway because of normal variations – reoffending rates typically fluctuate in response to police activity and other variables.

For this reason, providers won’t be rewarded if reoffending rates drop by less than 3% nor penalised if they rise by less than the same margin.

Because of the relatively small size of the cohorts of offenders in most Contract Package Areas, it is unlikely that the 3% improvement will be achieved – meaning that the PbR target is failing in its basic application – to incentivise high performance.

2 Size of the financial incentive

Providers will receive most of their funding on the new contracts not from PbR payments from reducing reoffending but via the Fee for Service for delivering community sentences.

Mr Mulheirn has calculated that providers will get less than half the amount of money per offender compared to that being spent by the Peterborough ONE project which has only succeeded in reducing reoffending by 2.9% – not enough to receive a PbR bonus.

It therefore doesn’t make financial sense for providers to invest more money in interventions than they are unlikely to get back in PbR payments.

3 The cohort approach

The MoJ is proposing that PbR payments are based on the reoffending rates of a new cohort of offenders every three months.

These cohorts are therefore likely to be relatively small – about 1,500 to 2,500 for most CPAs, which is one of the main reasons why the MoJ wants to set the +/- 3% margin to rule out statistical anomalies (indeed this could easily be +/-5% for the three-monthly cohorts).

The SMF paper recommends that the MoJ abandons the cohort approach and implements a cumulative payment model based on individual offenders.

It could then reduce the margin of statistical significance and would be more likely to reward effective providers – and penalise less effective ones.

4 Is the baseline stable?

Drawing from experience of the Work Programme, Mulheirn argues that the historical baseline for reoffending rates is unlikely to be reliable – the introduction of such a massive change, with an estimated reduction of funding of 20%+ will almost inevitably lead to a short-term increase in reoffending rates.

He argues that the MoJ should run the new system solely on a Fee for Service basis for the first two years to allow the new system to bed down and then set accurate reoffending baselines on which to calculate the payment by results system.

Conclusion

One of the reasons why this critique of the payment mechanism is so damning is that Ian Mulheirn (like myself) is a proponent of payment by results, believing it can stimulate innovation and effective approaches to entrenched social problems.

It is already clear from the Work Programme that a poor specification and contract with a focus predominantly on cutting costs is unlikely to turn out well.

Let’s hope that MoJ officials seriously consider the problems that Mulheirn identifies – and his recommendations to put them right.

 

 

Share This Post

Related posts

4 responses

  1. The MoJ recently told me that “We have not carried out statistical significance tests on the interim figures because, when it comes to the final results, neither pilot will be assessed on the basis of whether they have achieved a statistically significant change. Peterborough will be assessed by comparison with a national matched control group using a PSM methodology. Doncaster will be assessed against a baseline of calendar year 2009” (in response to my question: “You say “these interim figures show a fall in the frequency of re-conviction events at Peterborough” which is drop from 41.6% to 39.2%. At what threshold of probability is this statistically significant?”)

    So I am confused. Will PbR be on the basis of proper statistically significant differences or not?

    A 3% margin either way does NOT prove that anything other than a chance result is happening.

    More information on my FoI research here: http://ajustfuture.blogspot.co.uk/2013/07/tilting-at-lamposts.html

Leave a Reply

Your email address will not be published. Required fields are marked *

Probation posts sponsored by Unilink

 

Excellence through innovation

Unilink, Europe’s provider of Offender/Probation Management Software

Subscribe

Get every blog post by email for free