This post is based on last week’s paper by Ian Mulheirn, outgoing director of the Social Market Foundation, which argued that the Ministry of Justice’s has got its proposed payment mechanism for the new reducing reoffending contracts wrong. You can follow Ian on Twitter: @ianmulheirn
Getting the contract right
The MoJ published their initial payment mechanism for the Transforming Rehabilitation contracts back on 3rd June and asked for feedback.
They are currently developing a final version which should be published before the procurement process starts – scheduled for 23rd August.
In the interim, various commentators have discussed that it appears to be a very severe contract with little incentive for providers to want to enter the probation market – essentially the MoJ want new providers to deliver services to more offenders (50,000 extra short term prisoners) for considerably less money within a contract which reduces in value every year while expecting higher performance.
I criticised the mechanism for the way in which it potentially penalised better performing providers who met the frequency but not the binary payment by results measures.
Last week, the Social Market Foundation published a paper by its director Ian Mulheirn which analysed the payment mechanism in forensic detail and came up with the devastating conclusion that :
The payment mechanism encourages providers to cut spending on services and allow reoffending to drift marginally upwards.
The paper is tightly argued but if, like me, you don’t find economic charts like this:
this:
or this:
the easiest to understand, I will try and summarise the main points, hopefully with reasonable accuracy.
The paper identifies four difficulties with the current payment mechanism.
1 Statistical risk and incentives
Quite rightly, the MoJ doesn’t want to pay providers for reducing reoffending rates if that would have happened anyway because of normal variations – reoffending rates typically fluctuate in response to police activity and other variables.
For this reason, providers won’t be rewarded if reoffending rates drop by less than 3% nor penalised if they rise by less than the same margin.
Because of the relatively small size of the cohorts of offenders in most Contract Package Areas, it is unlikely that the 3% improvement will be achieved – meaning that the PbR target is failing in its basic application – to incentivise high performance.
2 Size of the financial incentive
Providers will receive most of their funding on the new contracts not from PbR payments from reducing reoffending but via the Fee for Service for delivering community sentences.
Mr Mulheirn has calculated that providers will get less than half the amount of money per offender compared to that being spent by the Peterborough ONE project which has only succeeded in reducing reoffending by 2.9% – not enough to receive a PbR bonus.
It therefore doesn’t make financial sense for providers to invest more money in interventions than they are unlikely to get back in PbR payments.
3 The cohort approach
The MoJ is proposing that PbR payments are based on the reoffending rates of a new cohort of offenders every three months.
These cohorts are therefore likely to be relatively small – about 1,500 to 2,500 for most CPAs, which is one of the main reasons why the MoJ wants to set the +/- 3% margin to rule out statistical anomalies (indeed this could easily be +/-5% for the three-monthly cohorts).
The SMF paper recommends that the MoJ abandons the cohort approach and implements a cumulative payment model based on individual offenders.
It could then reduce the margin of statistical significance and would be more likely to reward effective providers – and penalise less effective ones.
4 Is the baseline stable?
Drawing from experience of the Work Programme, Mulheirn argues that the historical baseline for reoffending rates is unlikely to be reliable – the introduction of such a massive change, with an estimated reduction of funding of 20%+ will almost inevitably lead to a short-term increase in reoffending rates.
He argues that the MoJ should run the new system solely on a Fee for Service basis for the first two years to allow the new system to bed down and then set accurate reoffending baselines on which to calculate the payment by results system.
Conclusion
One of the reasons why this critique of the payment mechanism is so damning is that Ian Mulheirn (like myself) is a proponent of payment by results, believing it can stimulate innovation and effective approaches to entrenched social problems.
It is already clear from the Work Programme that a poor specification and contract with a focus predominantly on cutting costs is unlikely to turn out well.
Let’s hope that MoJ officials seriously consider the problems that Mulheirn identifies – and his recommendations to put them right.
4 responses
The MoJ recently told me that “We have not carried out statistical significance tests on the interim figures because, when it comes to the final results, neither pilot will be assessed on the basis of whether they have achieved a statistically significant change. Peterborough will be assessed by comparison with a national matched control group using a PSM methodology. Doncaster will be assessed against a baseline of calendar year 2009” (in response to my question: “You say “these interim figures show a fall in the frequency of re-conviction events at Peterborough” which is drop from 41.6% to 39.2%. At what threshold of probability is this statistically significant?”)
So I am confused. Will PbR be on the basis of proper statistically significant differences or not?
A 3% margin either way does NOT prove that anything other than a chance result is happening.
More information on my FoI research here: http://ajustfuture.blogspot.co.uk/2013/07/tilting-at-lamposts.html
Thanks Jon. We will see what Straw Man II brings…