This is the third in a series of posts on the National Audit Office’s recent report: Outcome-based payment schemes: government’s use of payment by results. The report’s overall conclusion is that because of the lack of a clear evidence base, the PbR approach is currently laden with risk.
Does payment by results work?
That is the $64,000 question which the NAO seeks to answer in the second half of part two of its report by analysing the outcomes of public sector PbR schemes in terms of the six key benefits which are usually claimed by the advocates of a PbR approach:
- Cost-effectiveness
- Innovation
- Outcome focus
- Risk transfer
- User responsiveness
- Accountability
The NAO looks at examples of PbR delivering, and failing to deliver these benefits.
[divider]
Cost effectiveness
The NAO’s monitoring of the Work Programme found that it had delivered similar levels of performance to previous welfare-to-work programmes, but at £41 million lower cost (or 2% of the total cost of the scheme) and that the programme had improved performance further as the country’s economic and employment situation has improved.
However, the NAO says that the Troubled Families programme demonstrates the difficulties of demonstrating cost effectiveness.
[divider]
Innovation
The NAO found that the experts they consulted were most divided on the subject of whether PbR stimulates innovative approaches to tackle difficult social problems. Those involved in the Peterborough reoffending pilot claimed that PbR had facilitated new ways of, particularly joint, working. However, there appeared to be very little innovation in the Work Programme. While several commissioners expressed the hope that PbR would encourage innovation, many providers said that the risks associated with PbR meant that they were more likely to use tried and tested approaches. The NAO concludes that where commissioners want innovation, providers are likely to expect additional financial incentives.
[divider]
Outcome focus
As ever, not everything that is described as an outcome, turns out to be so. The PbR approach has been valuable in getting all the partners involved in the Troubled Families programme to focus on achieving long term change, while, for example, DFID’s malaria control programme in Tanzania pays for provision of bed nets rather than declining malaria rates.
[divider]
Risk transfer
From a commissioner perspective, an advantage to PbR can be the transfer of risk to the provider. For example, the Work Programme uses a prime and subcontractor model, with primes retaining overall accountability for the programme (although initially risk was passed down to smaller specialist providers). However, contractors do not bear all of the risks; departments may have to step in to manage contracts to drive improvements. The DWP terminated one of its contracts with Newcastle College Group based on the contractor’s under-performance and I wonder if the MoJ may need to do the same with some of the new probation providers.
[divider]
User responsiveness
The PbR model allowed specialist sub-contractors in the Work Programme to provide a more user responsive service to individuals with particular needs (related to their health for instance). However, the December 2014 evaluation of the WP showed low levels of referrals to specialist subcontractors, and some subcontractors leaving the supply chain.
[divider]
Accountability
I have a very different perspective from the NAO on PbR and accountability. For me, PbR is all about accountability on the very straightforward basis that if providers do not achieve their outcomes, they don’t get paid. The NAO, however, seems to see accountability as being linked to detailed, frequently monitoring meetings with commissioners. My view is that this micro-management kills PbR and any related innovation dead. It also tends to shift the focus back from outcomes to outputs/targets since commissioners want to know as early as possible whether an initiative is working, well before proper outcome data are available.
[divider]
Other considerations
The NAO also considers two other issues relating to the benefits of PbR.
It quite rightly notes that there are often significant additional costs and resources required in PbR contracts because of the technical challenges of setting the right payment mechanism.
The NAO also makes a plea for large government PbR schemes to be fully piloted first and laments that although the MoJ did pilot its through-the-gate reoffending initiative (in Doncaster and Peterborough prisons), it then moved to a national roll-out before the (rather disappointing) results were known.
For me, PbR is the perfect vehicle to experiment with new approaches to intractable social problems. Small scale pilots would enable a proper test with the emphasis on innovation and achieving outcomes.
To date, government has mainly used payment by results as a cost-cutting exercise with PbR focusing attention on the bottom line rather than on achieving more sustainable outcomes which benefit society and bring long term savings to the public purse.
