This is the second in a series of posts on the National Audit Office’s recent report: Outcome-based payment schemes: government’s use of payment by results. The report’s overall conclusion is that because of the lack of a clear evidence base, the PbR approach is currently laden with risk.
Whether to use PbR
In the second section of its PbR report, the NAO sets out the issues commissioners should consider when deciding whether or not to use payment by results. (It also examines the evidence for the benefits claimed for PbR, but that’s the subject of next week’s post).
The NAO emphasises that PbR is not the only commissioning game in town and recommends that commissioners assess the relative merits of a number of different approaches including in-house provision, fee-for-service outsourcing as well as PbR – to which I would add the option of setting straightforward financial incentives and penalties for achieving (or failing to achieve) specific goals.
The NAO recommends that when assessing the merits of different approaches, commissioners should acknowledge the technical challenges of using PbR and demonstrate that the proposed commissioning approach is value for money.
In its review of government PbR schemes, the NAO found that while the Department for International Development did provide a clear rationale for a PbR approach to supporting Rwandan education, the Department for Work and Pensions didn’t explain why PbR was the best approach to the Work Programme.
[divider]
When should commissioners use PbR?
The NAO very helpfully identifies ten features of services suited to a PbR commissioning approach. I was somewhat surprised to find that I don’t agree with all these criteria. The NAO features form a coherent and tight set of criteria focused on minimising the risk of wasting public money. However, from my perspective, they deny some of PbR’s potential.
The features and their descriptions are replicated in full from the NAO report, my views are in italics beneath each one.
1. Clear overall objectives, capable of being translated into a defined set of measurable outcomes
Well-defined, measurable outcomes make transparent the extent of the provider’s success, enabling commissioners to monitor the programme and calculate payments due.
All PbR schemes must have this.
2. Clearly identifiable cohort/population
Before the scheme starts commissioners need to specify which individuals they are targeting, so they can track the impact of the intervention.
True, although it would be an odd service which didn’t specify this.
3. Ability to clearly attribute outcomes to provider interventions
Commissioners need to be sure they are rewarding providers for their genuine contribution to desired outcomes. If external factors such as economic conditions are largely responsible for changes in outcomes, PbR may not be appropriate.
Reasonable in theory, much harder to achieve in practice. Certainly, performance in the Work Programme suffered because of the economic downturn. But PbR should be a catalyst for innovation, and a change in culture can sometimes achieve what a straightforward set of processes cannot.
4. Data available to set baseline
To show the impact of the scheme and set effective financial incentives, commissioners need to determine a clear baseline of performance before providers start work.
Absolutely.
5. An appropriate counterfactual can be constructed
To determine the effectiveness of the scheme, commissioners need a clear counter-factual to understand the additional impact of the scheme.
Again, I’m not convinced. If outcomes are clear and a provider achieves them, thereby making savings to the Treasury and Society – by reducing reoffending, getting the long term unemployed into proper jobs – do we need an elaborate and expensive account of what would otherwise happen?
6. Services are non-essential and underperformance or failure can be tolerated
Commissioners are likely to want closer control than PbR allows of essential services where failure might have dire consequences for public safety or the commissioner’s reputation.
This is the circumstance to which I take the most exception. The true value of PbR – almost wholly untested – is that commissioners should define their outcomes and measures and then step out of the way, stop strangling services with burdensome monitoring requirements, safe in the knowledge that if the provider fails, they need not pay. The problem with many of the PbR pilots has been that commissioners have staked their reputations on them and effectively curtailed the possibility of innovation by micro-management.
7. Providers exist who are prepared to take the contract at the price and risk
Commissioners will not be able to let the contract if providers do not bid.
Definitely, there may be several new private Community Rehabilitation Companies who may already be regretting bidding – the so called winner’s curse.
8. Providers are likely to respond to financial incentives
If providers are not motivated by financial incentives, commissioners should question the appropriateness of PbR as a mechanism for delivering the service.
I’m not convinced. I think the best PbR schemes should operate like a profit share – if providers meet outcomes which benefit the country, half the savings should go to the Treasury and half to providers who can choose to reinvest their half in further improvements in service delivery. The recent trend for most of the voluntary sector to operate identically to the private sector denies us all a different and valuable perspective on best practice.
9. Sufficient evidence exists about what works to enable providers to estimate costs of delivering services
If there is no clear evidence about the activities that are effective in achieving outcomes, providers may be unable to estimate the costs to them of seeking to achieve outcomes, and commissioners will find it harder to price the contract.
Again, PbR should be about stimulating innovation and new solutions. Why use PbR to commission a service that everyone knows how to do. If commissioners know the cost of paying incapacity benefit to one person for 12 months, then provided their contract pays a provider less for getting them into work for one year, where’s the risk?
10. Relatively short gap between provider intervention and evidence of outcome
PbR will be less attractive to providers if there is a long gap between the intervention (which requires upfront investment from the provider) and payment for a successful outcome. Providers may consequently prefer a higher fee for service and a lower PbR element if the gap is long.
True. One of the difficulties PbR has not resolved is how to set outcomes that are measurable over two -three years (the minimum length of time required for effective interventions to tackle intractable social problems such as reoffending and troubled families) while maintaining a reasonable cash flow.
Debate
So, what do you think?
Are the NAO’s features a copper-bottomed guarantee of an effective PbR scheme?
Or are you more in agreement with me that the attraction of PbR is the chance to move away from the straight-jacket of contemporary procurement practice and stimulate fresh approaches, under-written by the knowledge that if a provider fails, the commissioner doesn’t have to pay?
Please add your opinions via the comments section below.
