Verification
This is the ninth post in a blog series looking at the lessons I’ve learned from a recent review of the payment by results literature. In PbR schemes, payment is directly linked to whether specific outcome measures are achieved, so it’s not surprising that the measurement data comes under much closer scrutiny.
This post examines what the literature says about verifiying outcomes.
[button-blue url=”https://www.russellwebster.com/Lessons%20from%20the%20Payment%20by%20Results%20literature%20Russell%20Webster%202016.pdf” target=”_self” position=”left”]You can download the full literature review here[/button-blue]
[divider]
Different forms of verification
When commissioners devise a contract where payment is mainly contingent on providers meeting outcome measures, they need to be confident in the data relating to whether these measures are achieved.
There are two main issues.
- Is the provider working with the right people (i.e. not cherry picking those who will achieve the specified outcomes most easily)?
- Are the data reliable?
Independent verification is preferable and, sometimes, straightforward. For instance, the new Transforming Rehabilitation (probation) contracts specify reoffending targets for all offenders. Community Rehabilitation Companies, the new providers, have no influence over the offenders they work with (who are “selected” by the courts) nor the reoffending rate which is measured in a long established manner by a reliable third-party (the Police National Computer).
However, for some services, verifying outcomes is much more challenging. In the drug and alcohol recovery pilots, the contract design required new service users to be classified into one of five categories with different payment tariffs according to the severity of their substance misuse problem. It would obviously be unreasonable to make the same payment for a binge drinker who required a small number of brief interventions and a dependent heroin and crack cocaine user who might require a very intensive service for several months or even years.
In most of the pilots, commissioners did not trust providers to categorise new service users and introduced an independent third-party to make this assessment when the new service user made contact with the treatment service. A number of substance misuse experts criticised this approach for two main reasons:
- It introduced extra cost and bureaucracy into the treatment system.
- It placed an additional barrier in the way of those seeking help for their dependency on drugs and/or alcohol – making it no longer possible for them to access treatment directly (itself, a key indicator of a successful outcome).
Verification can be expensive
Commentators recommend that the process and costs of verification are borne in mind when outcome measures are decided. It is always better to adapt existing systems which are familiar to commissioners and providers, rather than introduce additional costly and duplicating recording processes, provided that:
- The data recorded reflect as closely as possible the outcomes desired of the service.
- The data entry system can be trusted.
Where providers are entering their own data, independent verification or auditing a sample of entries may be important, but should be carefully costed.
The research evidence also cautions against using too many outcome measures, highlighting the cost of collecting and verifying targets. Indeed, commentators in the international development arena (another area where PbR is becoming increasingly popular) noted that:
The most tangible cost of PbR relative to other forms of aid is the cost of verification.
Conclusion
The key lesson from the literature review is that when outcomes are the basis for payment it is important that all parties have total confidence in them.
Commissioners should beware constructing expensive and detailed systems for validating outcome measures which distract from the main purpose of the service.
Next week’s post turns our attention to the thorny issue of attributing outcomes — how much control providers have over the achievement of the contract’s outcomes. Providers shouldn’t be paid for outcomes they did not achieve, but nor should they be penalised when outcomes are affected by external factors.
I reviewed the literature as part of a project funded by the Oak Foundation to develop an interactive tool to assist commissioners and providers to decide whether a payment by results approach might be an effective approach to commissioning a particular service.
The tool is now live – please check it out at: www.PbR.russellwebster.com
