Keep up-to-date with drugs and crime

The latest research, policy, practice and opinion on our criminal justice and drug & alcohol treatment systems

Verifying outcomes for payment by results

Latest post in a series examining the lessons from the payment by results research literature focuses on the importance of verifying outcomes.

Share This Post


This is the ninth post in a blog series looking at the lessons I’ve learned from a recent review of the payment by results literature. In PbR schemes, payment is directly linked to whether specific outcome measures are achieved, so it’s not surprising that the measurement data comes under much closer scrutiny.

This post examines what the literature says about verifiying outcomes.

[button-blue url=”” target=”_self” position=”left”]You can download the full literature review here[/button-blue]


Different forms of verification

When commissioners devise a contract where payment is mainly contingent on providers meeting outcome measures, they need to be confident in the data relating to whether these measures are achieved.

There are two main issues.

  1. Is the provider working with the right people (i.e. not cherry picking those who will achieve the specified outcomes most easily)?
  2. Are the data reliable?

Independent verification is preferable and, sometimes, straightforward. For instance, the new Transforming Rehabilitation (probation) contracts specify reoffending targets for all offenders. Community Rehabilitation Companies, the new providers, have no influence over the offenders they work with (who are “selected” by the courts) nor the reoffending rate which is measured in a long established manner by a reliable third-party (the Police National Computer).

However, for some services, verifying outcomes is much more challenging. In the drug and alcohol recovery pilots, the contract design required new service users to be classified into one of five categories with different payment tariffs according to the severity of their substance misuse problem. It would obviously be unreasonable to make the same payment for a binge drinker who required a small number of brief interventions and a dependent heroin and crack cocaine user who might require a very intensive service for several months or even years.

In most of the pilots, commissioners did not trust providers to categorise new service users and introduced an independent third-party to make this assessment when the new service user made contact with the treatment service. A number of substance misuse experts criticised this approach for two main reasons:

  • It introduced extra cost and bureaucracy into the treatment system.
  • It placed an additional barrier in the way of those seeking help for their dependency on drugs and/or alcohol – making it no longer possible for them to access treatment directly (itself, a key indicator of a successful outcome).

Verification can be expensive

Commentators recommend that the process and costs of verification are borne in mind when outcome measures are decided. It is always better to adapt existing systems which are familiar to commissioners and providers, rather than introduce additional costly and duplicating recording processes, provided that:

  1. The data recorded reflect as closely as possible the outcomes desired of the service.
  2. The data entry system can be trusted.

Where providers are entering their own data, independent verification or auditing a sample of entries may be important, but should be carefully costed.

The research evidence also cautions against using too many outcome measures, highlighting the cost of collecting and verifying targets. Indeed, commentators in the international development arena (another area where PbR is becoming increasingly popular) noted that:

The most tangible cost of PbR relative to other forms of aid is the cost of verification.


The key lesson from the literature review is that when outcomes are the basis for payment it is important that all parties have total confidence in them.

Commissioners should beware constructing expensive and detailed systems for validating outcome measures which distract from the main purpose of the service.


Next week’s post turns our attention to the thorny issue of attributing outcomes — how much control providers have over the achievement of the contract’s outcomes. Providers shouldn’t be paid for outcomes they did not achieve, but nor should they be penalised when outcomes are affected by external factors.

I reviewed the literature as part of a project funded by the Oak Foundation to develop an interactive tool to assist commissioners and providers to decide whether a payment by results approach might be an effective approach to commissioning a particular service.

The tool is now live – please check it out at:

Share This Post

Related posts

PbR jargon demystified (4) S-Z

Last in a series of infographics which demystify the jargon and technical terms associated with the payment by results commissioning model.

Payment by Results
What we know about payment by results

This final post summarises the most important lessons to bear in mind if you are involved in commissioning, investing in or providing a payment by results-funded scheme.

The unexpected consequences of payment by results

The issue of providers “gaming” PbR contracts is a hot issue in the literature. Commentators take different views with some stating that it is only rational and efficient for providers to focus on the outcomes incentivised by PbR payments to the best of their ability while others describe similar behaviour as “gaming.”

Payment by Results
Getting payment by results contracts right

Designing PbR contracts can be a tricky business with plenty of examples in the literature of commissioners and providers wishing they had never signed on the dotted line. In addition to getting the outcomes and payment incentives right, two issues are particularly critical: the overall length of the contract and the he delay until, and interval between, incentive payments.

Payment by Results
Quality assurance in payment by results contracts

The main message from the literature is that commissioners need to be proactive in ensuring that PbR providers deliver a good quality service — particularly when service users include vulnerable groups

Payment by Results
Attributing outcomes in payment by results contracts

When outcomes are the basis for payment, it is important that the provider receiving the payments is responsible for achieving the outcomes. Targets should not be unduly influenced by external factors (such as the state of the economy for Work Programme type schemes) or by the work of other agencies who are not receiving payment from the contract.

Leave a Reply

Your email address will not be published. Required fields are marked *


Get every blog post by email for free