front cover1
Russell Webster

Russell Webster

Criminal Justice & substance misuse expert and author of this blog.

Payment by Results: Lessons from the Literature

Share on twitter
Share on facebook
Share on linkedin
Share on print
Share on email
The association of many PbR schemes with very robust cost-cutting and/or the privatisation of previously public markets has caused considerable controversy and confusion which has enabled researchers working from the same data to reach opposing conclusions about the same initiative.

The literature review

I’m currently developing an interactive tool to assist commissioners and providers to decide whether a payment by results approach might be an effective approach to commissioning a particular service. (For more details about this project and how to get involved, click here.)

As part of that project, I’ve just completed a review of the PbR literature which is free to download. A total of 93 studies were analysed for this review. A large majority (75) focused primarily on payment by results in the UK, seven focused on PbR in Australia, six on the USA and five on PbR internationally.

Almost one quarter of these studies (22) dealt with payment by results generally with the rest focusing on PbR in a particular social care sector. Three studies focused on housing/homelessness; two each on international development and children and one on defence. The remaining 62 studies focused on five main sectors as shown below:

PbR sectors

In the knowledge that not everyone has the time (or inclination) to read a 16,000 word literature review, this post provides a brief introduction to the review. I have developed the main themes of the review into a series of blog posts which will run every Wednesday for the next few weeks.

[divider]

A controversial approach

Payment by results is an increasingly common commissioning model; found across a number of disparate sectors − from defence to international development − and growing in popularity all over the world with initiatives particularly common in Australia, the United States and the UK.

The PbR “movement” can be traced back at least as far as the 1990s and even earlier; the first big wave was probably the trend for payment for performance in US healthcare. PbR has also been integral to the design of government initiatives to get unemployed people back to work in Australia and the UK since the 1990s.

However, it has become much more widespread, in the UK in particular, since 2010 when it was championed by the government in general and the Treasury in particular. This rapid growth of PbR schemes has taken place at the same time as substantial cuts in public services.

The association of many PbR schemes with very robust cost-cutting and/or the privatisation of previously public markets has caused considerable controversy and confusion which has enabled researchers working from the same data to reach opposing conclusions about the same initiative. By way of illustration, those who saw the Work Programme as being primarily about helping long term unemployed people back to work with the least investment of public finances will assess it as very successful — the Work Programme performed at the same level as the programmes preceding it, but was £41 million (2%) cheaper. Conversely, those who thought it was designed to get people with entrenched difficulties such as disability or addiction into work, will concluded that it has failed.

There has also been widespread criticism of the growth in PbR despite the lack of an evidence base to guide whether and in what circumstances it can be effective.

payment by results literature review

Does PbR work?

One of the main findings of the review is that PbR schemes are so varied, tend to be commissioned for such different reasons (to improve outcomes and/or stimulate innovation; to reduce costs, to transfer risk from government or commissioners, to encourage new markets), and are so often poorly evaluated, that it is not yet possible to pass judgement on whether the PbR model works.

There are a number of examples of positive PbR schemes in this and other countries. There are, probably, even more examples of badly designed PbR schemes which have failed.

However, there are enough good quality reviews and evaluations to identify key learning points which could prove invaluable for commissioners and providers as they design and operate PbR contracts. A number of researchers (including Fraser Battye) have advocated that more would be learnt from an approach which seeks:

to advance the debate on PbR from ‘for or against?’ to ‘when, how, for whom and under what circumstances?

One of the key themes to emerge from the literature review is the limited amount of discussion and negotiation which takes place between commissioners and providers at all stages of the process: from considering whether PbR might be an effective approach, though specifying outcomes and incentives, to designing the contract itself.

The principal objective of the literature review is to encourage and facilitate this dialogue by helping stakeholders to:

  • Clarify the purpose of an individual PbR scheme;
  • Identify critical success factors;
  • Identify common issues which cause problems and difficulties; and
  • Be aware of ways to address or at least mitigate these problems.

[divider]

Key themes

Over the next few Wednesdays, I will examine the main key themes which emerged from the literature review:

  • The different purposes of PbR schemes
    • Stimulating innovation
    • Transferring risk
    • Encouraging new market entrants
    • Reducing commissioning costs
    • More/better focus on outcomes
    • Reducing inequalities
  • Understanding the market
  • Defining outcomes
    • Clarity and complexity
    • Deadweight
    • Verification
    • Attribution
    • Individual vs cohort measurement
    • Segmentation
  • Setting price and incentives
    • Gaming
    • Responding to gaming
    • The dangers of competitive pricing
    • Incentives and reinvestment
  • Contract structuring
  • Monitoring and evaluation
  • Implications

I hope you find the posts useful whether you are a commissioner or provider, worker or service user, researcher or policy wonk.

 

You can download the full literature review here.

You can find a large section of this website dedicated to Payment by Results with a wide range of resources, all of which are free to access.

Update (7 July 2016) The PbR interactive tool is now live – please check it out at: www.PbR.russellwebster.com

 

Share on twitter
Share on facebook
Share on linkedin
Share on print
Share on email

2 thoughts on “Payment by Results: Lessons from the Literature”

  1. Russell, an impressive – and very public-spirited – piece of work. I have one suggestion best made privately, if you email me at [email protected] I’ll do it that way.

    I already had some knowledge of the subject but the full review is very much needed. I have speed read it and will re-read at length. The key thing is to bring evidence and analysis to bear in an objective and disinterested way, which is why I read your blog. There arent many who bother, most opine from previously assumed positions, and sometimes self interest!

    First thoughts.

    I judge PBR by results, which seems poetic justice. And if after 50 projects spending £15bn public money we still dont really know if it works, then we ought to be prepared at some point in the future to put our effort elsewhere, if no sign of impact is forthcoming. After all, there are many other ways of seeking to improve services, including the rather dull but proven one of better management. There’s nothing that says PBR ‘must’ work, after all.

    A second thought: paying for ‘results’ of course ‘works’. It’s called contracting. If I pay someone to put in a gas boiler and it doesnt work I dont pay him until he makes it work, and that gets a better result than paying him just for the hours and hardware installed even if it never works. PFI showed that – prisons built twice as fast and half the cost if payment dependent on meeting the deadline and spec and at fixed cost, compared to the old way of taking as long as it took and payng whatever it cost.

    So what’s new about PBR? I suppose its fixing on a ‘social result’. To take reducing reoffending, we used to pay prison operators for running defined programmes which we knew (from research) effectively addressed what we knew (from research) were criminogenic characteristics of prisoners eg literacy. That should logically have got a result, and did so, Grayling’s dismissive comments nothwithstanding. What I am not clear about is then why PBR linked to reconviction rates ought to get a better result than that earlier approach. It might if the way to get results wasnt known and experimentation was needed but what if we already know (broadly) what needs to be done? And is knowledge gathering about imponderables such as paths to desistance best done though PBR rather than research – especially if IPR means that knowledge, if forthcoming, wont be shared? AT this stage I merely ask such question.

    Anyway many thanks for a very thought provoking piece and for presenting knowledge – even inconclusive knowledge! – in such an accessible way.

  2. Hi Julian
    Thanks very much for your comment. I agree that the innovation vs established evidence-base debate is one of the most interesting issues within the PbR debate. Of course, I know people who are very critical of some of the accredited programmes and views on what works best tend to move on relatively quickly (and sometimes move round or back, rather than on).

    I think the main argument for PbR recently is that it “focuses the mind”. If there was sufficient money in the pot and accommodation for offenders could more easily be found/developed, then we might see PbR have an impact. But in 2016 particuarly, they are very big “ifs”…

    Best Wishes

    Russell

Leave a Comment

Your email address will not be published.