Police pay
Russell Webster

Russell Webster

Criminal Justice & substance misuse expert and author of this blog.

Arrested development: Why individual PbR will not work

Share on twitter
Share on facebook
Share on linkedin
Share on print
Share on email

Saturday’s newspapers sent shockwaves through all ranks of police officers who are waiting for Tom Winsor’s review of pay and conditions to be published.

The Telegraph announced:

“Police could be given performance-related pay for first time”

While the Star went with:

“Cops collar cash”

It’s not sure how much credence we should give to these reports, but the Telegraph and Star do make substantially similar claims:

“cash incentives for high-performing police officers who can successfully fight crime”

“police are set to pocket bonuses for the number of arrests they make”.

We should know for certain soon since the Winsor report is scheduled for publication tomorrow morning.

I’m feeling slightly guilty about the whole situation since I wrote a post a couple of weeks ago in which I idly speculated that it would be theoretically possible to pay probation officers on an individual payment by results basis – never expecting that anyone would seriously propose a similar approach for probation, police or anyone else.

Until last Saturday.

Fortunately, we have good evidence that individual PbR schemes just don’t work.

Earlier this week I wrote about a Freakonomics case study in which a Washington Emergency Room Doctor, Craig Feied, turned around a failing ER by installing a super-efficient computerised information system.

The system generated so much data it was used for medical research and an assessment of how good individual ER doctors were.

What the case study made very clear was that attempting to judge doctors on a payment by results basis just didn’t work. PbR is all about outcomes – improved patient health for doctors and, at least in part, increased detection and arrest rates for police officers.

When researchers tried to evaluate the effectiveness of doctors by their patient outcomes, it quickly became clear that this was a pointless exercise.

An assessment of ER doctors’  performance on a patient outcome basis was rejected for a wide range of reasons, all of which would be relevant to paying police on their clear-up rates:

Selection bias – patients aren’t randomly assigned patients.

The profile of people attending ERs varies markedly throughout different times of the day and the days of the week. In the same way, we would expect officers on duty in town centres on Friday and Saturday nights to make more arrests than those on the same duty on a Tuesday afternoon.

Sometimes the better doctors have higher patient death rates.

The sicker you are, the more likely you are to seek out the best cardiologist. In the same way, a more experienced and skilled officer may defuse a confrontation, rather than nicking everyone in sight.

Once individuals know that they are being measured and paid on performance, they start adjusting the way their work to fit.

This is perhaps the most worrying aspect of performance-related pay. A doctor who knows he is being paid on patient outcomes may start “creaming” – selecting low risk patients and rejecting those with more serious complaints who are most in need of treatment but who are most likely to reduce his/her outcome rates.

What would be the equivalent for police officers? More arrests for possession of cannabis? Less stop and search? Forced deployment to jobs where performance related pay bonuses are not likely?

Payment by results is a great opportunity to focus public services on outcomes that make a difference. A chance to break free from a culture where work priorities are driven by targets, Key Performance Indicators etc., rather than the needs 0f the public.

A key component of successful PbR schemes is that they focus different teams, departments and organisations on how they can most effectively collaborate for the greater good.

Individual performance-related pay completely undermines this approach.

As any study of Bankers’ bonuses will show.

Share on twitter
Share on facebook
Share on linkedin
Share on print
Share on email

3 Responses

  1. Quite agree with you. Recall failed performance related pay for NHS managers in late 80’s. Favored certain groups with more predictable work and working hours over others. Was very divisive. This scenario much more worrying.

  2. Just about every aspect of the Winsor Review is flawed and displays muddled, confused and incorrect thinking. Performance pay in the police would undoubtedly lead to widespread abuse in the power of arrest and undermine the basic duty of a police constable to apply the law without fear or favour.

  3. Good article. I would like to add some nerdy comments if I may.

    Individual payment by results has its place. In selling double-glazing, for example. It’s a pretty clean binary measure. Sell loads, get paid loads – your contribution to the business is tangible and duly rewarded. But sell nothing? No bonus. Unless you work in the banking sector, where pay still appears confusingly performance-unrelated.

    But in health, crime, prisons, probation? That might require a bit more thought.

    In Probation, if you base a PbR scheme on reoffending outcomes, as is suggested, you must first ensure a big-enough cohort size (I will return to this).

    Reoffending rates in AnyTrust are 32%. You are offered payment if you reduce them by five percentage points – to 27%. AnyTrust might think, OK, it’s possible, with freedom to innovate, but would understandably fear interference from external factors such as a steep improvement in police clear-up rates or a changing caseload including more high-risk-of-reoffending offenders.

    In this scenario, if AnyTrust remained at 32% reoffending rate, but with a much more demanding caseload, this should be recognised – and it is, by using an adjusted baseline.

    There are different models. The one used to contextualise local adult reoffending rates – the predicted probability of reoffending score – takes account of a whole host of variables including age, gender, index offence type, age at first conviction, number of previous convictions etc.

    You can mitigate to some degree the influences outside your control by using an adjusted baseline against which to judge success. In the example above, your adjusted baseline might be 37% (taking into account these circumstances). Your achievement in holding rates at 32% would be acknowledged, and paid for.

    In reality you might want to go further and split your cohort, setting separate, achievable outcome reoffending targets on what you know would constitute success in each area.

    As an example, young men (18-24) with an index offence of theft or burglary have much higher than average reoffending rates – in AnyTrust they can be up to 55%, more like short-sentence prisoner release rates than the overall Trust average of 32%. It would be insane to expect AnyTrust to reduce this specific cohort’s reoffending rates to anywhere near the Trust average. There is a clear and well-documented danger here of “parking”.

    But if you use the adjusted baseline, you realise this group has a “predicted reoffending rate” of 58% – an expectation of their reoffending rate based on some of the variables mentioned above.

    You might then set a realistic and achievable target of 50% for this group. You might develop a payment structure that allowed for different “offender types” and therefore encouraged appropriate interventions across the board.

    This is unlikely to be popular with ministers, who seem to like talking about “reoffending rates being unacceptably/stubbornly high” at every available opportunity. They might baulk at having to justify paying out on reducing reoffending to “only” 50% – though this would (and should) be considered a relative success.

    To return to cohort size – as long as the group is large enough, this adjusted baseline, or predicted rate, gives a pretty useful idea of the reoffending rates you might expect to see from particular subsets of offenders. It’s never miles away from reality.

    But it’s important to keep it as a rough indicator of the behaviour of a big group – to some extent it seems humans behave quite predictably in group situations.

    The same cannot be said for individuals. Paying probation officers on an individual PbR basis, using reoffending outcomes as the measure of success, is fraught with danger.

    Every individual on AnyTrust’s caseload has a “predicted probability of reoffending (PPoR) score”. By calculating the mean average of this score across big cohorts, you arrive at the adjusted baseline figure.

    But in looking at individual scores, this “minority report” can be emphatically wrong. To cherry-pick two examples, a 27-year-old man on AnyTrust’s caseload with a PPoR score of 95% had no proven reconviction over the following 12 months, while a 36-year-old man with a 4% PPoR did. In other words, the reoffending predictor felt (based on all the evidence) that this 36-year-old was a remote 25/1 shot to reoffend. But 25/1 shots do come in every now and again.

    PbR can work with reoffending as a measure of success, as long as the design is clever and realistic. But beware small numbers, take steps to avoid creaming and whatever you do, don’t try and predict what individual human beings are going to do next (even double-glazing saleswomen!)

Leave a Reply

Your email address will not be published.

Select Language

Keep up-to-date on drugs and crime

You will get one email with a new article every day.