Not everything that counts can be counted…
The first challenge of payment by results schemes is to set the right outcomes.
The second is to set the right method of measuring those outcomes.
Sometimes measurement is not too difficult – if a PbR scheme is about prevention re-offending, then the Police National Computer can be used to record reconvictions.
Of course, the PNC doesn’t count every offence, only those when the offender is caught and convicted. Nevertheless, it is a simple, national, impartial system that can’t be gamed.
Measuring progress for those recovery from drug and/or alcohol problems is not so straightforward.
Recovery generally takes years rather than weeks or months and is typically characterised by uneven progress with relapse the norm.
Last week I posted about the outcome metrics that Social Finance proposed to measure recovery from addiction in their new report and noted their slightly different approach from the national drug recovery PbR pilots.
In this post I want to concentrate on how the Social Finance report suggests that those outcomes are measured.
…And not everything that can be counted, counts
Professor John Strang and Dr Samantha Gross of the National Addiction Centre wrote the Social Finance report and deliberated how best to measure substance use at some length.
They discuss the issues of consent and the demands of monitoring at regular intervals for two years post treatment with inevitable dropouts from both treatment and post-treatment support.
They also consider whether recovering addicts should be given incentive payments to comply with follow-up testing.
The authors explore different forms of testing and the technical appendix to the report looks at the pros and cons of different biological tests – urine, blood, saliva, hair and sweat.
They conclude that urine and breath tests would be best on the grounds that they are most reliable and least invasive form of testing and that these biological tests should serve as a way of auditing recovery addicts’ own self-reported use of substances on which they have been dependent.
In order to prevent gaming of drug test results, they recommend that testing is undertaken by an independent organisation and argue that it cases where no test is available then no outcome payment should be made.
Challenges
The Social Finance report is explicitly a first exploration of these issues and freely admits that it may need to change thinking as projects go from the drawing board to real-life.
Listed below are the key questions which I think will require further work to answer fully:
- Will recovered drug users want to remain in contact with drug testing services once they are clean?
- Will they be resistant to contact from an independent agency with which they have had little or no prior contact?
- Will they be notified of when they are going to be tested – and if so will this skew results (especially for alcohol and heroin)?
- What impact will incentive payments to comply with testing have?
- How will tariff payments reflect that some people may start treatment when temporarily abstinent (following detox or prison) and subsequently use at a higher level than baseline?
- Will those who drop out of testing be more likely to have relapsed or have recovered and moved on?
- Will recovered drug users be tested if they are in prison?
Both Social Finance and myself would appreciate if you would add any more questions via the comments section below.
You can follow Social Finance on Twitter: @socfinuk
