The Justice Data Lab
Last week the Ministry of Justice announced the opening of its Justice Data Lab
The data lab is a service open to any organisation working with offenders which wants to evidence how effective their work is at reducing reoffending.
To use the service, organisations simply supply the data lab with details of the offenders who they have worked with and information about the services they have provided.
The justice data lab will then supply aggregate one-year proven re-offending rates for that group, and, most importantly, that of a matched control group of similar offenders.
The MoJ promises to provide the results in a “clear and easy to understand format, with explanations of the key metrics, and any caveats and limitations necessary for interpretation of the results.”
Those of us who have repeatedly called for an evidence-based approach to tackling crime and re-offending can’t but be delighted at this no cost service which is likely to be of particular benefit to voluntary sector organisations – including very small ones, the minimum cohort size is just 60.
Anyone who has engaged in the convoluted process of getting reconviction data from the Police National Computer will be delighted at what is promised to be a much simpler process.
The fineprint
One note of caution, the Justice Data Lab is being launched as a one year pilot initially which has led to some more cynical/suspicious/realistic/experienced (delete as appropriate) commentators suggesting that it is more to do with allaying voluntary sector concerns about the new PbR reoffending contracts than a permanent move towards transparency and best practice.
Organisations’ experience of using the Data Lab may be a useful indicator of the spirit in which it is being launched.
There are a number of criteria that organisations need to be aware of :
Time constraints
- The organisation must have worked with the offender between 2002 and 2010.
- Additionally, the individuals must have received a community sentence between 2002 and 2010 or been released from custody during the same time period.
Re-offending data for people sentenced/released in 2011 will be available some time within the first pilot year of operation.
Information provided
In addition to first and last names, date of birth and gender, organisations are encouraged to provide Prison Numbers and Police National Computer Identifiers if possible.
The date that offenders started and completed any intervention provided by the organisation are critical so that the MoJ can look at data relating to the right sentence.
The MoJ is particularly interested in “detailed and precise information” about the work that the organisation did with offenders.
In particular, it wants to know how offenders were selected for the service/intervention provided – to make sure that organisations aren’t cherry picking the easiest to work with offenders in order to inflate their success rates.
Exclusions
There is a long list of groups of offenders for whom the MoJ may not be able to conduct research including:
- vulnerable youths or adults, including individuals with mental or learning difficulties.
- offenders who have mental health issues, including those being treated through the Personality Disorder Programme
- services or interventions that are targeted at persons with specific needs, including those relating to drugs or alcohol.
- requests based on services or interventions targeted at persons who have committed sexual offences, or domestic violence offences.
At first glance, these exclusions include a very high proportion of offenders.
The MoJ has said that it will approve requests for reoffending data on a case-by-case basis – so we will have to wait and see how flexible they are.
Conclusion
Overall, the move towards an evidence-based approach to determining the effectiveness of reducing reoffending interventions must be the right thing.
Many of us – politicians, commissioners and providers – still place too much stock in the power of an individual story of redemption rather than an objective analysis of outcomes.
One of my favourite movie characters is Barton Keys, the insurance claim investigator played by Edward G. Robinson in Double Indemnity.
Keys relies on a combination of experience and data in assessing the validity of insurance claims.
His experience makes itself known on a visceral level as a “little man in his stomach” which alerts him to a probably fraudulent claim, but he is also a great man for relying on the evidence:
I’m sure he’d have sought data from the Justice Data Lab.
If you’ve applied to the Justice Data Lab, please share your experiences below.
3 Responses
Hi – taking advantage of the data lab will not be straight forward. I give some tips on how to do so here: http://socialimpactanalytics.com/2013/04/19/5-essential-tips-to-take-full-advantage-of-the-justice-data-lab/
Hi, Russell,
As a data analyst at Staffordshire & West Midlands Probation Trust, I produce reoffending reports internally and for partners who specialise in interventions for women, drugs, housing, for example. The reports consistently demonstrate the positive impact these organisations are having on reducing reoffending. This is the kind of thing that ought to give weight to funding proposals – if prospective benefactors see proof that it works.
On that basis alone, I strongly agree that any agency interested in the reoffending rates of its cohorts should have access to this information. This is what the Justice Data Lab is intended to do. I applaud it loudly as you do.
But there are a couple of things worth bearing in mind for any organisations thinking of making a submission.
The first is the time lag. If you’ve kept pretty accurate records of the names, dates of birth and periods of engagement of your clients for the last three years, then it’s worth looking into now. The richer the information, the better, of course, but as a pre-requisite, in theory that’s all you really need.
But if you haven’t even got that basic level of historic detail, you’d need to start collecting it now. Due to the time lag (we have to wait to see if any reoffending occurs, then allow six months for court convictions etc etc) it’s at least a couple of years before you get to understand your effectiveness in this respect. And remember, the Justice Data Lab is a one-year pilot.
The other thing – without wanting to get too technical – is the way the Justice Data Lab will contextualise your reoffending rates. They will use something called propensity score matching. This method compares your group with a control group made up to resemble your cohort in character as much as possible.
So if your organisation provides education and training to young black men who have just served a short prison sentence, the Justice Data Lab will create a large control group of men with an identical profile, but who didn’t receive your intervention, and measure your success (or not) against this benchmark.
That’s OK as far as it goes, but the MoJ don’t have data on everything. They don’t collect information on dynamic homelessness status or drug relapses or family breakdowns.
So if your organisation provides help and support for class A drug users in the criminal justice system, your control group will be full of people who aren’t class A drug users. There will also be some that are, but there will be no way of knowing for sure. And as that is the key factor in your work – in my eyes (as a data analyst) the control group is weak and any analysis not robust.
I suspect this is why the MoJ have asked providers of drug and alcohol services, along with those specialising in mental health, and others, to check first before making any submissions – though not ruling it out altogether.
In recent weeks, I have offered through the Justice Data Lab team and via social media – to any VCS or other organisations interested – answer any questions that people may have as far as I can or help with submissions. Go for it – it’s only a one-year pilot and it may not continue if it isn’t used, and it’s a free service too!
Thanks very much Jason for your offer of help. I hope lots of VCS agencies take it up.
The MoJ are prepared to use OASys data (Offender Assessment System) to match offenders with, for example, drug or alcohol problems.
It helps if agencies record this information themselves.