Will the MoJ data lab do us justice?

The new MoJ data lab promises small voluntary organisations access to information about their effectiveness - for free. It could be invaluable for organisations looking to win reducing re-offending contracts under the new payment by results framework. But how will it work in practice?

Share This Post

Share on twitter
Share on facebook
Share on linkedin
Share on email

The Justice Data Lab

Last week the Ministry of Justice announced the opening of its Justice Data Lab

The data lab is a service open to any organisation working with offenders which wants to evidence how effective their work is at reducing reoffending.

To use the service, organisations simply supply the data lab with details of the offenders who they have worked with and information about the services they have provided.

The justice data lab will then supply aggregate one-year proven re-offending rates for that group, and, most importantly,  that of a matched control group of similar offenders.

The MoJ promises to provide the results in a “clear and easy to understand format, with explanations of the key metrics, and any caveats and limitations necessary for interpretation of the results.”

Those of us who have repeatedly called for an evidence-based approach to tackling crime and re-offending can’t but be delighted at this no cost service which is likely to be of particular benefit to voluntary sector organisations – including very small ones, the minimum cohort size is just 60.

Anyone who has engaged in the convoluted  process of getting reconviction data from the Police National Computer will be delighted at what is promised to be a much simpler process.

 

data lab

 

The fineprint

One note of caution, the Justice Data Lab is being launched as a one year pilot initially which has led to some more cynical/suspicious/realistic/experienced (delete as appropriate) commentators suggesting that it is more to do with allaying voluntary sector concerns about the new PbR reoffending contracts than a permanent move towards transparency and best practice.

Organisations’ experience of using the Data Lab may be a useful indicator of the spirit in which it is being launched.

There are a number of  criteria that organisations need to be aware of :

Time constraints

  • The organisation must have worked with the offender between 2002 and 2010.
  • Additionally, the individuals must have received a community sentence between 2002 and 2010 or been released from custody during the same time period.

Re-offending data for people sentenced/released in 2011 will be available some time within the first pilot year of operation.

Information provided

In addition to first and last names, date of birth and gender, organisations are encouraged to provide Prison Numbers and Police National Computer Identifiers if possible.

The date that offenders started and completed any intervention provided by the organisation are critical so that the MoJ can look at data relating to the right sentence.

The MoJ is particularly interested in “detailed and precise information” about the work that the organisation did with offenders.

In particular, it wants to know how offenders were selected for the service/intervention provided – to make sure that organisations aren’t cherry picking the easiest to work with offenders in order to inflate their success rates.

Exclusions

There is a long list of groups of offenders for whom the MoJ may not be able to conduct research including:

  • vulnerable youths or adults, including individuals with mental or learning difficulties.
  • offenders who have mental health issues, including those being treated through the Personality Disorder Programme
  • services or interventions that are targeted at persons with specific needs, including those relating to drugs or alcohol.
  • requests based on services or interventions targeted at persons who have committed sexual offences, or domestic violence offences.

At first glance, these exclusions include a very high proportion of offenders.

The MoJ has said that it will approve requests for reoffending data on a case-by-case basis – so we will have to wait and see how flexible they are.

Conclusion

Overall, the move towards an evidence-based approach to determining the effectiveness of reducing reoffending interventions must be the right thing.

Many of us – politicians, commissioners and providers – still place too much stock in the power of an individual story of redemption rather than an objective analysis of outcomes.

One of my favourite movie characters is Barton Keys, the insurance claim investigator played by Edward G. Robinson in Double Indemnity.

Keys relies on a combination of experience  and data in assessing the validity of insurance claims.

His experience makes itself known on a visceral level as a “little man in his stomach” which alerts him to a probably fraudulent claim, but he is also a great man for relying on the evidence:

I’m sure he’d have sought data from the Justice Data Lab.

 

If you’ve applied to the Justice Data Lab, please share your experiences below.

Share This Post

Share on twitter
Share on facebook
Share on linkedin
Share on email

Related posts

Payment by Results
Prisons and prevention

New IPPR report advocates devolving responsibility for low level offenders to local authorities and City mayors. But do we need another probation service?

On Probation
Modelling cohorts for Transforming Rehabilitation

Last week the MoJ published modelled data for the 6 years from 2005 to 2010, showing the number of offenders in each PbR cohort and the 1-year re-offending rates among those offenders. The report provides an historical picture of probation performance in reducing reoffending aimed at those organisations interested in winning the new probation contracts. It presents performance on a Contract Package Area, rather than Probation Trust, basis and it looks specifically at the group of offenders for whom the new Community Rehabilitation Companies will be responsible.

On Probation
How is the justice data lab doing?

The purpose of the Justice Data Lab was to make it possible for small voluntary organisations to find out if their work with offenders made a difference to reoffending rates. It was launched as part of the Transforming Rehabilitation project as a way of government, commissioners and Prime providers having a way of comparing the impact of different providers delivering a range of interventions. Despite the strong publicity surrounding the launch of the Data Lab,

On Probation
Latest probation reducing reoffending rates

The latest MoJ reoffending rates show that probation trusts continue to reduce reoffending even under the pressure of the proposed wholescale changes under the government’s Transforming Rehabilitation agenda. This overall good progress does, however, conceal a considerable variation between trusts.

Payment by Results
The 9th Commandment of Payment by Results: Thou shalt join up commissioning

If successful recovery from addiction can only be achieved by a coordinated approach across the health, drug treatment, criminal justice, housing, social care and ETE (employment, training and education) sectors, which government departments should pay for which outcomes? Ideally the Ministry of Justice, Department of Health, DWP and Supporting People should all contribute to a pooled budget. But of course that’s not the way that departmental budgets work – indeed, there’s evidence that, despite the Community Budgets initiative – departmentalitis has actually got worse over recent years in the face of largescale and repeated cuts in expenditure.

3 Responses

  1. Hi, Russell,

    As a data analyst at Staffordshire & West Midlands Probation Trust, I produce reoffending reports internally and for partners who specialise in interventions for women, drugs, housing, for example. The reports consistently demonstrate the positive impact these organisations are having on reducing reoffending. This is the kind of thing that ought to give weight to funding proposals – if prospective benefactors see proof that it works.

    On that basis alone, I strongly agree that any agency interested in the reoffending rates of its cohorts should have access to this information. This is what the Justice Data Lab is intended to do. I applaud it loudly as you do.

    But there are a couple of things worth bearing in mind for any organisations thinking of making a submission.

    The first is the time lag. If you’ve kept pretty accurate records of the names, dates of birth and periods of engagement of your clients for the last three years, then it’s worth looking into now. The richer the information, the better, of course, but as a pre-requisite, in theory that’s all you really need.

    But if you haven’t even got that basic level of historic detail, you’d need to start collecting it now. Due to the time lag (we have to wait to see if any reoffending occurs, then allow six months for court convictions etc etc) it’s at least a couple of years before you get to understand your effectiveness in this respect. And remember, the Justice Data Lab is a one-year pilot.

    The other thing – without wanting to get too technical – is the way the Justice Data Lab will contextualise your reoffending rates. They will use something called propensity score matching. This method compares your group with a control group made up to resemble your cohort in character as much as possible.

    So if your organisation provides education and training to young black men who have just served a short prison sentence, the Justice Data Lab will create a large control group of men with an identical profile, but who didn’t receive your intervention, and measure your success (or not) against this benchmark.

    That’s OK as far as it goes, but the MoJ don’t have data on everything. They don’t collect information on dynamic homelessness status or drug relapses or family breakdowns.

    So if your organisation provides help and support for class A drug users in the criminal justice system, your control group will be full of people who aren’t class A drug users. There will also be some that are, but there will be no way of knowing for sure. And as that is the key factor in your work – in my eyes (as a data analyst) the control group is weak and any analysis not robust.

    I suspect this is why the MoJ have asked providers of drug and alcohol services, along with those specialising in mental health, and others, to check first before making any submissions – though not ruling it out altogether.

    In recent weeks, I have offered through the Justice Data Lab team and via social media – to any VCS or other organisations interested – answer any questions that people may have as far as I can or help with submissions. Go for it – it’s only a one-year pilot and it may not continue if it isn’t used, and it’s a free service too!

  2. Thanks very much Jason for your offer of help. I hope lots of VCS agencies take it up.
    The MoJ are prepared to use OASys data (Offender Assessment System) to match offenders with, for example, drug or alcohol problems.
    It helps if agencies record this information themselves.

Leave a Reply to Jack Cattell Cancel reply

Your email address will not be published.

keep informed

One email every day at noon