Developing the evidence base
Last Friday (19 February 2021), Her Majesty’s Inspectorate of Probation published the latest in their Research and Analysis Bulletin series: Building the evidence base for high-quality probation services: The role of probation providers. The bulletin was based on a review of providers’ publications, an online survey of all probation providers and targeted interviews with researchers – both those working for probation providers and those externally based. The aim was to identify new contributions to the probation evidence base ahead of the re-unification of the service this June.
Transforming Rehabilitation promised to usher in a new era of modernisation and innovation in the probation sector. However, the inspectorate found limited evidence of effective research. Below I have reproduced the key findings from the bulletin.
- The overall level of research activities undertaken by probation providers over recent years has been disappointing. Within the Transforming Rehabilitation reforms, the requirements for providers to engage in evaluation and research were left loosely defined, and it is clear that any focus on research activities has been hindered by resource constraints and the financial difficulties reported by a number of providers.
- Interviewees told the inspectorate that some research projects proved unfeasible due to the inability to arrange access to the necessary data, staff and/or service users. Responsible officers have been stretched and often had little time to dedicate to supporting research. In relation to data within case management systems, interviewees reported that it was common to find too many gaps, restricting its value for analysis.
- Resource constraints also limited the attention given to research findings, with interviewees highlighting that it was difficult to fully engage with leaders and key stakeholders. It was common to find no clear mechanisms built into structures to learn from and use research findings.
- While the overall picture of research and evaluation activities is disappointing, there have been exceptions and positive examples of collaborative working, with strong interpersonal relationships, and the use of differing forms of dissemination and communication.
The Inspectorate has been increasingly interested in not just inspecting practice but in contributing to the development of a more evidence-based and effective probation service. The bulletin sets out five key aspirations for the re-unified service:
- Embed an evaluation culture: There needs to be a much stronger commitment to building a research/evidence-based culture, hardwired into organisational-wide delivery models. A shift is required, whereby supporting, co-producing or instigating research is recognised as a key part of working in probation, with clear links to professional learning, development and even career progression. An appetite to embrace and learn from research findings which are both negative and positive is also required.
- Identify the critical evidence gaps: Research resources need to be maximised, requiring a strategic, joined-up and holistic approach to monitoring research activities, identifying the most critical evidence gaps, and considering which questions can be answered in the short, medium and longer term, and who may be well placed to answer them.
- Tailor the research methods: Research questions will vary markedly in nature, and a wide range of research methods are required, with a recognition that differing approaches can be highly complementary. There is room for action-based research, in-depth case study work and longer-term experimental designs, as well as newer innovative methods, for example, visual methods in data collection.
- Support internal and external researchers: Those undertaking research projects, whether internal staff or external researchers, need to be sufficiently supported. This involves much more than financial resource, requiring the time of senior staff and engaged gatekeepers who can facilitate the necessary access. Establishment of a research community or forum would be beneficial, so that researchers can share good practice, resources, learning and expertise.
- Focus on dissemination, engagement and impact: For evidence to be used, impacting upon policy and practice, it needs to be reported in clear and accessible ways. A range of dissemination and communication methods should be used, with a focus on ensuring meaningful engagement and interaction, further embedding an evaluation culture and collaborative working.