How to improve your practice
The latest in the probation inspectorate’s admirable Academic Insights series was published yesterday (16 December 2021). Written by Hazel Kemshall, Professor of Community and Criminal Justice at De Montfort University, it focuses on Bias and error in risk assessment and management. Risk is one of the dominant topics in contemporary probation practice with many of us concerned that the service has become over-preoccupied with managing risk to the detriment of its work to promote desistance. Prof Kemshall’s report is extremely helpful, recognising the complexity of the issue and the pressure on staff to get risk management right – and the personal consequences if they don’t. She acknowledges that personal bias, the risk assessment tools used and the organisational processes and culture all influence risk management practice.
Personal bias and sources of error
Prof Kemshall starts by acknowledging the size of the task: that probation officers have to make difficult decisions in complex situations, often with limited information. Risk management always involves difficult choices between possible benefits and possible harms. In this context she argues that intuitive reasoning and recourse to a range of subjective biases can easily take place, sowing the seeds of potential error, before listing a number of key sources of bias identified in the evidence base. This is a fascinating list and I urge readers who are interested to read the review in full rather than relying on my summary:
- Over perception and reaction to vivid and ‘catastrophic’ risks at times out of proportion to the likelihood of them happening. This can result in ‘dread risks’ where practitioners over-focus on the feared impact but fail to calculate the probability of the event occurring.
- ‘Dread risks’ can also create practitioner avoidance due to anxiety and challenge. A sense of being ‘out of one’s depth’. Feelings of stress and anxiety can result in practice paralysis which can be insidious and corrosive.
- Failure to recognise the personal values influencing individual practice and the role they play in both assessment and management decisions. The outcomes we see as desirable and worthy influence the choice of management strategies we employ in order to achieve case outcomes. This can be particularly acute in the balance of risks and rights, and rehabilitation and risk prevention.
- Misperception of causality. Because B follows A we tend to think A has caused B without fully analysing this and establishing the link. We then focus on A as the way to prevent risk but do the wrong thing. X lost his job and then did Y, therefore employment is the risk solution. Actually, the situation can be more complex. X does Y in situations of stress or challenge because it is comforting, or re-establishes power, control and self-esteem. Therefore, any situation of stress and challenge needs avoiding, mitigating or managing, and employment is not the simple risk solution it appears in this example
- The power of ‘unreal optimism’ — practitioners tend to perceive small change for the better with greater weight than is justified by objective evidence. This can result in risk minimisation and/or a tendency to overemphasise benefits when presenting risk options
- The role of confirmatory bias with risk assessors tending to select and weigh information that confirms initial views of risk, or which continue to confirm initial assessment. Practitioners then fail to refine their risk assessment, and consequently risk management strategies are mis-aligned to risk.
- Practice can become routinised, that is habitual and bureaucratic. When you do a lot of risk assessments, it’s easy to cut and paste either mentally or literally.
- Dodging the issue – in particularly difficult cases, it can be tempting not to ask the pertinent questions or not to pursue them in full.
- The tendency to seek ‘risk-free’ options in risk management practice, rather than accepting that risk management requires evidenced and robust weighing up of a number of risk options over time.
Professor Kemshall advocates getting into good habits to try to ensure that critical reasons replaces first impressions and our existing biases. She shares two useful exercises for us to work through to help tackle our own biases and values. Here is the first one:
- make the first impression and the evidence it is based on explicit
- explicitly check any first impression against contrary or disconfirming views/ perspectives/evidence
- make a deliberate and conscious effort to think of evidence for the opposing view
- consider whether the information disconfirms what you already believe
- consider why the information makes you uncomfortable and why you are tempted to discount it
- identify the other consequences or possible outcomes that the information requires you to consider
- identify the changes to the risk assessment and the risk management plan that the information requires you to make.
The second checklist aims to foster a self-critical, reflective approach to decision making:
- don’t overemphasise the ‘vivid, concrete, emotive, first impressions, or the most recent’
- don’t overlook the ‘dull, abstract, statistical and old’
- know your weaknesses and review honestly your emotional investment in the case
- constructively test your views, be evidential, seek information you do not have, and be prepared to have your views challenged and tested by colleagues
- change your mind in the light of new information and evidence
- consider alternative actions and outcomes
- use management supervision and advice, and seek challenge.
This last point reminds me of my own (admittedly antiquated) probation practice when all court reports on people from racially minoritised backgrounds were read by two colleagues at a draft stage in order to quality assure them. Peer conversations and challenges were a significant way of us all improving our own – and each other’s – practice.
After discussing the use of different risk tools, Prof Kemshall looks at the way in which an organisation can try to enshrine best risk management practice. Again, she provides a useful checklist for senior management to review how fit for purpose risk practice and policy is:
- does your risk policy have a clear statement of aims and objectives?
- is the underpinning value-base of the risk policy clear?
- does it clearly allocate roles and responsibilities?
- does it indicate how risk dilemmas are to be resolved and competing outcomes evaluated and balanced?
- does it state minimum staff competence, training requirements and management oversight?
- does it clearly allocate resources?
- has the policy been clearly explained to staff? If staff were asked about the policy, would they be able to explain the policy and what they are required to do?
- does the policy unhelpfully encourage shortcuts and non-compliance from staff?
I’d be fascinated to hear what current probation managers and staff think of current risk practice. Please use the comments section below to start a debate.
Thanks to Loic Leray for kind permission to use the header image in this post which was previously published on Unsplash.