The story so far
Last week (14 November 2017), the Department for Communities and Local Government finally published the final evaluation reports of the London Homelessness Social Impact Bond (SIB) where providers were paid on a payment by results basis for their effectiveness in helping rough sleepers.
Although inexplicably more than one year overdue, the reports are comprehensive and make for interesting reading. A bevy of reports were published:
- The impact evaluation report
- The qualitative evaluation
- Learning from the qualitative evaluation (focusing on the operation of the SIB)
- Learning from the operating model (known as the “navigator model”)
This four-year scheme produced a substantial amount of learning because two different organisations (St Mungo’s and Thames Reach) were each contracted to deliver services to half of a cohort of 831 entrenched rough sleepers. Each organisation chose a different form of SIB and it was possible to compare outcomes directly on a like-for-like basis.
The different SIB formats and the delivery model are covered in detail in my post on the initial interim evaluation. This initial evaluation came to the following conclusions:
- The payment by results approach had stimulated providers to develop an innovative and effective delivery model
- Performance against targets in the first year was mixed
- There was a significant amount of work involved for both commissioners and providers in developing a SIB and the payment mechanism for a PbR approach.
The second interim evaluation found promising results but highlighted a key challenge for the rest of the SIB term — how could providers balance their approach so that resources were focused on achieving a maximum return on payment for outcomes (keeping those in new-found accommodation) while continuing to support those who were still rough sleeping?
The graphic below shows the navigator model used by both providers for this project:
Final evaluation findings
The results show that, when compared to a well-matched comparison group, the intervention significantly reduced rough sleeping over a two year period.
On average the intervention group (the SIB group) had significantly fewer episodes of rough sleeping compared to the comparison groups.
After one year:
- The mean number of rough sleeping contacts (jargon for the number of times people were physically spotted sleeping rough) for the SIB group was 6.2 compared to 8.4 for the comparison group.
After two years:
- The mean number of rough sleeping contacts for the SIB group was 9.2 compared to 13.9 for the comparison group
By extrapolating from the two year results, it is estimated that around 3,900 rough sleeping episodes have been avoided (over two years) as a result of the intervention.
Furthermore (and critically) the intervention group was also significantly more likely than the comparison group to completely desist from sleeping rough in the two years following the start of the programme.
- 47% of the SIB group did not sleep rough at all in the one year following the start compared to 40% of the comparison group
- 40% of the SIB group did not sleep rough at all in the two years after the start compared to 33% of the comparison group
The impact evaluation also found that the intervention had a significant positive impact on the number of people arriving into long-term accommodation one and two years after the start of the programme, with significantly more people arriving at long-term accommodation from the SIB cohort than in the matched comparison group.
- 23% of the SIB group had an arrival to long-term accommodation in the one year following the start compared to 4% of the comparison group
- 37% of the SIB group had an arrival to long-term accommodation in the two years after the start compared to 7% of the comparison group
Conclusions
Provider staff, from senior to front line levels, were all proud of the achievements of the SIB and the many individual success stories they were able to identify, although they would have liked to have achieved more. Both providers were able to pay investors their principal sum with interest. This meant that any further outcome payments achieved during the payment tail will be retained by them for reinvestment in services (including the maintenance of small teams to support the cohort during these final months).
The model of support, taking a long term and personalised approach that builds a trusting relationship for persistent and challenging support, as discussed in previous reports, was seen to be effective. The SIB promoted innovation in the use of a keyworker model providing intensive support for this target group with complex needs. The PbR provided a focus on what outcomes could be achieved across the cohort and there was no evidence of ‘cherry picking’.
Investors in both SIB contracts were happy with overall performance and thus with their return on investment. Whilst disappointed that more outcomes could not be achieved, they understood this in the context of the SIB providing learning both about these investments but also about interventions aiming to achieve social outcomes with cohorts with complex needs.
While the findings from the impact evaluation are very positive, and clearly show that the intervention was effective against key outcomes, they do not necessarily prove that social investment and payment incentives drove the results. This is because it was not possible, within this impact evaluation, to disentangle the effect of the social investment model from the intervention service. However, the qualitative evaluation suggests that the Payment by Result (PbR) element of the SIB contract had a greater impact on the providers than the investors’ involvement.
Key learning points
The evidence from the evaluation suggests that the Navigator intervention model is effective in supporting entrenched rough sleepers with high levels of complex needs. The two providers had different cost models for their teams, with a lower cost team at Thames Reach, and broadly similar outcomes achieved up to the start of the payment tail. The evaluation evidence suggests that the key features of effective provision are:
- A relational and non-judgmental approach that is persistent and builds trust;
- A long-term approach that extends from initial street contact to sustained outcomes across the full pathway of support, which is usually split across different organisation.
- Support that can be split across different roles along the recovery pathway, if this is carefully negotiated on a case by case basis;
- Support that is provided in home languages and culturally sensitive;
- A focus upon a personalised package of flexible and responsive support tailored to individual circumstance and need;
- An immediate focus upon securing appropriate accommodation and providing practical and emotional support to sustain this;
- A focus upon supporting clients to access existing provision, including advocating for and coordinating appropriate support;
- Effective partnership working – both in identifying key partners and in building positive relationships with those who are receptive and resistant to joint working;
- Flexible funding that can be used to purchase goods and services quickly and according to individual need; and,
- Delivery by skilled, motivated practitioners.
