Learning about Payment by Results in Water, Sanitation & Hygiene (WASH)
Payment by Results is the favoured commissioning tool used by the UK government for its international aid programmes.
I’ve been following an excellent series of posts by the team responsible for verifying and evaluating the results of the Department for International Development’s programme to bring water, sanitation and hygiene (WASH) to 4.5 million people in the Democratic Republic of Congo and Kenya.
Although the programme runs until 2018, the monitoring and evaluation team (@WASHresultsMVE) is keen to share its learning as soon as possible.
It’s intriguing to see that many of the emerging lessons mirror those from quite different sectors such as worklessness and homelessness.
One of the main objectives of the WASH programme is to make its gains sustainable and the team’s most recent blog post pays considerable attention to this issue.
Advantages of PbR
The post shares learning from a recent event attended by several providers/suppliers of the WASH programme and members of the evaluation team. The main advantages of a PbR (compared a to traditional grant approach) contract were identified as:
- Greater flexibility
- More rigorous monitoring and evaluation (both internally by providers and externally by the verification team) leading to better and more reliable outcomes.
- A greater attention on sustainability (because it was linked to specific outcome measures)
- Greater transparency and accountability
- The need to achieve results drove innovation, especially in more challenging environments where standard approaches were not working.
Disadvantages
The main disadvantages highlighted were:
- Higher risks for providers and significant investment pre-contract.
- Initial lack of clarity about outcomes which required a longer lead-in period to get right.
- A disincentive for providers to share learning.
- Encourages the classic PbR problem of cherry picking — in this case the temptation to target resources at less vulnerable and more accessible communities.
- Unforeseen costs to providers in providing monitoring and verification information
- The tight timescales did mitigate against innovation in some areas.
Smartphone surveys
Andy Robinson, lead verifier, made an interesting point about the effective use of apps:
One positive outcome of the need for verified results has been the use of smartphone survey applications, which have greatly sped up and reduced the cost of the survey process; improved data processing and quality control; and made it much easier to verify large-scale results quickly. A key learning from the PBR programme is that these household surveys appear to be a far quicker and more effective way of evaluating programme outcomes than conventional evaluations.
Conclusion
Overall, the monitoring and verification team found that PBR approach appears to be improving M&E approaches and systems, encouraging more thinking about how to measure and evidence outcomes and sustainability, and providing reliable feedback on progress and performance at regular intervals during the life of the programme. This feedback enables regular improvements to be made to programme policy, planning and practice (unlike conventional programmes, which often are not rigorously evaluated until the end of the programme duration).
If you are considering commissioning, investing or providing under a PbR contract, you may be interested in my new PbR online tool which can be found at: www.PbR.russellwebster.com
The tool was funded by the Oak Foundation and is based on a literature review and wide ranging consultations with policy experts , commissioners, providers and service users.
The purpose of the new tool is to help commissioners, investors and providers consider whether it might be appropriate to use PbR for a particular service. The tool asks key questions on both the rationale for using PbR and key elements of the contract such as defining and validating outcomes and guarding against common PbR problems such as “creaming and parking” and unintended consequences.
The tool provides immediate feedback, followed up by summaries of key research. Everything is evidence-based and the tool is completely free to use.
3 responses
Glad you are enjoying our blog posts Russell! As you say, it is interesting how the experience of the WASH Results Programme, which is working across 12 countries in Africa and Asia , mirrors the lessons from the UK context.
Your blog highlights how data collected to verify results is being used to promote innovation and performance management within the delivery of the WASH Results Programme. We were also interested in this point that emerged in the conference session that our colleague Andy was reporting on, because this has not always been the reported experience of the WASH Results Suppliers to date. In fact, previous reflections (eg during the Learning Event earlier this year reported here https://www.gov.uk/dfid-research-outputs/dfid-wash-results-programme-learning-event ) have suggested that the high potential cost of failure (ie not being paid for work undertaken) has acted as a barrier to innovation by encouraging Suppliers to play it safe, also that the time spent collecting data for verification has, at times, squeezed out time for learning and course correction. It seems there is a mixed and/or evolving picture across Suppliers’ experiences that we need to understand more. The evaluation of the programme is looking into this, it is certainly an interesting area to watch.
We have a few blog postings in the pipeline: one sharing reflections from the evaluation so far and two that explore PBR and the issues of accountability and alignment with national priorities, both important considerations in international development. Hope those will be of interest too. Thanks again for kind words!
Catherine Fisher ( Learning Adviser for the Monitoring, Verification and Evaluation Team of the WASH Results Programme)
Thanks for your comment Catherine. It’s certainly true in other sectors that unless commissioners specify innovation (and fully resource it), providers can rely instead on proven best practice for fear of failure (and not being paid). This also seems to be a factor sometimes with investors of Social Impact Bonds who want reassurance that they will get return on investment.
Looking forwards very much to your next blog posts.
Russell