Evaluation
Bread and butter, or an opportunity for learning?
Evaluations can be seen as the ‘bread and butter’ of the consulting life. Trundle through them, usual recommendations, submit the invoice and watch as the report gathers dust on a shelf.
Not interested.
In my experience, evaluations provide a fantastic opportunity for learning: for the institution of course, but also for me and colleagues involved in the process. They require a very similar skillset to needs assessment: a good understanding of emergency contexts, good analytical skills, participatory techniques, the ability to see the big picture and then think beyond the obvious. I have learned an enormous amount from my involvement in evaluations and honestly feel privileged to have been allowed to observe and scrutinise the programming of other professionals.
Evaluations also bring a set of challenges all of their own, especially where programmes have been less successful, where a lack of transparency undermines learning or (worst of all) when the real purpose is just a tick in the box.
I enjoy evaluations in which I feel the process and the recommendations have a strong chance to lead to positive change. I think an evaluation report deserves a formal management responses – even if it disagrees with my recommendations – and I like agencies with the transparency and confidence to publish critical reports. Humanitarian response is hard. It never goes perfectly, and that’s ok. We learn and we improve.
An inappropriate ambition:
The key messages of an evaluation process must be simple, easy to communicate, and sufficiently memorable to catalyse change in an organisation. This suggests that the ideal format for communicating these messages might not be the traditional narrative… The cartoons below were amongst a number drawn during team debrief sessions during an evaluation mission – but were never used. Shame, really…
A full list of evaluation contracts is provided below – click on any job title to get a more detailed description and access to reports where these are available.
evaluation consultancy contracts
Evaluation of DRR CCA capacity building programme
2017 – Team Leader – Konterra / WFP for a USD 20m 7 year DRR/CCA capacity building programme
Key evaluation themes: …
Use of CTP in Gaza
2015-2016 – Sole consultant – for Oxfam and the food sector
Main tasks: undertake two research components and multi-agency technical support into cash-based emergency responses for INGOs working in Gaza.
Innovation: new participatory approach for exploring beneficiary preferences in detail
Evaluation of ECHO funded unconditional cash transfers, Gaza
2014 – Sole Consultant – for Oxfam GB, Gaza
Key evaluation themes: exploring the concept of a human over-ride to the PMTF used to target safety net beneficiaries; issues of inclusion and exclusion; lessons in credit and debt; challenges with the monitoring tools – and a specific focus on the coping strategy tools.
Evaluation of UNHCR relationship with UNV (and UNOPS)
2014 – Sole Consultant – for UNHCR
Key Questions: what value can UNV add in emergency operations; what is the appropriate balance of UNV and UNOPS within the workforce?
Pakistan Simulation Exercise: country-wide flooding
2013 – Team Leader – for IFRC and the Pakistan RC
Tasks: Design of simulation materials; management of facilitation team; facilitation of 3 day event; lead lesson learning workshop and document.
Evaluation of cash transfer projects
2013 – Team Leader – for IFRC and Bangladesh Red Crescent
Task: Lead the team to undertake a detailed review of three linked cash transfer programme components for floods / landslide response. Facilitate a lesson learned worship for key stakeholders.
Monsoon Floods Evaluation
2012 – Sole consultant (coordinating and drawing on multiple linked sectoral reviews) for IFRC and Pakistan Red Crescent.
Key task: Undertake a substantial summative evaluation of relief and integrated recovery programmes after the 2010 floods
Evaluation: voucher programme for food insecurity
2012 – Sole consultant – for Oxfam GB Gaza
Key areas of study: food security, economic and social impacts; and cost and effectiveness comparisons with food distributions
Cash Transfer Programming
2012 – Sole consultant for IFRC
Tasks: revision of online CTP training; training delivery; development of internal advocacy materials on CTP
Evaluation of unconditional cash grants project, Gaza
2012 – Sole consultant – for Oxfam GB Gaza
Key tasks: review the transfer modality, quantify impact, and comment on gender relations and sustainability
Regional Disaster Response Teams Skills Development Review
2012 – Sole consultant – for IFRC South Asia
Key question: how can the training and development of RDRT members be made most effective?
Evaluation of Mekong floods emergency and early recovery operation
2011 – Team Leader – for IFRC and Viet Nam Red Cross
Key questions: impact and lessons learned across emergency relief, cash grants and permanent shelter.
Review of CTP training materials and monitoring tools
2011 – Sole consultant – for CaLP
Key tasks: consult trainers and participants, review monitoring tools, recommend changes to approach, content and monitoring
Review of the Economic Security training programme
2011 – Team Leader – for ICRC
Tasks: review the training programme, consider alignment with revised strategy, and recommend improvements
Evaluation: Tsunami Recovery Programme
2010 – Team Leader – for Spanish Red Cross
Main focus: Learning lessons from four livelihoods projects on the west coast of Sri Lanka.
Innovation: Creation of an analytical framework for classifying livelihoods recovery projects by a range of influencing factors.
Evaluation of the MOFA funded ‘delegates programme’
2010 – Sole Consultant – for Netherlands Red Cross
Key question: what was the impact on disaster response capacity of recipient organisations?
The Palestine Red Crescent response to operation Cast Lead in Gaza
2009 – Team member – for IFRC and Palestinian Red Crescent
Key question: Identify lessons learned during PRCS’ activities responding to operation Cast Lead in Gaza, during January 2009.
Evaluation: ACT response to Cyclone Nargis
2009 – Team Member – for the ACT Alliance
Evaluation of multi-sector relief efforts in the Ayeyarwaddy Delta
I was leading on lessons learned; accountability; application of Sphere Standards and the Code of Conduct.
Evaluation of UN supported GoI community-based DRM programme
2008 – Team member – for ADPC / UNDP
Large evaluation of huge UNDP / Government of India programme
Team member for Maharashtra State field work