Real-Time Evaluation of Humanitarian Assistance Revisited

The concept of real-time evaluation (RTE) has evolved and spread considerably since its introduction some two decades ago, and since reviews first looked comprehensively at its application. It has formed an integral part of evaluation practice among humanitarian organizations, who are continuing to further develop similar approaches for immediate evaluative feedback.The most recent example is is the case of the emerging Inter-Agency Standing Committee concept of Real-Time Operational Reviews. These organizations face manifold challenges with regard to access to populations, lack of quality data, rapidly changing agendas and demands from policymakers in a generally chaotic (that is, unpredictable) crisis situation. While struggling with these challenges, they need to show results and the ability to learn if they want to sustain support from donors and the wider public – and make a difference on the ground. Real-time evaluation should support them in this difficult task.

However, the spread of ​“real-time” approaches seems to be as much a result of entrepreneurial evaluators who aim to demonstrate expertise and innovation to potential clients as it is a response to actual demand. Today, RTE is often presented as the newest and sharpest tool in the evaluator’s toolbox. This article will first turn to broader trends in evaluation theory and debate, such as evaluation under complexity and its potential to stimulate organizational learning, to help understand where real-time evaluation came from and why it is so popular. Our study then uses a systematic review of evaluation reports (see Appendices A and B for a list of reports covered and interviews conducted) and key informant interviews to see the extent to which current practice is congruent with the popular theoretical descriptions of RTE. By looking at the reports of practical application of RTE in humanitarian action, the study demystifies the approach and takes stock of its strengths and weaknesses. Finally, we suggest in what situations real-time evaluation makes most sense and how it could be improved.

We argue that in order to improve the practical benefit of real-time evaluation, humanitarian organizations need to be even more selective and modest in its use. Wherever real-time evaluation is used – it should prioritize endogenous learning in organizations over questions of accountability and control.

To read the full article, please visit the Journal of MultiDisciplinary Evaluation online.