Scoping Study: What Works in Protection and How Do We Know?

Executive Summary

Ever since accountability reforms were folded into the aid sector in the 1990s, humanitarian organizations and the larger relief system have developed and improved their ability to evaluate the impact of their work. Relief organizations have, in general, found it easier to measure the impact of their interventions in relation to material needs than activities geared to enhancing protection. For this reason, this scoping study asks ​“what works in protection and how do we know?” Three related questions will be discussed in the following order. The first is how to define humanitarian and human rights protection. The second is how to define success for different types of humanitarian and human rights protection interventions. The third question is how to measure the impact of different protection-oriented interventions. 

Scope and Methods of the Scoping Study

The primary responsibility for protecting civilian populations from violence and other forms of harm rests with national state authorities (e.g., the police, armed forces or the judiciary), international state actors (e.g. the UN, regional organizations, international justice institutions) and non-state authorities that control territory. A key objective for humanitarian and human rights actors is to change the policies and behavior of such primary ​“duty bearers” who have or may have a negative effect on civilian safety and well being. But the scope of humanitarian protection organizations is significantly broader. They implement specific measures to mitigate protection risks to communities and affected individuals and deliver specialized services to address the after-effects of violence and other patterns of harm. The research focus of this scoping study is limited to defining and reviewing the effects of different protection activities implemented by humanitarian and human rights organizations. However, while it is not the ambition here to assess the effectiveness of non-humanitarian protection efforts, this report does discuss the potential influence of political and military actors on the larger success or failure of humanitarian and human rights protection. 

This scoping study was commissioned by DFID as a desk study and undertaken by a team of two GPPi researchers and one independent consultant. Given that there is great variation in the way ​‘protection’ is defined or understood at the operational or program level, the team reviewed 173 documents, including academic works, evaluation reports and other grey literature, such as ​“how to” handbooks and guidelines, protection standards or policies. This includes 12 articles discussing the effectiveness of interventions in related fields, such as peacekeeping, peacebuilding or child protection in developments settings (see chapter 2). The study also draws on 40 semi-structured interviews with key informants from the UN, the Red Cross/​Crescent Movement, non-governmental organizations, academia and Western donor governments.

Summary of Protection Trends

In the context of humanitarian action, protection was traditionally the preserve of international law specialists and a few organizations mandated by international treaties and United Nations (UN) resolutions. After the end of the Cold War a diverse range of humanitarian and human rights organizations started to deploy staff to emergency settings where they engaged in efforts to enhance the protection of civilians. Today, protection has become an important element of the mission statement of a large number of humanitarian actors. Furthermore, the contextual scope of protection activities has broadened: In the past, only situations of armed conflict were seen as creating protection challenges, whereas now there is an increased recognition of protection needs in disaster settings associated with natural hazard events. Many of the latter also occur in situations of weak or contested governance or armed conflict settings. 

However, although there is more attention to protection than before, this scoping study revealed a tendency to consolidation in the protection sector. After more than two decades of continuous expansion of protection activities and multiplication of actors, ​“the pendulum may be swinging back,” as one interviewee put it. Incorporating protection perspectives into the design and delivery of relief programs is regarded as a minimum obligation by most humanitarian organizations but there is also a growing recognition that only a limited number of actors have the experience and will to engage primary duty bearers (i.e. state forces and armed groups) in a protection dialogue.

Defining success for different types of protection interventions

In this study, the term ​“humanitarian and human rights protection” in emergency settings refers to a set of activities that are concerned with countering violence and other patterns of harm such as sexual exploitation, discrimination, forced displacement and separation of families. Different protection activities are classified into three distinct types of interventions:

Protection Intervention Type 1: Providing Remedy to Individual Victims of Harm

This involves delivering material and non-material remedy to victims of violence and other patterns of harm and helping them to gain access to reparation and specialized care (e.g. medical assistance for rape victims, psychosocial counseling, legal aid, family tracing services). Such remedial action is essentially responsive. It attempts to mitigate suffering in emergency situations and does not attempt to reduce the incidence of particular patterns of violence or harm. To measure the effectiveness of such interventions, it is necessary to determine whether they helped to restore the dignity and well being of assisted victims and to prevent further harm.

Protection Intervention Type 2: Reducing Risk Exposure

This involves implementing specific risk mitigating measures to avoid or reduce the immediate exposure of civilian populations to violence and other patterns of harm. Mine-risk awareness campaigns are an example. Protection mainstreaming also falls under this category. Closely related to the ​“do no harm” principle, protection mainstreaming refers to efforts aimed at incorporating protection concerns into overall relief programming. While preventive in nature, risk-mitigating measures do not seek to address the deeper causes of violence and harm against civilians rooted in the policies or behavior of relevant state authorities or armed groups. Type 2 protection interventions prove successful when they help to reduce the incidence of particular incidents of harm (e.g. sexual violence) or physical injury (e.g. caused by anti-personnel mines) in specific geographic locations.

Protection Intervention Type 3: Changing Harmful Behavior of Primary Duty Bearers

Effective protection goes beyond efforts to reduce risk exposure and remedy the after-effects of sustained harm. It aims to secure an end to ongoing patterns of violence and harm that are detrimental to civilians and to inhibit their future occurrence. To persuade actors engaged in harmful practices to change their policies and behavior, humanitarian and human rights organizations carry out different forms of advocacy. The extent to which private and public advocacy helps to reduce the incidence of patterns of violence and harm in the affected country or sub-national region is the main determinant of success.

The scale of ambition that underlies each intervention varies considerably. For each of the three protection interventions, success is also defined differently. Humanitarian protection initiatives may be highly effective, for instance, in providing specialized care to the victims of violence – and thereby contribute to reduced civilian suffering – but they may fail to eliminate or at least reduce certain risks associated with harmful behavior. Thus, unqualified claims of general ​“failure to protect” need to be critically examined. Any attempt to determine the relative success or failure of humanitarian and human rights protection needs to distinguish between the three different types of interventions.

What Works in Protection: Key Findings Emerging from the Literature Review

The literature review revealed only a few sophisticated attempts at measuring the success of different types of protection interventions. Three main reasons account for the scant evidence on what works in protection across different contexts: 

• Quantity of information: Reviewed works focus on implementation challenges linked to capacity gaps, coordination issues and other practical matters. Questions of impact are addressed at the margins.
• Quality: About half of the different academic works reviewed lack an explicit research design and method, but clarity on design and method is a precondition for generating reliable data.
• Comparability: They lack a common conceptual framework to assess success in protection restricts the comparability of the findings that are presented in evaluative reports and scholarly enquiries.

It is generally easier to find negative examples of humanitarian protection efforts than positive ones – a central finding of the review. Analyzing past mistakes may yield important insights on what might work, but it does not provide concrete evidence of the circumstances under which certain types of interventions do prove effective.

Questions for Further Research

The report concludes with research recommendations based on the review of the literature and consultations with interviewees. The three research questions outlined below have been identified to address the current lack of a common methodology for indicator development and, more generally, to further our understanding about what works in protection: 

• What are common protection problems and related modes of action used across different contexts and organizations?
• What are appropriate methods and processes for determining impact and change triggered by different types of protection interventions?
• What are common external factors that enhance or limit the success of different protection interventions across contexts?

Some of the challenges in measuring and attributing success in relation to humanitarian and human rights protection are similar to those encountered in complex development interventions. Where relevant, the final chapter refers to the wider literature pertinent to this topic and discusses the potential value and relevance of solutions devised for development interventions and related fields, such as international peacebuilding.