With few measures currently agreed upon, how can we judge the impact of humanitarian assistance following natural disasters? In this guest post, Christopher Lee*, tells us about his new paper just published in BMC Public Health, which describes an impact assessment of humanitarian services carried out in Aceh, Indonesia, following the 2004 tsunami.

On a sunny morning in Banda Aceh in January 2011, I woke up to the familiar sound of morning prayers and the occasional chirping bird. I was going to be driving down the coast to Meulaboh, another major town in Aceh Province in northern Sumatra that was devastated by the 2004 Indian Ocean tsunami and subsequent earthquake.

The history of the tsunami in Aceh is a unique one. Prior to the tsunami, the province was embroiled in ongoing conflict between a separatist group the Gerekan Aceh Merdeka (GAM, or the ‘Free Aceh Movement’) and the Indonesian military. During this time, human rights abuses were reported on both sides, and Human Rights Watch estimated there were approximately 15,000 civilian casualties between 1976 and the imposition of marshal law in 2003.

The province was closed to international aid workers until six weeks after the tsunami, which precipitated an unprecedented humanitarian response and effectively ended the years of conflict.  The disaster was also the first major test for the Sphere Project, a set of rights-based standards and indicators for the provision of humanitarian assistance.

Around noon, the sun was high overhead. We had been driving for a few hours and now had to disembark from our car so that we could put it on a crudely made motorized raft to cross a small river.  I wondered, despite the hundreds of millions of dollars invested into this highway, how much had peoples’ lives changed not only a result of the tsunami but also as a result of the humanitarian response?

There exists no single measure that can capture the complexity of a recovered life. Natural disasters tend to cause a precipitous loss of life in the immediate stages, but years are spent on immediate assistance, reconstruction, and recovery.

What is measured often depends on who is doing the measurement; academics often focus on specific measures like post-traumatic stress, non-governmental organizations are often assessed by their specific outputs, and governments and intergovernmental organizations often focus on systems recovery and infrastructure. These measures often fail, however, in capturing the multiple dimensions by which survivors recover and now operate in the world.

In 2010, the UN General Assembly endorsed the Framework on Durable Solutions for Internally Displaced Persons, recognizing that like refugees, the internally displaced may effectively end displacement when they return to their original homes, integrate locally into new environments, or resettle elsewhere in the country.

Guided by existing human rights instruments, a set of indicators were developed to determine how effectively displacement had ended which include the attainment of an adequate standard of living, safety and security, access to livelihoods, justice and reparation, restitution of property, recovery of documents, family reunification, and participation in all levels of public affairs. We saw this as an opportunity to, for the first time, apply these rights-based measures to those affected by the tsunami.

Six years after the tsunami, we found that most of these measures had improved when compared to households’ recall of their pre-tsunami life. These findings, published in BMC Public Health suggested that the Indonesian government’s approach of ‘building back better’ was largely effective.

We next sought to determine how we could identify what aspects of the humanitarian response may have led to better outcomes. This question was crucial – we know that households that are wealthier and less affected by the immediate disaster tend to do better, but are there factors that, if intervened on by the humanitarian community, could actually lead to meaningful changes after the recovery process?

In order to create a single measure containing all of the indicators of the attainment of durable solutions, we used principal components analysis (PCA). PCA was an ideal approach because it can generate a single score that is self-weighted for the different types of variables used to construct it. After assigning every household a durable solutions score, we were then able to see what aspects of the recovery process were associated with higher or lower scores.

We found that shelter and legal assistance were beneficial regardless of when they were delivered, but that livelihood assistance was most effective when delivered later on in the recovery process. We also found that the duration that people were displaced was a consistent predictor of their current outcomes. Although everyone in the sample had effectively settled somewhere, the longer it took them to do so was linearly associated with diminished attainment of durable solutions scores.

We are now looking at this data to determine what factors are associated with prolonged displacement, which may help prioritize certain types of interventions to minimize the time spent in displacement and potentially have a long-term beneficial effect on survivors of disasters. We have found, for instance, that households that lived in camps following the tsunami were displaced on average almost a year longer than those who lived with friends and family members.

The Framework on Durable Solutions is publicly available and provides a human rights-based way to measure the end of displacement. We believe that these measures can be used by researchers, governments, and non-governmental organizations to assess the success of recovery and reconstruction processes and determine where responses have succeeded and failed.

We finally arrived in Meulaboh. The city had changed, but not beyond recognition. The barracks on the outskirts were gone, families zipped by on motorbikes, and I found to my delight that my favorite fried chicken restaurant was still there.

*Christopher Lee, MD MSc MPH, is a resident physician in internal medicine at the University of California, San Francisco and an associate of the Center for Refugee and Disaster Response at the Johns Hopkins Bloomberg School of Public Health