As an international development professional, I provided technical support to the Ministry of Health of Timor-Leste from mid 2007 to early 2012 in managing the National HIV/AIDS, TB and Malaria programs funded by the Global Fund. For Timor-Leste, this was a period of reconstruction and transition towards development.
Before emerging as a fully independent country in 2002, Timor-Leste (also known as East Timor) suffered from several hundred years of Portuguese occupation until 1975, Indonesian occupation until 1999, followed by widespread violence that destroyed almost 70% buildings and infrastructure and displaced about 75% of the population.
Timor-Leste received much attention from the international development community with a variety of project support approaches. The approaches ranged from state avoiding mechanisms and contracting Non-Government Organizations (NGOs) for service delivery to going through the government mechanisms for project implementation. There were other approaches such as global health partnerships that go through the government but employ different implementation mechanisms and fall in-between the state-avoidance and state building ends of the continuum.
It occurred to me that the presence of different aid modalities within the same context of transition and fragility in Timor-Leste’s health sector could be an interesting topic for an in-depth analysis of the role of institutional arrangements and health systems strengthening in the effectiveness of aid interventions.
By taking the context of Timor-Leste, I was encouraged to explore the feasibility of a comparative evaluation of aid management approaches in a particular health system.
Economists, for many years, tried to measure the effect of aid on national economic growth and saw mixed results. Buried under macro-economic analysis, such studies often lacked a particular focus on the health sector in a particular country. With new discourse on aid effectiveness and an attempt to alter what had not worked so far, donors, recipients, and civil society made the Paris Declaration in 2005 agreeing to the principles of country ownership, harmonization, alignment, mutual accountability and managing for results.
However, actual effectiveness of adhering to these principles in fragile situations in terms of achieving intended objectives and development results have not been adequately studied so far. By taking the context of Timor-Leste, I was encouraged to explore the feasibility of a comparative evaluation of aid management approaches in a particular health system.
I left Timor-Leste early 2012 and undertook a doctoral research at the School of Preventive and Social Medicine of the University of Otago, New Zealand with my supervisors Professor Robin Gauld and Professor Philip Hill. We were primarily interested in developing and piloting a comparative evaluation approach in a particular health system. It appeared that both health systems and aid management approach influence the outcomes of an aid intervention.
Following the tradition of a theory-driven “realist evaluation”, we believe that in a particular context, different aid management modalities would produce different outcomes, which can be fairly compared with each other to see what worked, why they worked, under which conditions, and for whom.
For data collection and analysis, we employed mixed methods combining different analytical tools and techniques including balanced scorecards for stakeholder perceptions, cost-effectiveness and technical efficiency analysis, data envelopment analysis (DEA), and correlations analysis. We also used mathematical modeling to see the likely effect of absence of an intervention on the current results. Results were then validated by comparing findings from different techniques.
With some identified limitations such as small sample size, analytical tools and techniques used in this research showed potential as an approach for comparatively evaluating national level aid funded projects, especially in difficult situations. The article published in Global Health Research and Policy presents our evaluation framework and explains step by step methods of implementing our approach.
Comments