Recent Evaluations

How well has DFID's Country Programmes performed? - Synthesis of DFID's Country Programme Evaluations (CPEs) published

DFID’s Evaluation Department (EvD) commissions a team of independent consultants to carry out a programme of evaluations DFID’s country programmes each year.

The individual studies are intended to improve performance, contribute to lesson learning and inform the development of future strategy at country level. Collectively, the CPEs are important in terms of DFID’s corporate accountability and enable wider lessons across the organisation to be identified and shared.

The 2005/6 programme examined DFID’s performance in Rwanda, Malawi, Ghana, Bangladesh and Mozambique. A synthesis of the 5 studies has now been published.

The report highlights a number of DFID’s strengths including our:

  • alignment to the Paris Declaration principles
  • high quality in-country presence
  • innovative thinking on riskier reform areas
  • growing financial weight and predicable resource flows
  • effective targeting around MDG objectives

However, the report also identifies areas for improvement around:

  • choice of aid instruments and balancing risk in the use of budget support
  • building on strong relationships developed through decentralised offices and flexibility, but finding an appropriate balance between government and non-state actors
  • strengthening M&E systems to focus more on assessing outcomes and supporting national M&E systems
  • preparation and execution of Country Assistance Plans (CAPs),

Read the full reportPDF document(833 kb)

Review of DFID environmental screening report

DFID commissioned this study to find out how environmental screening procedures were being implemented following publication of new guidance in 2003, the Environment Guide: A guide to environmental screeningPDF document(2.4 mb). The study shows that some things are done well and appear to be improving. But it also points to areas where we need to improve. We will be following up on the study with an action plan to address these areas. We will also carry out a further review of environmental screening in 2008/09 which will focus on how actions identified through screening are followed up in implementation.

Read the Review of DFID environmental screening reportPDF document(1.35 mb)

For more information please contact Anna Ballance (a-ballance@dfid.gov.uk ) in the Environment for Sustainable Development Team.

Evaluation of the UK’s strategy for tackling HIV and AIDS in the developing world - update

Measuring the success of the Taking Action Strategy: Invitation to contribute your views by 21 November 

A draft of Working Paper 3 of the Taking Action evaluation is now available.

This paper, prepared by an independent consultant, outlines proposed indicators of progress against the 'Taking Action' strategy (see below for a link to the strategy). These will be used to measure its success, in the final evaluation of the strategy in 2008/9. You are invited to send in your comments on these.

Getting the indicators right means taking into account four important aims:

(a) Measuring the things that are most important;

(b) Measuring things that can be measured in developing country conditions and without very high cost;

(c) Minimising the number of things we decide to measure and track, to minimise administrative overheads; and

(d) Using jointly agreed indicators and measures, that are collected internationally (for example by UNAIDS) or by joint country reviews, rather than inventing new ones for the UK government. This is due to our international agreements to harmonise our efforts with other donors and reduce the reporting burden on our partner countries.

Please consider these four aims when making your comments.

Please send written comments to the author, Dr Roger Drew, roger.drew2@btinternet.com  by 21 November.

Evaluation of DFID’s policies and programmes in support of Gender Equality and Women’s Empowerment (Gender Evaluation)

DFID’s Gender Evaluation is now complete. The main phase of this independent evaluation of DFID’s performance was conducted between May 2005 and August 2006, and built on earlier scoping and thematic review phases.

The evaluation asked about DFID’s effectiveness in delivering on our external gender equality objectives, the suitability of our internal systems for promoting gender equality, and DFID’s role in the international effort to address gender equality and women’s rights. It consists of three programme case studies (of India, Nigeria and the Western Balkans), together with thematic studies on DFID’s International partnerships, Budget support, and Justice and Rights. The main findings and recommendations of the report are drawn together in the Synthesis Report.

Main findings:

  • The pursuit of gender equality and women’s rights are necessary as goals in their own right and for reaching wider poverty reduction goals;
  • DFID has shown strength and leadership in policy making and knowledge development, and DFID-funded gender research and studies have had worldwide influence on gender knowledge and on other donors’ efforts;
  • DFID needs to do more to ensure these strengths in policy and knowledge are reflected in practice, since programming and results on gender equality are uneven;
  • The use of new aid modalities, in particular budget support, is both a challenge and an opportunity for DFID (and others) to pursue gender equality. The gender orientation of DFID’s budget support depends mainly on how gender is dealt with in a partner country’s PRS. Specific country experience from Uganda showed the importance and potential of strong domestic champions to promote national ownership of gender issues.
  • DFID’s performance management framework has helped to ensure gender equality is pursued in the education and health sectors, but has been less successful in other areas of our work;
  • The internal incentives for working on gender equality have not been sufficiently supportive, principally leadership.

The evaluation findings and recommendations are being used to strengthen DFID’s approach and performance in this area.

Read the full report and evaluation summary

Evaluation of the UK’s strategy for tackling HIV and AIDS in the developing world - update

Measuring the success of the Taking Action Strategy: Invitation to contribute your views by 31 October 

A draft of Working Paper 3 of the Taking Action evaluation is now available here (main and annexes). This paper, prepared by an independent consultant, outlines proposed indicators of progress against the 'Taking Action' strategy (see below for a link to the strategy). These will be used to measure its success, in the final evaluation of the strategy in 2008/9. You are invited to send in your comments on these.

Getting the indicators right means taking into account four important aims:

(a) Measuring the things that are most important;

(b) Measuring things that can be measured in developing country conditions and without very high cost;

(c) Minimising the number of things we decide to measure and track, to minimise administrative overheads; and

(d) Using jointly agreed indicators and measures, that are collected internationally (for example by UNAIDS) or by joint country reviews, rather than inventing new ones for the UK government. This is due to our international agreements to harmonise our efforts with other donors and reduce the reporting burden on our partner countries.

Please consider these four aims when making your comments.

Please send written comments to the author, Dr Roger Drew, roger.drew2@btinternet.com  by 31 October.

Latest report: Trends in UK Government Funding

The first working paperPDF document(1 mb) from the evaluation is now available . Some of the key findings are:

  • In general, the working paper is positive about the upward trends in UK government funding for tackling HIV and AIDS in the developing world. It finds that six of the seven priority areas of Taking Action are “reflected” or “strongly reflected” in UK support. It recommends that more attention is paid to the sustainability of actions supported (the seventh area).
  • The paper points out that there are "significant methodological challenges" in counting spending on HIV and AIDS. "This is not simply an issue of measurement but of fundamental differences in approaches to funding …it is clearly easier to measure ‘vertical’ funding of HIV and AIDS programmes. However, this risks considerably underestimating the contribution made by an agency that funds mainly in a different way", for example by strengthening national health systems. "The UK government is not alone" in facing measurement challenges, and the paper calls on DFID to "provide international leadership in this area".
  • Nevertheless, “indications are that the UK government is on track to meet the spending targets laid out in Taking Action.”

Readers are reminded that this is a working paper, and that its findings and conclusions may be modified as more evidence is received. If you have information or views you wish to share with the consultants, please contact Jane Gardner, the evaluation administrator j-gardner@dfid.gov.uk 

More….

The first product of the evaluation was an inception reportPDF document(3 mb) laying out how the consultants planned to address the evaluation questions. The second working paper, on ‘Reaching Women, Young People and Vulnerable Groups’ will be available in December. Country visits are taking place from May to September. Country case studies include China, Democratic Republic of Congo, Ethiopia, India, Russia, Zambia and Zimbabwe. The main report will be published by February 2007.

Background

Taking ActionPDF document(1 mb), the Government’s strategy for tackling HIV and AIDS in the developing world was launched by the Prime Minister in July 2004. The UK is giving high-level political leadership in this area and committing significant funds (at least £1.5 billion over 3 years, up from £270 million in 2002/3), including additional funding for major players such as the Global Fund and UNAIDS, and a strengthened commitment to support for women, orphans and vulnerable children.

The Secretary of State and Parliamentary Under Secretary of State are particularly concerned to ensure systems are in place to measure the impact of the additional resources allocated to tackling HIV and AIDS, and to ‘monitor, evaluate and challenge interventions’. With this in mind, a lesson-learning evaluation is now underway. This will concentrate on DFID, which is the lead government department but will also focus on the contribution by other UK Government Departments. The evaluation will aim to draw on existing information and joint working as much as possible, in support of the “three ones” principles.

This is an independent consultancy, carried out by a consortium led by SSS (USA-based consultancy firm). The lead author is Dr Roger Drew.

What has US $4 billion of Poverty Reduction Budget Support (PRBS) bought?

The Joint Evaluation of General Budget Support led by DFID on behalf of 24 donors and 7 partner governments has concluded. The evaluation assessed GBS from all donors in 7 partner countries over the last 5 years – a total of US$ 4 billion. The final reports will all be published on 9 May to coincide with an International Dissemination Conference to be hosted by the OECD DAC in Paris.

The evaluation asked to what extent and in what contexts GBS is a relevant, efficient and effective means of delivering sustainable poverty reduction and growth. 7 country level evaluations were conducted in Burkina Faso, Malawi, Mozambique, Nicaragua, Rwanda, Uganda and Vietnam. Conclusions and recommendations were brought together in a final Synthesis Report (which also includes material from last year’s evaluation of GBS in Tanzania). The Synthesis Report draws conclusions about the overall effects of the GBS instrument and provides good practice lessons and recommendations for its future use.

Key findings on the effects of GBS (cross referenced to paragraphs in the Synthesis Report) include:

  • GBS designs vary across countries and have evolved over time (5.13-5.15)
  • GBS has been a relevant response to certain acknowledged problems in aid effectiveness (5.21) 
  • GBS can be an efficient, effective and sustainable way of supporting national poverty reduction strategies. It played a clearly positive role in five of the seven case study countries (Burkina Faso, Mozambique, Rwanda, Uganda and Vietnam). In one country (Nicaragua) GBS was at such an early stage that firm conclusions about its effects were not yet possible. In another (Malawi), GBS was not successfully established during the evaluation period. (See Section 4 for a country-by-country overview of the assessment).
  • Provision of discretionary funds through national budget systems has produced the systemic effects on capacity, and particularly PFM-related capacity, that are posited in the evaluation framework (including transparency, comprehensiveness of the budget and budget efficiency), and these effects are government-wide in nature (5.52). GBS was not a panacea, but it strengthened government ownership and accountability (5.20-5.49), and in the short–medium term, there were useful effects on allocative and operational efficiency of public expenditures, including aid ; these were linked with medium–longer term systemic effects on improving the links between policy and results (5.63–5.64).
  • There were enough positive spillover effects to suggest that GBS can be more than the sum of its parts. In particular GBS tends to enhance the country-level quality of aid as a whole, through its direct and indirect effects on coherence, (1.18) harmonisation and alignment (5.28), on partner government transaction costs(5.14-5.42), and on the overall efficiency of resource use (5.43). This makes GBS a particularly valuable addition to the array of aid instruments in use, and highlights the need to employ GBS as part of a strategy that takes account of the interplay between different aid modalities (6.62 onwards).
  • As regards poverty reduction (5.84 onwards), it was too soon for the ultimate effects of GBS inputs during the evaluation period to be manifest. Its initial effects have been mainly through supporting the expansion of basic public services, notably in health and education (5.83). GBS is a vehicle that assists in implementing a poverty reduction strategy. Its ultimate effectiveness in reducing poverty is bound in with the quality of the PRS that it supports. Given the bias of early PRSPs towards the expansion of public services, most of the effects of GBS inputs (funds, dialogue, TA, etc) so far are likely to have been on access to services, rather than income poverty and empowerment of the poor.
  • It is important not to overload the GBS instrument, but we found in all cases a capacity to learn from experience (5.99) which suggests that GBS could become more effective, and have a broader scope, over time.
  • The evaluation also considered possible unintended effects of GBS. It did not find evidence of significant crowding-out of private investment nor of the undermining of domestic revenue effort (5.74–5.76). Malawi's experience showed that GBS could have a destabilising effect when basic conditions for disbursement are not met; in other countries GBS design has been improved to limit short-term unpredictability (5.68–5.72). Corruption can undermine all forms of aid; systemic strengthening of public finance management, which GBS supports, is an important part of a broad anti-corruption strategy (6.82–6.99). All of these potential adverse effects, however, also represent risks that need to be taken into account in the design of GBS (and other aid).
  • More generally, GBS as presently designed, is vulnerable to a number of risks, including political risks, that threaten its ability to operate as a long-term support modality (6.72–6.98). Its sustainability depends on making it more resilient.
  • GBS is part of a family of programme-based approaches (3.8, 6.62), and many of our findings are also relevant to programme based approaches in general.

GBS Synthesis ReportPDF document(5 mb)

GBS Burkina Faso Country StudyPDF document(5 mb)

GBS Malawi Country StudyPDF document(2 mb)

GBS Mozambique Country StudyPDF document(3 mb)

GBS Nicaragua County StudyPDF document(4 mb)

GBS Rwanda County StudyPDF document(3 mb)

GBS Uganda County StudyPDF document(2 mb)

GBS Vietnam County StudyPDF document(615 kb)


Last updated 21 February 2007