Preface

The Multilateral Organisation Performance Assessment Network (MOPAN) is a network of donor countries with a common interest in assessing the organisational effectiveness of multilateral organisations. MOPAN was established in 2002 in response to international fora on aid effectiveness and calls for greater donor harmonisation and coordination.

Today, MOPAN is made up of 16 donor countries: Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Ireland, The Netherlands, Norway, Republic of Korea, Spain, Sweden, Switzerland and the United Kingdom. For more information on MOPAN and to access previous MOPAN reports, please visit the MOPAN website (www.mopanonline.org).

Each year MOPAN carries out assessments of several multilateral organisations based on criteria agreed by MOPAN members. Its approach has evolved over the years, and since 2010 has been based on a survey of key stakeholders and a review of documents of multilateral organisations. MOPAN assessments provide a snapshot of four dimensions of organisational effectiveness (strategic management, operational management, relationship management, and knowledge management). In 2012, MOPAN is piloting a new component to examine an organisation’s development results in addition to its organisational effectiveness.

MOPAN 2012

In 2012, MOPAN assessed six multilateral organisations: the African Development Bank (AfDB), GAVI Alliance (formerly the Global Alliance for Vaccines and Immunisation), the Joint United Nations Programme on HIV/AIDS (UNAIDS), the United Nations Development Programme (UNDP), the United Nations Children’s Fund (UNICEF), and the World Bank.

MOPAN Institutional Leads liaised with the multilateral organisations throughout the assessment and reporting process. MOPAN Country Leads monitored the process in each country and ensured the success of the survey.

Acknowledgements

We thank all participants in the MOPAN 2012 assessment of the African Development Bank. The Bank’s senior management and staff made valuable contributions throughout the assessment and document review processes and provided lists of their clients to be surveyed. Survey respondents contributed useful insights and time to respond to the survey. The MOPAN Institutional Leads, Canada, Switzerland and the United Kingdom, liaised with the Bank throughout the assessment and reporting process. The MOPAN Country Leads oversaw the process in the field and ensured the success of the survey. Consultants in each country provided vital in-country support by following up with clients to enhance survey response rates.

Roles of Authors and the MOPAN Secretariat

The MOPAN Secretariat, led by Ireland in 2012 and co-chaired by Germany, worked in close cooperation with the MOPAN Technical Working Group to launch and manage the survey. MOPAN developed the Key Performance and Micro-indicators, designed the survey methodology, coordinated the development of lists of survey respondents, and approved the final survey questionnaire. MOPAN also directed the design of the approach to document review. MOPAN oversaw the design, structure, tone, and content of the reports.

Universalia and Epinion developed the survey instrument and carried out the survey and analysis. Universalia carried out the document review and wrote the reports.

Epinion is a leading consulting firm in Denmark that analyses and evaluates data to support decision making. It conducts specially designed studies for public and private organisations based on data collected among an organisation’s employees, members, customers, partners, and other sources. Epinion has 75 employees and 200 interviewers. Website: www.epinion.dk

Universalia Management Group is a Canadian consulting firm established in 1980 that specialises in evaluation and monitoring for international development. Universalia has made significant contributions to identifying best practices and developing tools in the fields of organisational assessment; planning, monitoring, and evaluation; results-based management; and capacity building. Website: www.universalia.com.

Acronyms

Figures

Figure 1.1 Comparison of MOPAN Results for the AfDB in 2009 and 2012
Figure 2.1 Respondent Rating Scale
Figure 2.2 The African Development Bank - Distribution of Responses (n=152) on all Questions Related to Micro-Indicators
Figure 2.3 Number of Survey Respondents for the AfDB by Country and Respondent Group
Figure 2.4 MOPAN Ranges and Descriptions
Figure 3.1 Overall Ratings of the AfDB's Organisational Effectiveness by Respondent Group
Figure 3.2 Overall Ratings on Key Performance Indicators (mean scores, all respondents and document review ratings)
Figure 3.3 Quadrant I: Strategic Management, Survey and Document Review Ratings
Figure 3.4 Quadrant I: Strategic Management, Mean Scores by Respondent Group
Figure 3.5 KPI 1: Providing Direction for Results, Ratings of Micro-Indicators
Figure 3.6 KPI 2: Corporate Focus on Results, Ratings of Micro-Indicators
Figure 3.7 KPI 3: Focus on Thematic Priorities, Ratings of Micro-Indicators
Figure 3.8 KPI 4: Country Focus on Results, Ratings of Micro-Indicators
Figure 3.9 Quadrant II: Operational Management, Survey and Document Review Ratings
Figure 3.10 Quadrant II: Operational Management, Mean Scores by Respondent Group
Figure 3.11 KPI 5: Resource Allocation Decisions, Ratings of Micro-Indicators
Figure 3.12 KPI 6: Linking Aid Management to Performance, Ratings of Micro-Indicators
Figure 3.13 KPI 7: Financial Accountability, Ratings of Micro-Indicators
Figure 3.14 KPI 8: Using Performance Information, Ratings of Micro-Indicators
Figure 3.15 KPI 9: Managing Human Resources, Ratings of Micro-Indicators
Figure 3.16 KPI 10: Performance-oriented Programming, Ratings of Micro-Indicators
Figure 3.17 KPI 11: Delegating Authority, Ratings of Micro-Indicators
Figure 3.18 Quadrant III: Relationship Management, Survey and Document Review Ratings
Figure 3.19 Quadrant III: Relationship Management, Mean Scores by Respondent Group
Figure 3.20 KPI 12: Supporting National Plans, Ratings of Micro-Indicators
Figure 3.21 KPI 13: Adjusting Procedures, Ratings of Micro-Indicators
Figure 3.22 KPI 14: Using Country Systems, Ratings of Micro-Indicators
Figure 3.23 KPI 15: Contributing to Policy Dialogue, Ratings of Micro-Indicators
Figure 3.24 KPI 16: Harmonising Procedures, Ratings of Micro-Indicators
Figure 3.25 Quadrant IV: Knowledge Management, Survey and Document Review Ratings
Figure 3.26 Quadrant IV: Knowledge Management, Mean Scores by Respondent Group
Figure 3.27 KPI 17: Evaluating External Results, Ratings of Micro-Indicators
Figure 3.28 KPI 18: Presenting Performance Information, Ratings of Micro-Indicators
Figure 3.29 KPI 19: Disseminating Lessons Learned, Ratings of Micro-Indicators
Figure 4.1 Development Results Component – Overall Ratings
Figure 4.2 KPI A: Evidence of the Extent of Progress towards Organisation-Wide Outcomes, Ratings
Figure 4.3 KPI B: Evidence of the Extent of Contribution to Country-Level Goals and Priorities, Rating
Figure 4.4 AfDB Stakeholder Survey - DRC, Mean Scores by CSP Priorities
Figure 4.5 AfDB Stakeholder Survey - Ghana, Mean Scores by CSP Priorities
Figure 4.6 AfDB Stakeholder Survey - Morocco, Mean Scores by CSP Priorities
Figure 4.7 AfDB Stakeholder Survey - Niger, Mean Scores by CSP Priorities
Figure 4.8 AfDB Stakeholder Survey - Nigeria, Mean Scores by CSP Priorities
Figure 4.9 KPI C: Evidence of Contributions to Relevant MDGs, Overall Rating and Criteria
Figure 4.9 KPI D: Relevance of Objectives and Programme of Work to Stakeholders, Overall Rating and Survey Mean Scores by Country

Executive Summary

This report presents the results of an assessment of the African Development Bank (AfDB) conducted by the Multilateral Organisation Performance Assessment Network (MOPAN). MOPAN assesses the organisational effectiveness of multilateral organisations based on a survey of stakeholders, a review of documents, and interviews with headquarter-based staff. In past years, MOPAN has not assessed an organisation’s development results, but is testing a component on this with four organisations in this year’s assessment.[1]

The AfDB’s mission and objective is to spur sustainable economic development and social progress in regional member countries by mobilising and allocating resources and providing policy advice and technical assistance to support development efforts. The Bank provides concessional and non-concessional loans, grants, and technical assistance to clients in regional member countries.

In recent years, the AfDB has undertaken a reform process to improve its effectiveness, efficiency and corporate governance. Since the creation of the Quality Assurance and Results department (ORQR) in 2008 it has implemented a number of initiatives, including the development of a results framework covering the whole Bank Group, new reporting tools at the organisation-wide level, new quality at entry processes, strengthened practices in the area of human resource management, and further decentralisation through the establishment of new Field Offices.

In 2012, MOPAN assessed the AfDB based on a survey of key stakeholders, a review of documents, and interviews with AfDB staff. The survey targeted AfDB’s clients and MOPAN donors based in-country and at headquarters. Six countries were included in the MOPAN survey: Democratic Republic of Congo, Ghana, Morocco, Niger, Nigeria and Zimbabwe. A total of 152 respondents participated in the survey. MOPAN’s document review assessed the AfDB through an examination of publicly available corporate documents and country programming documents from five of the six countries selected. MOPAN assessments provide a snapshot of four dimensions of organisational effectiveness (strategic management, operational management, relationship management, and knowledge management). The main findings of the assessment of the AfDB in these performance areas and in a pilot component on development results are summarised below.

Strategic Management

In the area of strategic management, MOPAN established criteria to determine if a multilateral organisation has strategies and systems in place that reflect good practice in managing for results. Overall, the 2012 assessment found that:

• The AfDB’s performance in strategic management is seen to be adequate overall by both survey respondents and the document review. Its key strengths include an organisational strategy that is based on a clear mandate and a demonstrated commitment to transparency. The strong alignment between Country Strategy Papers (CSPs) and national development plans was also noted.

• The AfDB is considered adequate in addressing the six thematic areas identified as key cross-cutting priorities by MOPAN and the AfDB: gender equality, environment, climate change, private sector development, fragile states, and regional economic integration. Donors at headquarters see the mainstreaming of gender equality as an area for improvement. The document review noted the Bank’s increased commitment to these priorities demonstrated through the introduction of new policies, practices and processes in recent years, but also highlighted the lack of compliance mechanisms and need for better monitoring of results.

• Areas for improvement include linking organisation-wide results statements to higher level outcomes or impacts and improving the consistency of Country Strategy Paper results frameworks in the formulation of results at outcome and output levels. This is a challenge that all multilateral development banks are facing.

Operational Management

In operational management, MOPAN established criteria to determine if a multilateral organisation manages its operations in a way that supports accountability for results and the use of information on performance. Overall, the 2012 assessment found that:

• The AfDB has transparent systems in place for the allocation of resources and survey respondents believe the Bank generally follows the criteria established for resource allocation.

• The Bank has introduced some tools to facilitate the implementation of results-based budgeting, but this has not yet become standard practice in the Bank. There remains considerable room for improvement in linking disbursements to results achieved.

• The AfDB has sound practices and processes in place for financial accountability. External and internal audits are seen as strong and adhere to international standards. The Bank’s policies and guidelines for combating fraud and corruption were commended in the document review.

• The AfDB makes use of performance information to improve its operations, but could improve its systems for tracking the implementation of evaluation recommendations that are accepted by management and reported to the Board.

• The AfDB is commended for its use of performance-oriented programming and there is clear evidence of how it uses the Performance Based Allocation system on its website.

• The AfDB has made considerable efforts to improve its human resource management practices in recent years, and new performance management systems have been developed. In spite of this, the organisation has not yet institutionalised these practices or instilled a performance-based culture.

• Delegation of authority to the country level remains limited, but the current decentralisation process and the establishment of new field offices in several African countries are steps in that direction.

Relationship Management

In relationship management, MOPAN established criteria to determine if a multilateral organisation is engaging with its partners at the country level in ways that contribute to aid effectiveness. Overall, the 2012 assessment found that:

• The AfDB is seen as adequate in its support of clients and national plans and its contributions to policy dialogue. Many survey respondents commented positively on the Bank's recent decentralisation efforts, highlighting its increasing field presence as key to facilitating better dialogue and enhanced support to RMCs.

• There is some room for improvement in the efficiency of the Bank’s administrative procedures, its use of country systems, the extent to which it participates in joint missions, and the extent to which it provides financing through programme-based approaches.

Knowledge Management

In knowledge management, MOPAN established criteria to determine if a multilateral organisation has reporting mechanisms and learning strategies that facilitate the sharing of information inside the organisation and with the development community. Overall, the 2012 assessment found that:

• The AfDB was commended for the independence of its Evaluation Unit (OPEV) and was generally seen to involve beneficiaries and clients in evaluations.

• Although the Bank’s Evaluation Policy refers to accepted international standards for quality, the Evaluation Unit does not appear to have a formal process for reviewing the quality of evaluations.

• The AfDB was seen to adequately present performance information, including performance on its Paris Declaration commitments, although there is room for improvement in reporting on its contributions to development outcomes.

• The Bank was commended for its updated results framework, its new reporting practices at the organisation-wide level (e.g., the Annual Development Effectiveness Review) and its organisation-wide reporting on key Paris Declaration performance indicators.

• Other areas identified for improvement include AfDB’s use of indicators and lack of specific country targets to report on the Paris Declaration Indicators.

Development Results

The 2012 pilot assessment focused on the evidence of the AfDB’s contributions to development results.

• Evidence of the extent of progress towards organisation-wide outcomes: The Bank has made progress towards lower level results (i.e., outputs) in 9 of its 10 focus areas and has met most of the targets defined in ADER 2012. Its contributions to higher level results (i.e., outcomes) remain unclear.

• Evidence of the extent of contribution to country-level goals and priorities: While the AfDB reports on the country’s progress towards national goals, it provides little evidence of the extent to which it contributed to these results.

• Evidence of the extent of contribution to relevant MDGs: Explicit links to specific MDGs are rare in both Country Strategy Papers and completion reports and reports do not explain AfDB’s contributions.

• Relevance of objectives and programme of work to stakeholders: The AfDB received mixed ratings from survey respondents but generally strong ratings for responding to client countries’ key development priorities.

Trends since 2009

The AfDB was assessed by MOPAN using the Common Approach methodology in 2009 and 2012. In both years, the Bank’s performance was largely perceived as adequate overall by survey respondents. There was a positive trend in stakeholders’ perceptions of the Bank’s practices in two areas: management of human resources and adjustment of procedures to take into account local conditions and capacities, both of which were rated inadequate in 2009 and adequate in 2012. There was a slight decline over the years in stakeholder perceptions of the Bank’s allocation of resources based on established criteria.

Conclusions

AfDB stakeholders surveyed in 2009 and 2012 largely considered the Bank’s performance as adequate in most areas assessed. In both years, survey respondents indicated the need to increase its delegation of decision-making authority to the country level.

According to survey respondents, the Bank has made progress in human resource management practices as well as in its ability to adjust procedures to local conditions and capacities, both of which were rated inadequate in 2009 but adequate in 2012.

In human resource management, survey respondents particularly noted the Bank’s efforts to instil a culture that is transparent and meritocratic. In adjusting procedures, they noted some improvement in the Bank’s use of procedures that can be easily understood and completed by partners, its ability to quickly respond to changing circumstances, and its flexibility in the implementation of projects and programmes.

In both 2009 and 2012, surveyed stakeholders perceived the Bank’s procedures as too lengthy and noted the need for the Bank to increase country-level decision making.

The AfDB remains strongly committed to a results-focused reform agenda and is improving its capacity to manage for development results. However, the frameworks and systems that it uses to manage for and report on organisation-wide results need further improvement, particularly in formulating and presenting the intended links between lower level and higher level results.

As in previous MOPAN assessments, the AfDB continues to be recognised for its strong African identity and regional ownership, which has a positive impact on its legitimacy among African governments and its understanding of regional needs and priorities. Indeed, both survey respondents and the document review noted the strong link between AfDB’s strategy and its clear mandate. The Bank has been implementing an ambitious set of reforms in recent years – focusing, among other things, on instilling a results-oriented culture throughout the organisation. Although the Bank received relatively modest ratings from survey respondents on indicators related to its reform agenda, that agenda is starting to yield concrete results. With the establishment in 2008 of the Quality Assurance and Results Department (ORQR) – which is responsible for managing results reporting and implementing the Bank’s results agenda – the Bank has confirmed its commitment to, and has strengthened its capacity in, this area.

An important part of the Bank’s reform process included the revamping of its results measurement frameworks (RMFs), which since 2010 have been consolidated into one results framework known as the “One Bank” results measurement framework. The reform process also led to the introduction of the Annual Development Effectiveness Review (ADER), an organisation-wide report on development effectiveness. The Annual Development Effectiveness Review represents a major improvement in the Bank’s reporting on results and provides an annual overview of its performance according to core sector indicators. While this Review identifies some of the tangible outputs achieved by Bank-supported operations, evidence of contributions to higher level development outcomes in the region is still limited. There is a notable gap between aggregate outputs (and intermediate outcomes) achieved by Bank- supported projects and higher level outcomes in Africa. The Bank does not present a theory of change that could contribute to minimising the gap between these levels of results. This is a challenge that all multilateral development banks are facing.

The Bank is also in the process of developing Development Effectiveness Reviews (DERs), which are thematic and country reviews that analyse achieved results in particular areas of the Bank’s portfolio. It is anticipated that the publication of these reviews will provide a clearer indication of the Bank’s contributions to country level results as well as results achieved in several areas of focus.

The Bank is recognised for aligning its country strategies with national development priorities and its country strategies reflect a sound analysis of the country context. Assessing the extent to which the AfDB contributes to country level development outcomes remains a challenge.

Both survey respondents and the document review found that the Bank’s country strategies are clearly aligned with national Poverty Reduction Strategic Plans (or equivalent). Indeed, in all Country Strategy Papers (CSP) reviewed, there were clear and explicit links between AfDB’s expected results and those identified in national plans.

However, the quality and format of CSP results frameworks vary, and the use of results statements and indicators is inconsistent. Further, while most of these frameworks link expected outputs and outcomes to results at the country level, not all statements of results are appropriate to their results level, links between outputs and outcomes are not always clear, and some output statements are in fact outcome statements and vice versa. Therefore, it can be difficult to discern AfDB’s contribution to country level outcomes. In addition, none of the Country Strategy Papers analysed provide any explicit theories of change, and most of the results frameworks assessed are based on very high level results statements that are difficult to correlate with the results of the Bank’s operations. That being said, all Country Strategy Papers build on a solid analysis of the country context and more recent results frameworks have improved in quality and consistency as compared to the previous generation.

The AfDB is seen to have sound policies and processes for financial accountability.

The Bank received strong ratings for its policies and practices for audit and combating corruption. It is considered adequate in other areas, such as its risk management, procurement and contract management processes, as well as procedures undertaken to respond and follow up on irregularities.

AfDB’s human resource management is the focus of an on-going reform process. According to both document review and survey respondents, the Bank is still facing challenges in this area.

In 2009, the Bank adopted a new performance management system (PMS) and developed a handbook that outlines its practices for managing staff performance and performance review procedures for evaluating the performance of senior management in general (although it is not clear if this applies to the level of Vice Presidents).

There are still challenges related to the implementation of the performance management system, including delays in completing each stage of the performance cycle and a tendency towards overly high ratings. A recent review of this system noted several factors that hindered its full application and effectiveness, including: lack of management ownership, poor leadership, lack of people management skills among managers, and inadequate compliance mechanisms to enforce consistent use of the new system.

There is also a perceived lack of transparency in AfDB’s system to manage and reward staff performance. While the Bank has a performance-based pay system that rewards staff based on merit, the effectiveness of the system is dependent on the operationalisation of the performance management system.

The Bank has increased its country presence significantly in recent years through the establishment of additional field offices, but its delegation of decision making authority to the country level remains a work in progress.

As of 2012, the AfDB has a field presence in 34 out of 54 regional member countries and ensures its presence in other countries through its regional office and regional resource centres located in Dakar, Nairobi and Pretoria, respectively. Delegation of decision making is an operational area that has significant implications for the efficient and effective delivery of aid at the country level. While the Bank adopted a Roadmap[2] in 2011 that mapped out the its vision for transferring decision-making authority to Field Offices over the period 2011-2015, decision- making authority remains quite centralised, with very little authority having been delegated to country or regional levels. For example, project budget reallocations and project approvals cannot be made at the country level and both survey respondents and the document review considered the Bank inadequate in this area. However, it must be noted that for many multilateral development banks, the restructuring of projects must be approved by Boards of Directors.

The AfDB is currently considering allocating more decision-making authority to regional levels and has recently converted the Bank’s field offices in Nairobi and Pretoria to Regional Resource Centres (RRCs) that cover the eastern and southern Africa regions. It is also updating its Delegation of Authority Matrix and, according to OPEV, has made some progress in delegating authority for procurement to Field Offices.

The Bank has made progress on most of its Paris Declaration commitments. There is room for improvement in its use of country systems and its participation in joint missions and programme-based approaches.

According to the 2011 OECD/DAC Survey on Monitoring the Paris Declaration, the Bank has made progress in ensuring that aid flows are recorded in partner countries’ budgets, reducing the use of parallel project implementation units and disbursing technical cooperation through coordinated programmes.

However, less than 10 per cent of the Bank’s missions are being jointly undertaken, and the recent Quality of Official Development Assistance Assessment (QuODA) places the Bank among the multilateral development banks with the lowest share of coordinated joint missions. The 2011 OECD Survey and an AfDB evaluation from 2011 indicated that the Bank has made limited progress in the use of common arrangements or procedures, and the Bank received moderate ratings in the MOPAN assessment from both document review and survey respondents for its somewhat limited use of country systems.

According to the AfDB evaluation, the Bank’s participation in programme-based approaches is limited by its application of procurement rules of origin, lack of guidance to staff, and its generally cautious approach to financial management. However, the Bank’s cautious approach must also be considered in context as it has faced challenges in dealing with corruption and lack of government capacity in many Regional Member Countries.

Development Results Component

The Bank’s progress towards organisation-wide outcomes is not clear.

MOPAN donors at headquarters were positive about AfDB’s contributions to infrastructure development (in particular transport) and regional integration, but see its performance in other areas as adequate.

While AfDB’s data on results at the organisation-wide level is consolidated in the Annual Development Effectiveness Review, it does not give a complete picture of the Bank’s results achievement and some details are better presented in sector evaluations and studies. Reports provide evidence of progress towards planned lower level results in the One Bank results framework, but limited evidence of AfDB’s overall contribution to higher-level change.

Although the Bank’s country-level results are aligned with national priorities, the extent to which its operations contribute to the achievement of country-level goals or relevant Millennium Development Goals is not always clearly evident in its reporting.

At the country level, stakeholders gave the Bank generally strong ratings for responding to their countries’ key development priorities but mixed ratings on its overall relevance and its contributions to MDGs.

AfDB’s Country Strategy Paper results statements are aligned with national goals and priorities and the Bank reports on the country’s progress towards national goals. However, the Bank’s reports do not provide a clear picture of the nature, magnitude, or relative importance of AfDB contributions to these results and most reports do not describe the Bank’s role or contribution in support of its clients’ efforts to achieve the MDGs.

The Bank has been strengthening its country reporting tools and systems, which may lead to better evidence of its contributions to country development outcomes in the future.

Overall MOPAN Ratings of the AfDB

The chart below shows the ratings on the 19 key performance indicators that MOPAN used to assess the AfDB in 2012. These indicators were designed to measure organisational effectiveness (practices and systems) and not development results on the ground. The AfDB received ratings of adequate on 18 of the 19 key performance indicators assessed by survey respondents, and document review ratings ranging from inadequate to strong.

1. Introduction

1.1 MOPAN

This report presents the results of an assessment of the African Development Bank that was conducted in 2012 by the Multilateral Organisation Performance Assessment Network (MOPAN). In 2012 MOPAN assessed six multilateral organisations: the African Development Bank (AfDB), GAVI Alliance (formerly the Global Alliance for Vaccines and Immunisation), the Joint United Nations Programme on HIV/AIDS (UNAIDS), the United Nations Development Programme (UNDP), the United Nations Children’s Fund (UNICEF), and the World Bank.

Background

MOPAN was established in 2002 in response to international fora on aid effectiveness and calls for greater donor harmonisation and coordination. The purpose of the network is to share information and experience in assessing the performance of multilateral organisations. MOPAN supports the commitments adopted by the international community to improve the impact and effectiveness of aid as reflected in the Paris Declaration on Aid Effectiveness, the Accra Agenda for Action, and the Busan High Level Forum. MOPAN’s processes and instruments embody the principles of local ownership, alignment and harmonisation of practices, and results-based management (RBM).

MOPAN provides a joint approach (known as the Common Approach) to assess the organisational effectiveness of multilateral organisations. The approach was derived from existing bilateral assessment tools and complements and draws on other assessment processes for development organisations – such as the bi-annual Survey on Monitoring the Paris Declaration on Aid Effectiveness and annual reports of the Common Performance Assessment System (COMPAS) published by the multilateral development banks. In the long term, MOPAN hopes that this approach will replace or reduce the need for other assessment approaches by bilateral donors.

MOPAN assesses four dimensions of organisational effectiveness

MOPAN has defined organisational effectiveness as the extent to which a multilateral organisation is organised to contribute to development and/or humanitarian results in the countries or territories where it operates.

Based on a survey of stakeholders and a review of documents, MOPAN assessments provide a snapshot of a multilateral organisation’s effectiveness in four dimensions:

• Developing strategies and plans that reflect good practices in managing for development results (strategic management)

• Managing operations by results to support accountability for results and the use of information on performance (operational management)

• Engaging in relationships with direct partners and donors at the country level in ways that contribute to aid effectiveness and that are aligned with the principles of the Paris Declaration (relationship management)

• Developing reporting mechanisms and learning strategies that facilitate the sharing of knowledge and information inside the organisation and with the development community (knowledge management).

In 2012, MOPAN also piloted a new component to assess a multilateral organisation’s contributions to development results. This component was tested with four of the six organisations assessed this year (AfDB, UNDP, UNICEF, and the World Bank).[3]

Purpose of MOPAN assessments

MOPAN assessments are intended to:

• Generate relevant, credible and robust information MOPAN members can use to meet their domestic accountability requirements and fulfil their responsibilities and obligations as bilateral donors.

• Provide an evidence base for MOPAN members, multilateral organisations and direct partners to discuss organisational effectiveness and in doing so, build better understanding and improve organisational effectiveness and learning over time.

• Support dialogue between MOPAN members, multilateral organisations and their partners, with a specific focus on improving organisational effectiveness over time, both at country and headquarters level.

The MOPAN methodology is evolving in response to what is being learned from year to year, and to accommodate multilateral organisations with different mandates. For example, the indicators and approach for the 2012 MOPAN review of a global fund and organisations with significant humanitarian programming were adapted to reflect the reality of these organisations.

1.2 Profile of the African Development Bank (AfDB)[4]

Mission and Objective

The African Development Bank (AfDB) is an international development finance institution that was founded in 1964. Together, the AfDB, the African Development Fund (ADF) established in 1973, and the Nigeria Trust Fund (NTF) established in 1976, constitute the African Development Group.

The AfDB is owned and financed by 77 member countries (53 African countries and 24 non- African countries). It is currently located in Tunis, Tunisia, having temporarily moved there from its official headquarters in Abidjan, Côte d’Ivoire. The Bank has field presence in 34 borrowing member countries and an external representation office (ERO) in Tokyo.

The Bank’s mission is to assist African countries achieve “sustainable economic development and social progress”. Its core objectives are to mobilise resources for investment and provide policy advice and technical assistance to support the development efforts of regional member countries (RMCs).

Governance and structure

At the top of the AfDB’s organisational structure is the Board of Governors, which comprises ministers and high level officials of economic, financial and development institutions that meet once a year and that delegate day to day decision making to a 20-member Board of Directors. The Bank’s Board of Directors is chaired by a president who is always an African and who is elected by the Board of Governors for a five-year term that is renewable once. Three AfDB offices report directly to the Board of Directors: the Operations Evaluation unit (OPEV), the Compliance Review and Mediation Unit (CRMU) as well as the administrative tribunal (TRIB). As of December 2011, the AfDB Group had 1902 employees.

Strategy and services

The AfDB has sought “to promote the investment of public and private capital in projects and programmes that are likely to contribute to the economic development of its borrowing regional member countries”.

In 2008, the AfDB developed a Medium-Term Strategy (MTS) for the period 2008-2012, to guide its work and to serve as a roadmap for all of its operations. The MTS identified four main operational areas for the Bank to focus on (i.e., infrastructure development, governance, private sector development, and higher education), and noted the need to further integrate the key cross-cutting themes of gender, environment and climate change into its operations.

To fulfil its mission, the Bank provides concessional and non-concessional loans, grants, and technical assistance to clients in regional member countries. Since its establishment in 1964, it has provided 3,661 loans and grants totalling UA 60.06 billion as of December 2011, most of which were invested in sectors related to the operational areas identified in the current MTS, namely: infrastructure, economic and financial governance, human and social development, industry, mining and quarrying, agriculture and rural development and environment.

Results Agenda

Since the creation of the Quality Assurance and Results department (ORQR) in 2008, the AfDB has been implementing a series of reforms aimed at strengthening its focus on results. This has involved prioritising several areas.

Results orientation: In 2010, the Bank introduced the “One Bank” results measurement framework (RMF), a consolidated results framework that serves as a corporate tool to monitor progress toward the goals set in the 2008-2012 Medium-Term Strategy. In 2011, it also launched an annual organisation-wide report, the Annual Development Effectiveness Review (ADER), which among other things reports on the Bank’s contribution to development of Africa.

Quality at entry: In 2011, the Bank introduced the Readiness Review (RR) tool to improve the quality at entry of the AfDB’s operations and country strategies.

Decentralisation and harmonisation: In 2011, the Bank mapped out its vision for transferring decision-making authority to Field Offices (FOs) over the period 2011-2015 in Improving Performance on Aid Effectiveness and Promoting Effective Development - Turning Commitments Into Action. In 2011 it opened field offices in four fragile states: Burundi, Togo, South Sudan and Central African Republic (CAR).

As of 2012, the AfDB has field presence in 34 out of 54 Regional Member Countries. In addition to Field Offices (FOs) it ensures its presence through two Regional Resource Centres (RRCs) in Pretoria and Nairobi, a Regional Office (RO) in Dakar, as well as customised presence in Guinea Bissau, Sao Tomé and Principe, and Mauritius.

Human resource management: As indicated in previous MOPAN assessments, the AfDB has been undertaking initiatives aimed at resolving human resources issues, particularly those related to recruitment, performance management, and the compensation and benefits framework. As of 2012, the Bank had undertaken new recruitment and retention initiatives and was in the process of revising its compensation framework for the period 2013-2016.

For additional information, please consult the AfDB website: http://www.afdb.org

1.3 Previous Assessments

Since its establishment in 2003, MOPAN has conducted four assessments of the African Development Bank (2004, 2007, 2009, and 2012). Although the MOPAN methodology has been expanded since 2004, findings from previous MOPAN surveys can provide insight into the evolution of perceptions of the Bank’s surveyed stakeholders.

2004 and 2007 MOPAN Surveys

The 2004 and 2007 MOPAN assessments consisted of surveys of the perceptions of MOPAN member staff about the Bank’s performance at the country-level. The 2004 MOPAN survey was carried out in five countries (Benin, Burkina Faso, Mali, Rwanda, and Uganda), and the 2007 survey in six countries (Benin, Egypt, Ethiopia, Mali, Senegal, and Zambia).

Both surveys highlighted several areas in which stakeholders felt the Bank could improve: its in- country presence, its contributions to policy dialogue, and its involvement in advocacy and capacity development activities. The 2007 survey found that the Bank had made modest improvements relative to the 2004 survey in terms of ensuring alignment with national strategies and coordinating with other donors in-country.

Trends since 2009

In 2009, MOPAN introduced the Common Approach, which incorporated the views of national partners/clients of the Bank, and thus had a broader scope than previous surveys. In 2010, it added a document review. Figure 1.1 compares the 2009 and 2012 survey ratings of the AfDB on Key Performance Indicators. While it illustrates some interesting trends, some caution is required in making comparisons given that there were changes between 2009 and 2012 in: the survey scale (from a 5-point to 6-point scale), the wording of some questions, and the countries and respondent groups surveyed. In addition, some 2009 survey questions were removed and assessed only by document review in 2012. With these caveats in mind, the 2012 survey illustrates an improving trend in 6 of the 19 indicators assessed.

Figure 1.1 Comparison of MOPAN Results for the AfDB in 2009 and 2012

Strategic management

Operational management

Relationship management

Knowledge management

* The 2009 MOPAN survey of AfDB was carried out in Ethiopia, Mozambique, Senegal, and Uganda.
** The 2012 MOPAN survey of AfDB was carried out in the Democratic Republic of Congo, Ghana, Morocco, Niger, Nigeria and Zimbabwe.

2. MOPAN Methodology – 2012

2.1 Overview

Background

MOPAN continues to refine its assessment framework. In 2009, the MOPAN Common Approach replaced the Annual MOPAN Survey, which had been conducted since 2003. The Common Approach is broader and deeper than the previous surveys and includes the following components:

• Expanded survey – The MOPAN survey now brings in the views of direct partners or clients of multilateral organisations, peer organisations (or other relevant stakeholder group), and those of donors, that is, MOPAN members at both headquarters and country level.

• Document review – Since 2010, survey data are complemented by a review of documents prepared by the multilateral organisations being assessed and other sources.

• Interviews – In 2012, MOPAN complemented survey data and document review with consultations and interviews at the headquarters of multilateral organisations assessed.

In 2012 MOPAN also tested a new component to assess the results of multilateral organisations.[5]

As MOPAN’s methodology has changed significantly in the last three years, comparisons of this year’s assessments and previous assessments should take this into consideration.

The following is a summary of the MOPAN methodology in 2012.[6]

MOPAN 2012

In 2012, MOPAN assessed the effectiveness of six multilateral organisations: the African Development Bank (AfDB), GAVI Alliance (formerly the Global Alliance for Vaccines and Immunisation), the Joint United Nations Programme on HIV/AIDS (UNAIDS), the United Nations Development Programme (UNDP), the United Nations Children’s Fund (UNICEF), and the World Bank. The assessment was conducted in Cambodia, Democratic Republic of Congo, Ghana, Honduras, Philippines, Morocco, Niger, Nigeria, and Zimbabwe.[7]

The MOPAN Common Approach examines organisational systems, practices, and behaviours that MOPAN believes are important for aid effectiveness and that are likely to contribute to results at the country level. It groups these organisational capacities in four areas of performance: strategic management, operational management, relationship management, and knowledge management.

Key Performance Indicators and Micro-indicators – Within each performance area, organisational effectiveness is described using key performance indicators (KPIs) that are measured with a series of micro-indicators (MIs).

The micro-indicators are assessed using data from a survey and document review. The survey collects perception data from a variety of stakeholders (see Section 2.2) and the review of documents relies on a set of criteria that provide a basis for the assessment of each micro- indicator (see Section 2.3). However, not all micro-indicators are assessed by both the survey and the document review; consequently, some charts do not show survey scores and document review scores for each KPI or MI.

The African Development Bank was assessed using 19 KPIs and 72 MIs. The full list of MIs assessed is provided in Volume II, Appendix V (KPI and MI Data by Quadrant).

2.2 Survey

To gather diverse perspectives on the multilateral organisations being assessed, MOPAN generally seeks the perceptions of the following primary respondent groups:

• Donor Headquarters Oversight (HQ): Professional staff, working for a MOPAN donor government, who share responsibility for overseeing/observing a multilateral organisation at the institutional level. These respondents may be based at the permanent mission of the multilateral organisation or in the donor capital.

• Donor Country Office Oversight (CO): Individuals who work for a MOPAN donor government and are in a position that shares responsibility for overseeing/observing a multilateral organisation at the country level.

• Direct Partner/Client (DP):[8] Typically, individuals who work for a national partner organisation (government or civil society) in a developing country. Respondents are usually professional staff from organisations that receive some sort of direct transfer from the multilateral organisation or that have direct interaction with it at country level (this could take the form of financial assistance, technical assistance, policy advice, equipment, supplies, etc.). The definition of “direct partner” varies according to the context of each organisation assessed. In some cases, direct partners include staff members from international agencies that are implementing projects in conjunction with the multilateral organisation being reviewed.

• Other: For some organisations, other respondent categories are also used, such as peer organisations, co-sponsoring agencies, and/or recipient/host governments.

MOPAN donor respondents are chosen by MOPAN member countries. The other respondents are identified by the multilateral organisation being assessed.

The survey is customised for each organisation assessed and can be completed online in English, French, or Spanish or offline (paper, email, or interview) in these same languages. See Volume II (Appendix II) for the survey. Individual responses to the survey are confidential to the independent consultants managing the online survey or collecting data offline in the field.

Respondent Ratings – Survey respondents are presented with statements describing an organisational practice, system, or behaviour and asked to rate the organisation’s performance on a scale of 1 to 6 as shown below.

Figure 2.1 Respondent Rating Scale

In some cases, not all survey questions are answered, either because: 1) the individual chose not to answer, or 2) the question was not asked of that individual. In these cases, mean scores are calculated using the actual number of people responding to the question. As noted in the methodology (Volume II, Appendix I), ‘don’t know’ survey responses are not factored into the calculation of mean scores. However, when the proportion of respondents answering ‘don’t know’ is considered notable for a micro-indicator, this is indicated in the report. The responses of various categories of respondents on the six choices, plus ‘don’t know’ are summarised across all survey questions in Figure 2.2.

Figure 2.2 The African Development Bank - Distribution of Responses (n=152) on all Questions Related to Micro-Indicators

While there were responses in all six possible choices, relatively few responses overall were at the ‘weak’ end of the scale. Over one-third of responses provided by all surveyed groups fell under the adequate range although most responses ranged from adequate to strong. More than a quarter of the responses from MOPAN donors in-country were ‘don’t know’. This implies that, as observed in the 2009 MOPAN survey, donors in-country may have more limited familiarity with the Bank’s operations at the country level.

Survey Response Rate

MOPAN aims to achieve a 70 per cent response rate from donors at headquarters and a 50 per cent response rate among the population of respondents in each of the survey countries (i.e., donors in-country and clients). The number of respondents targeted in each category (i.e., the total population) and the actual response rates are presented in Figure 2.3 below. Response rates of all categories of respondents exceeded the 50 per cent target rate. While there are variations in the response rates by category and location of respondents, the African Development Bank survey results reflect the views of 152 respondents.

Figure 2.3 Number of Survey Respondents for the AfDB by Country and Respondent Group

Actual Number of Respondents (Total Population)

Converting Individual Scores to Group Ratings

As noted above, individuals respond to survey questions on a six-point scale where a rating of “1” is considered a judgment of “very weak” up to a rating of “6” intended to represent a judgment of “very strong.” A mean score is calculated for each respondent group (e.g., donors at HQ). Since mean scores are not necessarily whole numbers (from 1 to 6) MOPAN assigns numerical ranges and descriptive ratings for each range (from very weak to very strong) as shown below.

Figure 2.4 MOPAN Ranges and Descriptions

The ranges are represented to two decimal places, which is simply the result of a mathematical calculation and should not be interpreted as representing a high degree of precision. The ratings applied to the various KPIs should be viewed as indicative judgments rather than precise measurements.

Data Analysis

First level survey data analysis includes calculations of mean scores, medians, standard deviations, frequencies, (including analysis of ‘don’t know’ and missing responses), as well as content analysis of open-ended questions. The ‘don’t know’ responses are removed from the calculation of mean scores, but the proportion of respondents choosing ‘don’t know’ is retained as potentially useful data.

A weighting scheme is applied to ensure that no single respondent group or country is under- represented in the analysis. The weighting is intended to correct for discrepancies/variation in: the number of individuals in each respondent group, the number of countries where the survey took place, the numbers of donors in-country, direct partners, and other respondent groups within each country where the survey took place. Weighted figures are carefully reviewed and analysed before inclusion in the multilateral organisation reports.

Second level analysis examines differences in the responses among categories of respondents and other variables. When significant differences are found, these are noted in the report.[9]

For a full description of survey data analysis see Volume II, Appendix I.

2.3 Document Review

The document review considers three types of documents: multilateral organisation documents, identified with the help of the organisation; internal and external reviews of the organisation’s performance, found on the organisation’s web site or provided by the organisation; external assessments such as the Survey on Monitoring the Paris Declaration, the Common Performance Assessment (COMPAS) report, and previous MOPAN surveys

Ratings for key performance indicators (KPIs) are based on the ratings for the component micro-indicators in each KPI. For each micro-indicator, a set of criteria are established which, taken together, are thought to represent good practice in that area. The criteria are based on existing standards and guidelines (for example, UNEG or OECD-DAC guidelines), on MOPAN identification of key aspects to consider, and on the input of subject-matter specialists. The rating on any micro-indicator depends on the number of criteria met by the organisation. In cases where the micro-indicator ratings for one KPI are highly divergent, this is noted in the report.

While the document review assesses most micro-indicators, it does not assign a rating to all of them (when criteria have not been established). Consequently, some charts do not show document review scores for each KPI or MI. Documents are also used to aid in the understanding of the context in which the multilateral organisations work.

The document review and survey use the same list of micro-indicators, but some questions in the document review are worded differently from those in the survey. The document review and survey also use the same rating scale, but scores are presented separately on each chart in the report to show their degree of convergence or divergence.

2.4 Interviews

As of 2012, interviews are conducted at the headquarters of multilateral organisations with individuals who are knowledgeable in areas that relate to the MOPAN assessment.

Interviewees are asked to provide knowledge, insight, and contextual information that could assist the MOPAN Assessment Team in analysing document review data, and to identify other relevant documents for the Assessment Team to consider. This helps ensure that the Assessment Team has all the appropriate and necessary documents, enhances the Team’s ability to triangulate data from various sources, and assists the Assessment Team in the analysis of the key performance indicators by providing contextual information.

Interviews are conducted with a small number of staff who work in the primary units that relate to areas of the MOPAN assessment. Interviewees are identified by the multilateral organisation in conjunction with the Assessment Team and MOPAN. An interview guide is prepared and interviewees are advised of the content areas beforehand.

Data gathered during interviews is used to understand the context in which the agency is working, as well as how decisions are made. In the event that survey data present a picture that is very different from the document review, information from interviews can help clarify how the multilateral organisation approached a certain issue.

2.5 Basis for Judgment

From 2003 to 2009, the basis for judgment in MOPAN assessments was the perceptions of survey respondents. With the introduction of the document review in 2010 and interviews in 2012, judgments now draw on a variety of sources that can be compared and triangulated.

To the extent possible, the assessment standards and criteria are tailored to reflect the nature and operating environment of the multilateral organisations under review.

The MOPAN approach uses multiple data sources and data collection methods to validate findings. This helps eliminate bias and detect errors or anomalies.

The MOPAN reports gain trustworthiness through the multiple reviews and validation processes that are carried out by members of the network and by the multilateral organisations themselves.

2.6 Reporting

Institutional Reports

Individual institutional reports are produced for each multilateral organisation assessed. The results of the document review are presented alongside the survey results and discussed in light of the perception-based scores and interviews in order to further substantiate and contextualise the overall findings. For those agencies that were evaluated in 2009, a brief analysis of trends is included.

Country Data Summaries

A summary of survey results is produced for each multilateral organisation in each of the countries surveyed where sufficient survey data exists. Country Data Summaries provide feedback to those who participated in the MOPAN assessment and provide input for a dialogue process. They are not published and are shared only with individuals who attend the country workshop on the MOPAN assessment findings, which takes place in the first quarter of the year following the assessment.

2.7 Strengths and Limitations of Methodology

MOPAN continues to improve methodology based on the experience of each year of implementation. The following strengths and limitations should be considered when reading MOPAN’s report on the AfDB.

Strengths

• The MOPAN Common Approach is based on the core elements of existing bilateral assessment tools. In the long term, the intent is to replace or reduce the need for other assessment approaches by bilateral donors.

• It seeks perceptual information from different perspectives: MOPAN donors (at headquarters and in-country), direct partners/clients of multilateral organisations, peer organisations, and other relevant stakeholders. This is in line with donor commitments to the Paris Declaration on Aid Effectiveness and the Accra Agenda for Action regarding harmonisation, partner voice, and mutual accountability.

• It complements perceptual data with document review and interviews, thus using multiple sources of data. This should enhance the analysis, provide a basis for discussion of agency effectiveness, and increase the validity of the assessment through triangulation of data.

• The reports undergo a validation process, including multiple reviews by MOPAN members and review by the multilateral organisation being assessed.

• MOPAN strives for consistency across its survey questions and document review for each of the multilateral organisations, while allowing for customisation to account for differences between types of multilateral organisations.

Limitations

MOPAN Framework

• The countries are selected based on established MOPAN criteria and comprise only a small proportion of each institution’s operations, thus limiting broader generalisations.

• The Common Approach indicators were designed for multilateral organisations that have operations in the field. For organisations that have limited field presence or that have regional structures in addition to headquarters and country operations, there have been some modifications made in the data collection method and there will be a need for greater nuance in the analysis of the data.

Data sources

• The MOPAN Common Approach asks MOPAN members and the organisations assessed to select the most appropriate individuals to complete the survey. While MOPAN sometimes discusses the selection with the organisation being assessed, it has no means of determining whether the most knowledgeable and qualified individuals are those that complete the survey.

• The document review component works within the confines of an organisation’s disclosure policy. In some cases, low document review ratings may be due to unavailability of organisational documents that meet the MOPAN criteria (some of which require a sample of a type of document, such as country plans, or require certain aspects to be documented explicitly). When information is insufficient to make a rating, this is noted in the charts.

Data Collection Instruments

• Three issues potentially affect survey responses. First, the survey instrument is long and a fatigue factor may affect responses and rates of response. Second, respondents may not have the knowledge to respond to all the questions (e.g., survey questions referring to internal operations of the organisation, such as financial accountability and delegation of decision-making, seem difficult for many respondents, who frequently answer ‘don’t know.’) Third, a large number of ‘don’t know’ responses may imply that respondents did not understand certain questions.

• The rating choices provided in the MOPAN survey may not be used consistently by all respondents, especially across the many cultures involved in the MOPAN assessment. One potential limitation is ‘central tendency bias’ (i.e., a tendency in respondents to avoid extremes on a scale). Cultural differences may also contribute to this bias as respondents in some cultures may be unwilling to criticise or too eager to praise.

• Because one of MOPAN’s intentions is to merge previously existing assessment tools into one, and to forestall the development of others, the survey instrument remains quite long.

Data Analysis

• While the document review can serve to evaluate the contents of a document, it cannot assess the extent to which the spirit of that document has been implemented within the organisation (unless implementation is documented elsewhere).

• Mean scores are used in the MOPAN reports to provide central tendency values of the survey results. The mean has the advantage of being the most commonly understood measure of central tendency, however, there is a disadvantage in using the mean because of its sensitivity to extreme scores (outliers), particularly when population samples are small. The assessment team reviewed the median and standard deviations in analysing the survey results. Volume II, Appendix V provides the standard deviations for each survey question.

Basis for Judgment

• Although MOPAN uses recognised standards and criteria for what constitutes good practice for a multilateral organisation, such criteria do not exist for all MOPAN indicators. As a result, many of the criteria used in reviewing document content were developed by MOPAN in the course of the assessment process. The criteria are a work in progress and should not be considered definitive standards.

• The Common Approach assessment produces numerical scores or ratings that appear to have a high degree of precision, yet can only provide general indications of how an organisation is doing and a basis for discussion among MOPAN members, the multilateral organisation, and other stakeholders, including direct partners.

Despite some limitations, the Assessment Team believes that the MOPAN reports generally provide a reasonable picture of systems associated with the organisational effectiveness of multilateral organisations.

2.8 Testing a New Component in 2012: Assessing Development Results

Overview

Until 2012, MOPAN assessments focused on the organisational effectiveness of multilateral organisations by examining organisational practices, systems, and behaviours that MOPAN believes are important for managing to achieve development results. In 2012, MOPAN tested a component to assess how multilateral organisations report on development results achieved – with four of the six organisations assessed: AfDB, UNDP, UNICEF, and the World Bank.[10]

Sub-Components

• KPI A – Evidence of extent of progress towards organisation-wide outcomes[11]

• KPI B – Evidence of extent of contributions to country-level goals and priorities

• KPI C – Evidence of extent of contributions to relevant MDGs

• KPI D – Relevance of objectives and programme of work to stakeholders.

The assessments at the institutional/organisational level (KPI A) and at the country level (KPI B) are separated due to differences in focus, scope and reporting on results at these two levels.

KPIs B, C, and D all focus on the country level and are applied in five countries. Each multilateral organisation is asked to identify the countries where they are likely to have the best data on results.

KPI D assesses relevance as a measure of the extent to which surveyed stakeholders consider the multilateral organisation supports country priorities and meets the changing needs of direct partners and the target population.

Methodology

Various types of qualitative and quantitative data are sought to answer a set of questions about the multilateral organisation’s performance in the indicators noted above. Data are collected using three different methods: document review, stakeholder survey, and interviews with staff at HQ and, if feasible, in country offices.

• The document review draws largely on organisational performance reports and country level performance reports and evaluations.

• The stakeholder survey asks donor and direct partner respondents to rate the organisation’s achievement of planned results and the relevance of its activities at the country level. The questions are tailored, as required, to each organisation.

• Interviews are conducted to identify reliable data, identify key staff to consult in country offices, and to help contextualise the analysis of results.

Analysis of institutional level data focuses on the extent to which planned results from the strategic period were achieved, based largely on performance reports and thematic evaluations. Analysis of country level data focuses on the organisation’s contribution to results in the sample of countries selected for the MOPAN assessment.

The judgment of an organisation’s performance on each KPI draws on a set of criteria. The Assessment Team uses a “best fit approach,” a criteria-referenced basis for judgment that is suitable when criteria are multi-dimensional, there is a mix of both qualitative and quantitative data, and it is not possible to calculate a simple sum of the data points.[12]

The ratings reflect the Assessment Team’s judgment and analysis of data from all sources. The ratings are qualitative in nature and defined according to a 4-point scale – strong, adequate, inadequate, or weak. As in the 6-point scale used in the survey, a rating of “strong” signals that the organisation is approaching best practice, while a rating of “weak” signals that the organisation still has important limitations in demonstrating progress towards its stated results, and particularly its contributions to development outcomes.

Limitations to the Methodology

The methodology was designed to draw on the evidence of results achieved, as presented in the reports of a multilateral organisation. However, there is a critical difference between assessing the actual results achieved on the ground and assessing the evidence of results in the organisation’s reports to its key stakeholders. This is a limitation that is inherent in the current approach.

MOPAN will review the experience with this pilot component and make adjustments in the methodology, as required.

3. Main Findings: Practices and Systems that Enable the Achievement of Results

3.1 Introduction

This chapter presents the findings of the 2012 MOPAN assessment of the practices and systems of AfDB. Findings are based on respondent survey data and document review.

• Section 3.2 presents overall ratings on the performance of the AfDB and summarises respondent views on its primary strengths and areas for improvement;

• Section 3.3 provides findings on each of the four areas of performance (strategic, operational, relationship, and knowledge management).

3.2 Overall Ratings

This section provides a summary of overall ratings. It includes: survey respondent ratings of AfDB’s overall organisational effectiveness, survey respondent views on AfDB’s strengths and areas for improvement, and survey and document review ratings for all key performance indicators.

Survey ratings of the AfDB’s organisational effectiveness

MOPAN has defined “organisational effectiveness” as the extent to which a multilateral organisation is organised to support direct partners/clients in producing and delivering expected development results. Respondents were asked the question: “How would you rate the overall organisational effectiveness of the AfDB?” As shown in Figure 3.1, AfDB’s clients gave the largest proportion of ratings on the high end of the scale.

Figure 3.1 Overall Ratings of the AfDB’s Organisational Effectiveness by Respondent Group 

Respondents’ Views on AfDB’s Strengths and Areas for Improvement

The survey included two open-ended questions that asked respondents to identify AfDB’s greatest strengths and areas of improvement. All 152 respondents answered both questions.[13] Their comments are summarised below with an analysis of the level of homogeneity among respondent groups as well as areas where the respondent groups felt differently about certain issues.

Overall, survey respondents highlighted the Bank’s ability to be the advocate for Africa, and considered the Bank’s greatest strength to be its knowledge of the continent and familiarity with African issues. Other areas of strength were the Bank’s good standing as a financial institution and its proximity to clients, which allowed for better dialogue and more effective collaboration.

All groups surveyed agreed that the bank's greatest strength lies in the nature of the organisation itself, i.e., a regional bank led largely by individuals who are familiar with and sensible to issues particular to the African continent and to Africans. Survey respondents also commented positively upon the Bank's decentralisation efforts, highlighting its increasing field presence as key to facilitating better dialogue and enhanced support to Regional Member Countries (RMCs).

Clients and donors in country offices emphasised the Bank’s talented pool of experts and sound financing capacities, particularly for projects deemed “difficult”. Clients also noted the Bank’s efforts to develop capacities at the country level by providing strong technical support to RMCs.

Donors at headquarters highlighted the Bank’s legitimacy, seeing it as a trusted and credible institution with a strong strategic focus.

Respondents underlined the need for the AfDB to improve the effectiveness and efficiency of its general procedures on the ground. They also highlighted the need for the Bank to speed up its decentralisation processes and to improve the management of its human resources.

Close to half of the respondents who commented on AfDB’s areas for improvement (45 per cent) felt that the Bank’s heavy bureaucratic measures constitute a major hindrance to the effectiveness and efficiency of its operations. Respondents highlighted delays in approval processes, which negatively affect disbursement and procurement procedures and thus affect the Bank’s operations at the implementation level. More particularly, more than half of the clients who commented on this question (58 per cent) emphasised the need for more speedy disbursement and procurement procedures.

Many survey respondents (27 per cent) also suggested that there is a need to improve the management of the Bank’s human resources overall. Respondents noted the need for increased HR capacity at country and institutional levels, for a more effective mix of skills amongst staff, for continuous in-house training, and for better managerial skills from management.

Clients and donors in-country emphasised the need for increased country presence and greater delegation of authority for decision- making to country levels. Many donors in-country (32 per cent) further commented on the need for greater collaboration and better co- ordination with other donors and actors at the country level.

Overall Ratings of Key Performance Indicators

Figure 3.2 below shows scores from the document review and the survey on key performance indicators (KPIs) in the MOPAN 2012 assessment of the AfDB. The white bar presents the survey score, while the black square presents the document review score. For example, on the first indicator, “providing direction for results”, the AfDB received a score of 4.07 (adequate) in the survey and a score of 5 (strong) in the document review.

In the overall ratings from the survey and document review, the AfDB was seen to perform adequately or better on the majority of key performance indicators.

The AfDB received scores of adequate on 18 of the 19 KPIs assessed in the survey and a score of inadequate in its delegation of decision-making authority.

The AfDB received scores of adequate or better on 13 out of 17 KPIs assessed in the document review.[14]

The survey and document review ratings differed on seven KPIs – three of which were rated lower by the document review than by survey respondents, and the opposite for the remaining four. The reasons for these differences are discussed in the following sections.

Figure 3.2 Overall Ratings on Key Performance Indicators (mean scores, all respondents and document review ratings)

3.3 AfDB’s Performance in Strategic, Operational, Relationship, and Knowledge Management

3.3.1 Overview

This section presents the results of the 2012 Common Approach assessment of the AfDB in four performance areas (quadrants): Strategic, Operational, Relationship, and Knowledge Management.

The following sections (3.3.2 to 3.3.5) provide the overall survey and document review ratings for the KPIs in each quadrant, the mean scores by respondent group, and findings based on an analysis of survey and document review ratings in each quadrant.

When there were notably[15] divergent ratings between survey respondent groups or between the survey results and document review ratings, these are noted and the information gleaned from interviews with staff is integrated when it has a bearing on the analysis. The survey data for each KPI and MI by quadrant are presented in Volume II, Appendix V. The document review ratings are presented in Volume II, Appendix VI.

3.3.2 Strategic Management

The AfDB has improved its management for results at a strategic level, with tools such as the “One Bank” Results Measurement Framework (RMF) and Annual Development Effectiveness Reviews (ADERs). However, the institutionalisation of a results culture remains a work in progress at the Bank.

Figure 3.3 shows the overall survey and document review ratings for the four KPIs in the strategic management quadrant.

Overall, the AfDB was rated in the mid-range of the adequate band by both survey respondents and document review on most aspects of strategic management. The Bank was rated strong for the clarity of its mandate, and adequate overall for its policy on results management. The document review acknowledged the Bank’s increased focus on results in recent years, but suggested that there is room for improvement in the quality of its corporate and country-level results frameworks. Overall, the Bank received adequate ratings for its integration of cross- cutting themes, but donors at headquarters and the document review noted limitations in its strategies for mainstreaming in some areas (for example, in mainstreaming gender equality).

There are indications that the AfDB may see improvements in these areas in the coming years with the application of the new reporting tools and guidelines developed by the organisation.

Figure 3.3 Quadrant I: Strategic Management, Survey and Document Review Ratings

Figure 3.4 shows the mean scores for the four KPIs for all survey respondents, and by category of respondent.

Figure 3.4 Quadrant I: Strategic Management, Mean Scores by Respondent Group

KPI 1: Providing Direction for Results

Finding 1: Overall, survey respondents considered the AfDB adequate in terms of being results-oriented and client-focused. It was rated as strong by the document review in making key documents available, which underlines the Bank’s commitment to transparency.

Survey respondents rated the AfDB adequate in its ability to instil a culture that is results- oriented and client-focused and also considered that the Bank’s senior management demonstrates adequate leadership in results management. The document review rated the Bank strong on making key documents available to the public.

Figure 3.5 KPI 1: Providing Direction for Results, Ratings of Micro-Indicators

MI 1.1 – Value system supports results-orientation and client focus

In this MI, which was assessed by survey only, all respondent groups were asked two questions: whether the Bank’s institutional culture reinforces a focus on results, and if it is client focused. The majority of respondents across respondent groups (68 per cent) rated the AfDB as adequate or above on both questions.

However, the views of donors at headquarters were split on the two questions related to this MI. On the first question, 45 per cent rated the Bank adequate or above in reinforcing a culture of results, and 55 per cent rated it inadequate or below. On the second question, the majority (80 per cent) noted AfDB’s emphasis on clients’ needs and rated the organisation adequate and above.

MI 1.2 – Leadership on results management

This MI was assessed by survey of only MOPAN donors at headquarters. They were asked whether AfDB’s senior management shows leadership in results management. More than half (60 per cent) rated the AfDB’s performance as adequate or above.

MI 1.3 – Key documents available to the public

All three respondent groups were asked about this MI. The majority of survey respondents (73 per cent) rated the AfDB adequate or above when asked whether the Bank makes key documents available to the public.

The document review rated the AfDB as strong in this area. Most of the Bank’s key documents (financial statements, strategies, evaluations, etc.) are available on the organisation’s website in the continent’s major languages (i.e., French and English). The document review further highlighted the 2012 review of the Bank’s Policy on Disclosure and Access to Information, which illustrates the Bank’s continuous efforts to reduce the restrictions on types of information made available to its major stakeholders. The revised policy is expected to become effective in February 2013.

KPI 2: Corporate Focus on Results

Finding 2: Donors at headquarters rated the AfDB strong for the clarity of its mandate, and adequate overall for its policy on results management. The document review acknowledged the Bank’s increased focus on results in recent years, but suggested that there is room for improvement in the quality of its results frameworks.

All five MIs in this KPI were assessed by the document review and two were assessed in the survey by donors at headquarters only.

Donors at headquarters provided mixed ratings of the Bank’s performance. The Bank was perceived as strong in having an organisational strategy based on a clear mandate, but adequate in ensuring the application of results management across the organisation.

The document review rated the Bank’s performance adequate overall and noted the ambitious reform agenda that the Bank has been implementing for the past years. AfDB’s focus on results has been strengthened since 2008 with the establishment of the Quality Assurance and Results Department (ORQR), which is responsible for managing results reporting and implementing the Bank’s results agenda. Part of the Bank’s reform process includes the revamping of its results measurement frameworks (RMFs), which since 2010 have been consolidated into one results framework: the “One Bank” RMF. That process has also led to the introduction of the Annual Development Effectiveness Review (ADER), an organisation-wide report on development effectiveness.

Figure 3.6 KPI 2: Corporate Focus on Results, Ratings of Micro-Indicators

MI 2.1 – Organisational strategy based on clear mandate

Donors at headquarters were asked two questions on this MI: whether the AfDB has a clear mandate, and whether AfDB’s organisation-wide strategy is aligned with the mandate. Overall, more than half of the respondents (67 per cent) rated the Bank as strong or above on these questions.

The document review confirmed this perception, and provided a rating of very strong. Explicit links to its mandate can be found in the Bank’s objectives as identified in The Agreement Establishing the African Development Bank[16] and the strategic priorities identified in the current Mid-Term Strategy 2008-2012 (MTS). The Articles of Agreement of the AfDB have been revised multiple times since its establishment in 1963, thus ensuring continuing relevance of the Bank’s work and priorities.

MI 2.2 – Organisational policy on results management

In the survey, donors at headquarters were asked if the AfDB ensures the application of results management across the organisation. The respondent ratings were almost evenly split, with almost half rating it as adequate and above and the other half rating it as inadequate or below.

The document review provided an adequate rating but noted that the AfDB is currently undergoing several institutional reforms, some of which include revamping the organisation’s practices for planning, monitoring, and reporting on corporate results. In the past two years, the Bank has introduced several results monitoring and reporting tools such as the “One Bank” Results Measurement Framework which replaces several previous individual results frameworks for each of the three financing entities of the Bank. The new results framework defines organisation-wide results indicators and guides the organisation’s reporting on results.

The Bank’s Capacity Development Strategy also puts emphasis on capacity development of the Bank’s staff at the country level to ensure proper development of country planning and reporting tools such as the Results-Based Country Strategy Papers (RBCSPs). Finally, the recently introduced Annual Development Effectiveness Review (ADER) puts these tools and intentions into practice at the organisation-wide level. However, since most of these practices are still new, the Bank’s results agenda is still a work in progress, which is also acknowledged by the Bank.[17]

MI 2.3 – Plans and strategies contain results frameworks

This MI was assessed by document review only, which found that the One Bank Results Measurement Framework includes adequate results frameworks.

The One Bank RMF is based on a set of Core Sector Indicators (CSIs) organised in four levels. Level 1 contains indicators that measure Africa’s overall development progress in eight areas; Level 2 indicators present the aggregate outputs of the Bank’s operations in similar areas, showing how the Bank has contributed to Africa’s development; Level 3 indicators help assess how well the Bank manages its portfolio of operations, and finally, Level 4 indicators describe how well the Bank manages the organisation overall.[18]

The One Bank framework includes results indicators for both management and development results. While all results indicators are appropriate to their level, the framework does not include actual results statements under Level 2, but only core sector indicators at the output and intermediate outcome level.

MI 2.4 – Results frameworks link outputs to final outcomes/impacts

This MI was assessed by document review only, which rated the AfDB as inadequate. The Bank’s RMF and reporting on results in ADER mainly focus on results achieved at the output level (apart from results under Level 1 that cannot be attributed to AfDB’s interventions). From this perspective, there seems to be a big leap between the output statements reported under Level 2 (outputs achieved by AfDB-supported projects) and the regional development outcomes included in Level 1. Neither the results framework nor the MTS 2008-2012 provide a theory (or theories) of change that could explain the link between outputs achieved by projects supported by the Bank and higher level results recorded on the African continent.[19]

This limitation is acknowledged in the One Bank RMF where it is stated that while “the links between Levels 1 and 2 of the RMF can only be reported in specific country settings, we can illustrate the mechanisms by which investments in Bank priority areas contribute to country outcomes in general terms. Although many outcomes in Level 1 do not appear directly related to sectoral outputs in Level 2, AfDB operations contribute to them indirectly.” In the same paragraph, the Bank includes a number of illustrative examples of how Level 2 outputs can contribute to higher level development results. While these suggest a series of changes that are expected to take place, they do not elaborate on how changes will occur or on the assumptions underlying these changes. Although the document review rated the Bank as inadequate in establishing clear linkages between Level 2 outputs and Level 1 outcomes in the results framework, it is appreciated that this is a challenge that all MDBs, and not only the AfDB, are facing. In addition, according to information from the Bank, there is on-going work at the sector level that attempts to reduce this gap between outputs and outcomes. As an example, the Bank mentions the new Programme Based Operations (PBO) policy that includes a results framework with better links between output, outcome and impact level indicators.

MI 2.5 – Plans and strategies contain performance indicators

This MI was assessed by document review only, which rated the AfDB as adequate for including performance indicators in organisation-wide plans and strategies. The AfDB began using a standard set of Core Sector Indicators (CSIs) in 2010 to monitor its contribution to development results across priority sectors. Organisation-wide results from the One Bank RMF and ADER include performance indicators that are adequate, clear, and targeted. However, while the One Bank RMF consistently includes both output and outcome indicators, the ADER mainly reports on the output level (apart from reporting on Level 1 that is not directly linked to the Bank’s performance). In addition, as highlighted under MI 2.3, the Bank does not systematically report on all the indicators identified in the One Bank results framework.

KPI 3: Focus on Thematic Priorities

Finding 3: Survey respondents rated the Bank adequate in addressing cross-cutting priorities. The review of documents found that the AfDB has adequate policy frameworks and tools in place, but often lacks compliance mechanisms.

The assessment of this KPI focused on six thematic areas identified as key cross-cutting priorities by MOPAN and the AfDB: gender equality, environmental sustainability, climate change, private sector development, engagement in fragile states, and promotion of regional economic integration. All of these were assessed by both the document review and the survey.

Survey respondents rated the AfDB adequate overall in all thematic areas. A relatively high percentage of donors in-country (33 per cent) answered ‘don’t know’ to questions about the Bank’s performance in these areas. AfDB’s clients were positive about the Bank’s emphasis on engagement with fragile states, its promotion of regional integration, and its efforts to mainstream environmental sustainability in its operations. Donors at headquarters provided lower ratings on the Bank’s ability to mainstream gender equality into its operations.

The document review rated the AfDB as adequate in all cross-cutting areas and highlighted the need for enhanced compliance mechanisms and better monitoring of results. At the time of the assessment, two of the Bank’s thematic strategies (climate change and regional integration) had not yet been evaluated or reviewed.[20] The lack of systematic evaluations of the Bank’s cross-cutting strategies could prevent the identification/application of lessons learned.

Figure 3.7 KPI 3: Focus on Thematic Priorities, Ratings of Micro-Indicators

MI 3.1 – Gender equality

When asked whether the AfDB sufficiently mainstreams gender equality in its operations, respondents rated the Bank as adequate overall. However, there were statistically significant differences between the views of clients who rated the Bank adequate and those of donors at headquarters who perceived the organisation’s performance to be inadequate.

The documents reviewed highlighted that the AfDB has put further emphasis on gender equality in recent years. Indeed, its renewed commitment to gender mainstreaming has, in the past two years, resulted in numerous initiatives including: the integration of gender indicators in the Bank’s results measurement framework (RMF), the establishment of a gender division responsible for monitoring and reporting on gender results,[21] and the development of specific tools such as ‘country gender profiles’ and ‘gender checklists’ to inform the design processes of country strategies and help staff integrate gender issues into the Bank’s programming. According to Bank staff, since 2011, systematic capacity building sessions have been conducted for staff in Field Offices and at Headquarters to help disseminate and ensure proper use of these tools.

While the AfDB has made commendable efforts to strengthen its capacity for gender mainstreaming, a recent evaluation highlighted inconsistencies in the mainstreaming approach to date and recommended several areas for improvement, particularly the need for accountability and incentive mechanisms at managerial levels, systematic monitoring of gender results, increased financial resources and gender expertise at both institutional and country levels.[22] The evaluation highlighted the need for continuous “leadership and commitment” from the Bank’s senior management in this area. At the time of this assessment, the Bank was in the process of developing a new Gender Strategy aimed at institutionalising and better integrating gender equality in all of the Bank’s operations. According to the Bank, the 2013-2017 strategy (scheduled to be adopted this year) will focus on two specific implementation modalities: the establishment of a compliance mechanism and the optimisation of human and financial resources in this area.

MI 3.2 – Environment

Overall, survey respondents rated the AfDB as adequate in mainstreaming environment in its operations. However, there were significant statistical differences between the views of clients, who rated the Bank’s performance as strong, and those of donors in country offices who rated the organisation as adequate.

The document review rated the AfDB as adequate in mainstreaming environment in its operations. The Bank has developed several measures and guidance tools to enable effective environmental and social assessments in both the design and implementation stages of the Bank’s operations. According to the Bank’s Environmental and Social Assessment Procedures (ESAP), all potential Bank projects are subject to a Strategic Environmental and Social Assessment (SESA) or an Environmental and Social Impact Assessment (ESIA) depending on the type of project. The Bank has also developed guidance notes for clients to this effect. ESIA guidelines, for example, were designed to ensure that both environmental and social issues are mainstreamed in Bank projects throughout the project cycle.

Despite the guidance notes in place, an internal review indicated that there were inconsistencies in ensuring compliance with environmental and social safeguards during the implementation process of the Bank’s projects.[23] At the time of the assessment, the Bank was also reviewing its ESAP and ESIA guidelines. Furthermore, a recent review of the Bank’s draft Integrated Safeguard System indicated that the Integrated Safeguard Tracking System (ISTS) and several other procedures aimed at strengthening compliance mechanisms have not yet been fully integrated into the Bank’s operations.[24]

MI 3.3 – Climate change

When asked whether the AfDB has sufficient focus on climate change, the majority of respondents provided ratings of adequate or above (56 per cent). However, donors in country offices appeared to be less knowledgeable of the Bank’s performance in this area and 44 per cent answered ‘don’t know’ to the question about its performance in this area.

The Bank’s increased commitment to climate change has recently been demonstrated through its inclusion as one of the key cross-cutting priorities in the Medium-Term Strategy (MTS), the adoption of a Climate Risk Management and Adaptation Strategy (CRMA) in 2009, the establishment in 2010 of the Energy, Environment, and Climate Change department (ONEC), and the inclusion of key performance indicators related to climate change in the One Bank results framework.

A review of the Bank’s programme and budget documents indicates that the Bank has increased support to climate change initiatives in recent years. This is most clearly observed in the 2011-2013 and 2012-2014 programme and budget documents. While the 2011-2013 programme and budget document mostly put emphasis on providing financial support to the operationalisation of the newly established ONEC department,[25] the 2012-2014 programme and budget document reported on the need to fund activities related to building capacities both at headquarters and in countries.[26] It is our understanding that while the Bank has acquired the level of expertise necessary at the headquarters; some capacity is still missing at the country level.

The document review rated the Bank as adequate based on the evidence presented above, the review of the Bank’s environmental and social assessment procedures (ESAP) described in MI 3.2 above), and the fact that the Bank’s 2011-2015 Action Plan on Climate Change has not been implemented yet.

MI 3.4 – Private sector development

The majority of survey respondents (70 per cent) rated the AfDB as adequate or above on the extent to which it promotes private sector development.

The document review rated the Bank as adequate on this indicator. Private sector development is at the heart of the Bank’s strategy and is recognised as one of the Bank’s core priority areas in the 2008-2012 MTS. There is evidence of steady increase in AfDB’s volume of financing for private sector operations. In the period 1996–2006, the average volume of new private sector operations rarely exceeded UA 200 million. Over the 3-year period 2008–10, the total volume of approved new operations (the financing objective) was expected to reach a cumulative UA 2.55 billion).[27] However, the 2010 Mid-Term Review of the 2008-2011 Business Plan for Private Sector Operations, the noted that: “although PSD is widely recognised as a prerequisite for economic growth and poverty alleviation, it is yet to be fully integrated into the Bank’s institutional culture [and that] roles, functions, and business processes need to be further developed in order to deliver the Bank-wide corporate mandate for PSD.” [28]

MI 3.5 – Fragile states

The majority of survey respondents (61 per cent) rated the AfDB as adequate or above for the extent to which it engages with fragile states. Donors in-country were generally less familiar with the Bank’s work in this area as 47 per cent answered ‘don’t know’ on this MI.

The Bank’s commitment to supporting countries emerging from conflict is clearly articulated in the 2008-2012 MTS and was further confirmed with the adoption of a Strategy for Enhanced Engagement in Fragile States (SEEFS). A 2010 review of the Proposed Adjustments to the Enhanced Approach to Fragile States indicated that the Bank had made some progress in the implementation of SEEFS.[29] Among the successes highlighted was the full operationalisation and staffing of the Fragile State Unit (FSU) which was created in 2008. Since the creation of this Unit and of the Fragile State Facility (FSF), the Bank has broadened its presence on the continent and this has ultimately contributed to facilitating policy dialogue and coordination with implementing partners in fragile states.

A 2012 evaluation of the Bank’s assistance to fragile states, however, also noted several areas for improvement; particularly the need for improved measurement and reporting of results achieved, better dissemination and use of lessons learned, and better coordination with other donors on the ground.[30] It also highlighted the lack of realistic implementation plans and support systems to implement the Strategy’s vision of a deeper and more considered AfDB-wide engagement in fragile states. The AfDB therefore received an adequate rating by the document review although the Bank’s clear commitment and efforts in this area were acknowledged.[31]

MI 3.6 – Regional economic integration

When asked whether the AfDB sufficiently promotes regional economic integration, the majority of survey respondents (72 per cent) rated the Bank as adequate or above.

The document review noted the efforts made by the Bank to promote regional integration in collaboration with other key African actors such as the African Union (AU), the New Partnership for Africa’s Development (NEPAD), and Regional Economic Communities (RECs). The Bank has also provided advisory and technical support to Regional Member Countries through the operationalisation of the Short-Term Action Plan (STAP) developed by NEPAD as well as the Program of Infrastructure Development of Africa (PIDA) lead by the AU and NEPAD. While regional integration has been one of the AfDB’s key priorities since its inception,[32] the Bank’s focus in this area was further sealed with the adoption of the Economic Cooperation and Regional Integration Policy (2000), the integration of regional integration as a key priority in the 2003-2007 MTS, and the adoption of a Regional Integration Strategy (2008) which was accompanied by an operational framework for regional integration. The Bank has a policy framework in place, but its regional integration strategy and indicators are now under review. In addition, reporting on results, which is done through Regional Integration Strategy Papers (RISPs), was only introduced in 2011.

KPI 4: Country Focus on Results

Finding 4: The AfDB was rated strong by survey respondents and very strong by the document review for ensuring consistency between national development plans and Country Strategy Papers (CSPs). However, the document review noted some inconsistencies in the quality of the CSP results frameworks.

AfDB clients and MOPAN donors in-country were surveyed for all MIs in this KPI. They rated the Bank as strong in linking results at project, programme, sector and country levels, and in ensuring that its country strategies are aligned with Poverty Reduction Strategic Plans (PRSPs) or other national development plans.

The document review was based on an assessment of the Bank’s country level documents for the Democratic Republic of Congo (DRC), Ghana, Morocco, Niger and Nigeria. All the CSPs analysed included results frameworks that described expected results at the country level. Most of the reviewed results frameworks, however, lacked consistency in the formulation of results at outcome and output levels. However, a review of the results frameworks from the more recent generation of CSPs indicated a positive trend in their quality and consistency.

Figure 3.8 KPI 4: Country Focus on Results, Ratings of Micro-Indicators

MI 4.1 – Frameworks link results at project, programme, sector and country levels

MOPAN donors in-country and clients were asked whether AfDB’s Country Strategy Papers (CSPs) link results from project, sector and country levels. The majority (79 per cent) rated the AfDB as adequate or above.

In contrast, the document review rated the AfDB as inadequate based on the review of CSPs developed for the Democratic Republic of Congo (DRC), Ghana, Morocco, Niger and Nigeria. This relatively low rating was due to the varying quality and format of the results frameworks and their inconsistent use of results statements and indicators. While most of the CSPs link expected outputs and outcomes to results at the country level, not all statements of results are appropriate to their results level and linkages between output and outcomes are often unclear. Several CSPs include output statements that are in fact outcome statements and vice versa.

That being said, this rating is based on a small sample of CSPs that were developed before the Readiness Review process was in place.[33] The quality of the Bank’s country level results frameworks has improved in recent years as a consequence of the Bank’s increased focus on results, the establishment of the Quality Assurance and Results Department (ORQR), and the introduction in 2011 of a Readiness Review process to improve the quality-at-entry of CSPs.

MI 4.2 – Frameworks include indicators at project, programme, sector and country levels

MOPAN donors in-country and clients were asked whether AfDB’s CSPs include indicators at all levels (country, sector, and project/programme). The majority of respondents (71 per cent) rated this MI as adequate or above, but 30 per cent of donors in-country answered ‘don’t know’.

The document review rated the AfDB as inadequate on this MI. The assessment included CSPs from different periods; the oldest dating from 2005 and the newest from 2012. While the results frameworks show signs of improvement over time, there are inconsistencies in the use of indicators. Most of the indicators reviewed are relevant, but many are difficult to monitor due to lack of clear targets, and very few provide intended data sources. In some frameworks, the results statements are actually indicators, while other CSPs distinguish between indicators and results statements. Most of the results frameworks assessed did not use baselines systematically and in some cases, output and outcome targets are also unclear. The use of the terms ‘outputs’ and ‘outcomes’ is also inconsistent and some of the results frameworks reviewed label expected results as ‘outputs’ when they are actually long-term development outcomes such as “real growth in GDP” or “reduced inflation rates”.

MI 4.3 – Expected results consistent with PRSP or national plan

MOPAN donors in-country and clients were asked whether AfDB’s CSPs contain statements of expected results consistent with national development strategies. The majority of respondents (74 per cent) rated this MI as adequate or above.

The document review provided a rating of very strong on this indicator. In the CSPs reviewed, there was a clear and explicit link between AfDB’s expected results and those identified in national Poverty Reduction Strategic Plans (PRSPs) or equivalent. In all CSPs, the Bank’s country results frameworks were directly linked to strategic goals of national governments (typically by including a column with national goals).

MI 4.4 – Expected results developed in consultation with clients/beneficiaries

This MI was assessed by survey only. MOPAN donors in-country and clients were asked whether the AfDB consults with clients to develop its expected results. The majority of respondents (68 per cent) rated this MI as adequate or above. However, clients were more positive than donors in-country and provided a rating of strong. This difference is statistically significant.

MI 4.5 – Results for cross-cutting priorities included in country level frameworks

MOPAN donors in-country and clients were asked whether AfDB’s country strategy papers include results for the thematic areas it defines as cross-cutting priorities. The majority of respondents (64 per cent) rated this MI as adequate or above.

The document review also rated the Bank adequate in this area. While none of the country strategy papers reviewed included all six cross-cutting priorities identified by the AfDB and MOPAN (gender equality, environmental sustainability, climate change, private sector development, engagement in fragile states, and promotion of regional economic integration), all CSPs reviewed included references to one or more cross-cutting priorities. Regional integration and private sector development were well-represented, although most of the CSPs identified these as core priorities rather than cross-cutting priorities. Only two of the five CSPs reviewed explicitly mentioned gender as a cross-cutting priority, which may reflect the limitations in AfDB’s approach to mainstreaming gender equality noted in the discussion under MI 3.1.

3.3.3 Operational Management

Survey respondents perceived the Bank’s performance in Operational Management to be adequate overall. The greatest room for improvement for the Bank was found to be in the management of its human resources and delegation of decision-making authority.

Figure 3.9 below shows overall survey and document review ratings for the seven KPIs in the operational management quadrant. Figure 3.10 shows mean scores for the seven KPIs for all survey respondents, and by respondent groups.

The AfDB received survey ratings of adequate for all KPIs assessed under operational management with the exception of its decentralisation processes, which were found to be inadequate. The document review on the other hand gave the Bank mixed ratings that varied from inadequate to strong. While the Bank was rated strong for its resource allocation and financial accountability policies and processes, it was found to be inadequate in its management of human resources.

The Bank’s degree of decentralisation (delegation of authority) was found to be the greatest area of concern for survey respondents, and the document review similarly highlighted it. Both decentralisation and human resource management have been the focus of the Bank’s reform efforts. The Bank is in the process of establishing greater presence in Regional Member Countries. As part of this decentralisation, the Bank is revising its current Delegation of Authority Matrix. In the area of human resource management, the Bank has implemented a set of practices related to performance management, but some of these are still not fully institutionalised.

Figure 3.9 Quadrant II: Operational Management, Survey and Document Review Ratings

Figure 3.10 shows the mean scores for the KPIs for all survey respondents, and by respondent groups.

Figure 3.10 Quadrant II: Operational Management, Mean Scores by Respondent Group

KPI 5: Resource Allocation Decisions

Finding 5: The AfDB is noted for the transparency of its resource allocation decisions. The Bank’s criteria for allocating resources are publicly available and the majority of survey responses confirm that allocations generally follow these criteria. The Bank has also made adequate progress in improving the predictability of its disbursements.

This KPI aimed to assess whether the AfDB makes transparent and predictable aid allocation decisions, and compiled results of three MIs. One of these MIs was assessed by survey only, while the other two were only assessed by document review.

Figure 3.11 KPI 5: Resource Allocation Decisions, Ratings of Micro-Indicators*

* The document review for MI 5.3 draws on data from the 2011 Survey on Monitoring the Paris Declaration and the AfDB’s reports on its Paris Declaration commitments.

MI 5.1 – Criteria for allocating funding publicly available

This MI was rated by document review only. The AfDB was rated as very strong in making its criteria for allocating funding publicly available. The Bank’s Credit Policy[34] outlines the criteria used for determining countries’ eligibility for concessional funding (i.e., from the AfDF), non- concessional funding (i.e., from the AfDB), or a blend of Bank resources (i.e., both AfDF and AfDB funding).[35] In 2011 the Bank adopted a policy known as the Transition Framework for Graduating and Reversing Countries, which lays out criteria for moving countries from one category of funding to another. The primary criteria are per capita income and creditworthiness.[36] The Bank’s criteria for allocating African Development Fund (AfDF) resources are furthermore clearly explained and made available to the public via the AfDF’s website[37] and through AfDF reporting (i.e., through annexes of replenishment reports). Allocations for countries eligible for AfDF resources for example, are calculated based on a three-step formula that is laid out in the Performance-Based Allocation Framework of the ADF resource allocation report.[38]

MI 5.2 – Resource allocations conform to criteria

This MI was rated by survey only. The majority of respondents surveyed (77 per cent) rated the AfDB as adequate or above on this MI.

MI 5.3 – Resources released according to agreed schedules

This MI was rated by document review only. The AfDB was rated as adequate on the MI. The assessment was based on several sources including the Survey on Monitoring the Paris Declaration (indicator 7 on aid predictability), the Annual Development Effectiveness Review (ADER) of the Bank, as well as the 2011 OPEV Evaluation of the Paris Declaration Implementation at the AfDB[39]. The Bank has demonstrated commitment to aid effectiveness by integrating four Paris Declaration indicators into its Results Measurement Framework, including PD indicator 7 which “measures the gap between aid scheduled and aid effectively disbursed and recorded in countries’ accounting systems”. According to AfDB’s Roadmap for Improving Performance on Aid Effectiveness and Promoting Effective Development, the Bank had set itself the goal of increasing the predictability of disbursements from 54 per cent in 2007 to 80 per cent by 2012.[40] However, the most recent data collected by the OECD[41] and the AfDB[42] itself indicates that although the Bank has made progress in terms of ensuring predictability offunding by increasing the percentage to 61 per cent by 2011, it has fallen short of its aspirational target of 80 per cent.[43]

KPI 6: Linking Aid Management to Performance

Finding 6: Survey respondents rated the Bank adequate in linking loans and grants to results. The document review found that the AfDB is inadequate in linking overall budget allocations and disbursements to results.

The assessment looked at two different dimensions related to this KPI: whether the AfDB links budget allocations to expected results, and whether its reports on results are linked to disbursements.

Donors at headquarters, the only respondent group asked, rated the Bank adequate on both MIs. The document review, which focused on the organisation’s budget and overall disbursements, was less positive and rated the Bank inadequate on both micro-indicators.

Figure 3.12 KPI 6: Linking Aid Management to Performance, Ratings of Micro-Indicators

MI 6.1 – Allocations linked to expected results

When asked whether the AfDB links loans and grants to expected organisational results, the majority of donors surveyed at headquarters (67 per cent) rated the Bank’s performance adequate or higher.

The Bank received a rating of inadequate on this MI, based on a review of documents. AfDB’s Programme and Budget reports do not show explicit links between resource requirements and expected development results.[44] The Bank does however seem to be making efforts to redress this situation in the longer term, by ensuring that its annual budgetary allocations are aligned with expected results. Since 2007, the Bank has been implementing a series of budget reforms aimed at addressing issues related to: (i) lack of alignment between the Bank’s Strategy, Work Programmes and resource allocation; (ii) highly centralised budget authority (approximately 85 per cent of the budget was centrally managed); (iii) staff headcount control; and (iv) poor accountability.[45] According to the Programme and Budget 2012-2014, the second phase of the reform process is currently underway and has, among other things, ensured that the use of resources is recorded against output/activity codes (Work Breakdown Structure elements) so as to link expenditures to deliverables. As part of the reform process, the Bank reintroduced the Activity Time Recording System (ATRS) in 2010. This system is currently being tested within limited organisational units. The ATRS is linked to the Cost Accounting System (CAS) that will support UA Budgeting[46] in determining the overall costs of the Bank’s products and services. With the introduction of these systems, the Bank will, in principle, be able to track the cost of outputs, which is a necessary pre-condition for fully implementing a results-based budget. These various efforts could allow the Bank to improve its performance in linking allocations to results – but they are still in early stages and have not yet been fully integrated into the Bank’s operations. That being said, linking resources allocations to expected development results is a challenge for all MDBs.

MI 6.2 – Disbursements linked to reported results

Donors at headquarters were asked about the extent to which AfDB’s reports on results include the amount disbursed to achieve those results. Approximately half (48 per cent) rated the Bank adequate or above, while 33 per cent rated it inadequate or below.

The document review gave the Bank a rating of inadequate on this MI. A review of the Bank’s organisation-wide performance reports did not find a clear linkage between financial disbursements and results achieved. Although the Annual Reports produced by the AfDB provide an overview of its strategic directions and priorities as well as of its operations and major management and institutional reforms, they do not systematically link disbursements with development results achieved. Similarly, the Bank’s Annual Development Effectiveness Review does not systematically link disbursements to results achieved.

KPI 7: Financial Accountability

Finding 7: The AfDB was found to have sound policies and processes for financial accountability by the document review, and received strong ratings for its internal audit and anti-corruption efforts. Survey respondents expressed positive views about the Bank’s practices for both external and internal audit.

All MIs under this KPI were assessed by document review and survey except for one on the Bank’s anti-corruption policy and processes, which was assessed by document review only. The Bank’s clients and donors at headquarters were particularly positive about its financial audit processes, and rated the Bank strong on the way it conducts internal and external audits. More than one-third of the MOPAN donors in country offices answered ‘don’t know’ on all questions in this area.

The Bank received mixed ratings from the document review, ranging from adequate to very strong.

Figure 3.13 KPI 7: Financial Accountability, Ratings of Micro-Indicators

MI 7.1 – External financial audits performed across the organisation

Donors at headquarters (the only respondent group surveyed for this MI) were asked whether AfDB’s external audits are meeting the needs of donors. Nearly half (43 per cent) rated the AfDB strong or above.

The document review assessed whether external financial audits meeting recognised international standards are performed across the organisation, and rated the AfDB very strong. An external auditor conducts an organisation-wide financial audit annually, and provides a letter that confirms the audits are conducted in accordance with International Financial Reporting Standards (IFRS).

MI 7.2 – External financial audits performed at the regional, country, or project level

Survey respondents (donors in country offices and AfDB’s clients) were asked whether the AfDB performs appropriate external audits of programmes and projects at the country level. A total of 44 per cent of survey respondents rated the Bank strong or very strong.

The document review gave the AfDB a rating of adequate on this MI. The documents reviewed indicated that the AfDB has established clear rules and procedures for the conduct of audits within the organisation. The Bank’s Guidelines for Financial Reporting and Auditing of Projects note that it is the duty of the borrower to conduct external audits of projects financed by the Bank and that these are expected to be conducted each financial year by an independent auditor and comply with international standards (either International Standards of Auditing (ISA) or the auditing standards of the International Organisation of Supreme Audit Institutions (INTOSAI)).[47] It is our understanding that the extent to which the accounts of audited projects are made public depend upon the borrower's standard practice and AfDB’s disclosure policy. Thus, there were no audit reports available for review.

MI 7.3 – Policy on anti-corruption

This MI was rated by document review only. The review rated the AfDB very strong for its policy and guidelines to combat fraud and corruption. The Bank’s efforts are guided by the Bank Group Policy on Good Governance and the corporate-approved Guidelines for Preventing and Combating Corruption and Fraud.[48] The Bank has also put into place several mechanisms for addressing and sanctioning fraudulent behaviours from either Bank staff or clients, and has a policy of “zero tolerance” in this regard for staff members and executive directors, which is articulated in its Code of Conduct. The Bank’s Governance Strategic Directions and Action Plan for 2008-2012 lays out the Bank’s plans for combating corruption at country, sector and regional levels, as well as in the Bank’s adherence to the Uniform Framework for Preventing and Combating Fraud and Corruption, which consists of an agreement between several International Financial Institutions (IFIs) aimed at enforcing a “unified and coordinated approach to fight corruption and prevent it from undermining the effectiveness of their work.”[49]

MI 7.4 – Systems for immediate measures against irregularities

Although the Bank’s performance on this MI was rated adequate overall by survey respondents, there were significant statistical differences between the views of clients (who rated the Bank as adequate) and those of donors in country offices (who rated it as strong).

The document review, which assessed the actual procedures for response and follow up on irregularities identified during an external financial audit, rated the Bank adequate. The review of the Bank’s Guidelines for Financial Management and Financial Analysis of Projects indicated that there are procedures in place to address irregularities identified during external financial audits conducted at the project level. The guidelines delineate appropriate actions to be taken by relevant parties (i.e., borrowers and executing agencies; Bank staff and management) in the case of poor compliance with the Bank’s financial and audit covenants. The lack of compliance is generally reported in audit review memos for each project audit report and irregularities are followed up with relevant actions during supervision missions and during subsequent audits. According to the guidelines, the Bank’s procedures require that borrowers and EAs be notified of any violation of the loan’s financial or audit agreement and given the opportunity to propose a remedial plan of action.[50] According to the Bank, it is also standard procedure, in the event of serious misgivings, to apply sanctions such as the withholding of further disbursements until all issues have been adequately addressed.

Financial audit irregularities are generally reported to the Audit and Finance Committee of the Board (AUFI) which meets regularly with external auditors (at least twice in the year) to discuss the follow-up of implementation of their prior year audit recommendations.

MI 7.5 – Internal financial audit processes provide credible information

Nearly half of respondents at headquarters (40 per cent) rated the Bank as strong or very strong on the way it conducts internal financial audits to provide credible information to its governing bodies. They were the only respondent group asked about this indicator.

The document review rated the AfDB very strong. The Bank has designated the Auditor General’s Office (OAGL) as the unit responsible for conducting independent internal audits of the Bank’s operations. The latter operates independently from operational areas and reports directly to the president of the Bank, the Audit and Finance Committee and the Bank’s Board of Directors. Although AfDB’s policy does not allow for disclosure of internal audits to the public, document review indicates that the Bank has put into place policies and procedures that conform to the Institute of Internal Auditors' standards (IIA).

MI 7.6 – Effective procurement and contract management processes

Only donors in country offices and the Bank’s clients were asked whether AfDB’s procurement and contract management processes for the provision of services or goods are effective. Both surveyed groups rated AfDB’s performance as adequate overall.

The document review rated the AfDB adequate on this MI. Documents reviewed indicated that the Bank has developed clear rules, procedures and guidelines for the procurement of goods, works and other related services, and that these are made readily available to the Bank’s clients on the organisation’s website. The Bank’s Rules and Procedures for Procurement of Goods and Works indicate that the organisation places great emphasis on the timeliness of delivery as well as the quality and efficiency of products and services. Indeed, the Bank has introduced a number of measures to improve both the speed and quality of its procurement processes over the past years.[51] In practice however, the speed of procurement and other administrative procedures is an area that requires on-going attention. For example, the 2012 ADER report indicates slow progress in relation to procurement,[52] mainly due to challenges that most RMCs face in trying to meet the Bank’s strict fiduciary standards.[53]

MI 7.7 – Strategies for risk management

Approximately 70 per cent of donors at headquarters (the only group surveyed for this MI) rated the Bank’s performance as adequate and above on this MI.

The document review rated the Bank as adequate for the strategies that it has in place for risk identification, mitigation, monitoring and reporting. According to the 2011 Annual Report, the AfDB is currently in the process of formalising “the Bank’s framework for managing operational risks in accordance with evolving practices”, which is a reference to the Operational Risk Management Framework.[54] In addition, the Bank reports on the development of a new framework to address fiduciary risks, which encompasses its work in budget, audit and anti- corruption. Thus, the Bank appears to be developing appropriate tools for ensuring sound management of two types of risk - fiduciary and operational - but the frameworks were not available to be assessed during the course of the document review.

KPI 8: Using Performance Information

Finding 8: Overall, survey responses and the review of documents note that the AfDB is making adequate use of performance information. It stands out as very strong in its practices for allocating ADF resources to countries and projects based on performance.

The survey in general found the Bank to be adequate in its use of performance information to guide programming. Clients rated the Bank as strong for its use of information to plan new interventions at country level.

The document review rated the Bank’s performance as adequate overall, but found its performance to be very strong in allocating ADF resources to countries and projects based on performance.

Figure 3.14 KPI 8: Using Performance Information, Ratings of Micro-Indicators

MI 8.1 – Using information to revise and adjust policies

A majority of donors at headquarters (72 per cent) rated the AfDB adequate or above on the way it uses project, sector and country information on performance to revise corporate policies.

The document review provided a rating of adequate on this MI. Information on organisation- wide performance is available in the Annual Development Effectiveness Reviews (ADERs) and OPEV high level evaluation reports. Documents from OPEV provide some evidence that Bank Management takes into account recommendations from performance reports when revising and adjusting policies. According to OPEV,[55] recent evaluations have fed into the Bank’s decision- making on topics such as decentralisation, systems for ensuring quality at entry and supervision of the Bank’s investments. One example is the development of the Staff Guidance note on Implementation Progress and Results Reporting for Public Sector Operations that was developed on the basis of the Evaluation of Project Supervision at the AfDB 2001-2008 and other previous studies of project supervision at the Bank.

MI 8.2 – Using information for planning new interventions

Survey respondents were asked whether the AfDB uses information on country or sector performance to plan new interventions at the country level. The majority (76 per cent) rated the organisation as adequate or above on this MI. While donors in country offices rated the organisation adequate, clients rated it as strong in this area. This difference is statistically significant.

The document review rated the Bank’s performance as adequate. Information about the Bank’s country level performance is found in four sets of documents: Project Completion Reports (PCRs); Country Portfolio Performance Reviews (CPPRs), CSP Mid-Term Progress reports and Country Assistance Evaluations (CAEs). Findings from these reports are presented to Management and help to inform the preparation of Country Strategy Papers (CSPs). Document review of a small sample of CSPs shows that Country Assistance Evaluations are not consistently used in the preparation of strategies, particularly due to the coverage of CAEs in relation to the number of client countries.[56] The review also confirms that the Bank regularly uses the other country level reviews (i.e., findings from CPPRs, CSP Mid-Term Progress reports and PCRs) to plan new interventions or adjust on-going operations and strategies.

MI 8.3 – Proactive management of unsatisfactory initiatives

Although the AfDB was generally rated as adequate or higher on the way it manages “unsatisfactory” projects from previous fiscal years, 43 per cent of surveyed donors in country answered ‘don’t know’ to this question.

The document review provided a rating of adequate on this MI, based on information provided in the Annual Development Effectiveness Reviews of the Bank as well as COMPAS indicator 8, which refers to the number and percentage of projects that were unsatisfactory in Fiscal Year 09 and that became satisfactory in Fiscal Year 10. Although data from the last two COMPAS[57] reports indicate that there has not been an improvement in the percentage of unsatisfactory projects that became satisfactory in the following year, the reports confirm that the Bank has instituted measures to reduce the number and percentage of projects having an unsatisfactory performance.[58]

ADER 2012 reports on the Bank’s efforts to minimise and manage unsatisfactory projects in its portfolio. According to the report, the Bank has taken steps to prevent problems by increasing its supervision. Formal supervision through two missions a year is reported for almost 57 per cent of its projects, close to its 2012 target of 60 per cent. It also aims to ensure that the percentage of “problem” projects stays below 5 per cent, and the ADER 2012 report confirms that the Bank is below that in 2011.

MI 8.4 – Evaluation recommendations are acted upon

A small majority of donors at headquarters (55 per cent) rated the Bank as adequate or higher in tracking the implementation of evaluation recommendations reported to the Board.

The document review rated the Bank as adequate on this MI. Documents consulted indicated that the Bank’s Evaluation Policy requires a management response and an action plan for all of the Bank’s higher level evaluation recommendations (i.e., corporate evaluations, sector reviews, thematic evaluations etc.). Documents reviewed revealed that the evaluations conducted by the Bank are generally followed up with a management response. However, the OPEV Three-Year Rolling Work Programme and Budget for 2012-2014 highlights the following:

“Preliminary discussions conducted by OPEV in 2011 showed that various stakeholders - especially OPEV, the General Secretariat (SEGL), and ORQR - were concerned with monitoring the implementation of evaluation recommendations. A preliminary review of the practices of other multilateral institutions revealed a variety of systems, each with a different degree of complexity and comprehensiveness and with different human, budgetary, and IT resource needs. In 2012, OPEV will continue the necessary consultations to determine the appropriate system for the Bank.”[59]

Thus, although there is some evidence that evaluation recommendations receive the necessary management response, the Bank still lacks a formal system for tracking the implementation of those recommendations that are accepted by management and reported to the Board.

MI 8.5 – Resources allocated based on performance

This MI was assessed by document review only. The AfDB received a rating of very strong for allocating ADF resources to countries and projects based on performance.

The Bank’s Performance Based Allocation system applies to its concessional resources. A number of discussion papers and issue papers leading up to ADF 11 and ADF 12 replenishment processes describe how the PBA process has been continuously reviewed and how the focus on performance has been reinforced. There is clear evidence of the performance component of the allocation system in the Country Policy and Institutional Assessment (CPIA) posted on the Bank’s website. Historical data describing the CPIA scores and ratings used for the Country Performance Assessment (CPA) can also be found on the organisation’s website.

KPI 9: Managing Human Resources

Finding 9: The document review and survey results indicate that the AfDB has human resource management systems in place, but faces challenges in applying them and demonstrating that its management of staff performance is transparent and results-oriented.

Survey respondents found the Bank’s management of human resources to be adequate overall, although donors at headquarters were more critical on several MIs.

The document review also acknowledged the efforts made by the AfDB to improve its organisational performance, but found that the organisation still faces challenges in instilling a performance-based culture.

Figure 3.15 KPI 9: Managing Human Resources, Ratings of Micro-Indicators

MI 9.1 – Results-focused performance assessment systems for senior staff

When asked whether the AfDB uses results-focused performance assessment systems to evaluate senior staff, 38 per cent of donors at headquarters rated the organisation adequate and above while 28 per cent provided a rating of inadequate or below. However, 35 per cent of donors surveyed answered ‘don’t know’ to this MI, which suggests that not all of the MOPAN members are aware of the extent to which senior staff are subject to performance reviews that are focused on results.

The document review rated the AfDB as inadequate in this area. The Bank adopted a new Performance Management System (PMS) in 2009 and developed a handbook that outlines the Bank’s practices for managing staff performance and performance review procedures. The Bank’s Performance Management Handbook makes references to the processes and procedures for evaluating the performance of senior management, at least to the level of Directors or Unit Heads. The performance assessment is based on an evaluation of agreed individual objectives and key performance drivers (behavioural qualities) that, in principle, would draw on self-assessment, peer or client assessments, and subordinates’ assessments. It is unclear whether the same procedures apply to the level of the Bank’s Vice Presidents. Yet as noted in MI 9.2 below, there are still numerous challenges to implementing the Performance Management system, including delays in completing each stage of the performance cycle, tendencies to inflate ratings and a generalised lack of recognition by management of the importance of managing personnel in such a way that encourages an overall culture of performance.

MI 9.2 – Transparent system to manage staff performance

Donors at headquarters (the only group asked to assess this MI) were asked two questions: whether the AfDB uses a transparent system to manage staff performance and if it uses a transparent system to reward staff performance. The Bank was rated as inadequate on both questions, though it bears noting that 33 per cent of donors answered ‘don’t know’ to the question about the Bank’s practices for rewarding staff performance.

The document review also rated the Bank as inadequate on this MI. While the AfDB has put into place a system that rewards staff based on merit (the Performance Based Pay system), the effectiveness of the system is dependent on the full operationalisation of the Bank’s Performance Management System, which has faced some challenges. A recent review of the Bank’s PMS highlighted a few areas that hindered the full application and effectiveness of its rewards and incentives system, including the degree of management ownership of the PMS and meagre compliance mechanisms to enforce a consistent use of the new PMS.

MI 9.3 – Country staff deployment supports partnership development

This MI was assessed by survey only. Donors in-country and clients were asked to assess whether the Bank’s deployment of international staff allows for effective partnerships at country level. Overall, 57 per cent of respondents rated the Bank as adequate and above. However, many donors in country offices rated the Bank’s performance as inadequate or below (49 per cent), while the majority of the Bank’s clients rated its performance as adequate and above (79 per cent). The differences in the ratings were statistically significant.

MI 9.4 – Staff recruitment and promotion is transparent and based on merit

This MI was assessed by survey only. While a small majority of donors at headquarters rated the Bank as adequate or above for recruiting staff based on merit (53 per cent), almost half of the respondents rated it as inadequate or below (45 per cent).

KPI 10: Performance-oriented Programming

Finding 10: The AfDB was generally rated adequate by survey respondents and in the document review for its performance-oriented country/regional programming processes.

The majority of survey respondents shared the perception that AfDB’s country/regional programming processes are performance oriented. The Bank’s clients also perceived the AfDB as strong in setting targets that enable monitoring of progress during project implementation.

The document review rated the AfDB as adequate overall on this KPI, due to its efforts to ensure that projects are technically sound and designed to maximise development impact, that is, that there is good “quality-at-entry” of approved projects.

Figure 3.16 KPI 10: Performance-oriented Programming, Ratings of Micro-Indicators

MI 10.1 – New initiatives subject to benefits/impact analysis

The majority of donors surveyed at headquarters (77 per cent) rated the AfDB adequate or higher in terms of subjecting new loans and credits to impact analysis prior to approval.

The document review rated the Bank adequate on this MI. The Bank “requires that financial analysis and economic analyses are undertaken for projects”[60] during the appraisal process. Prior to approving any projects, the Bank first seeks to evaluate the feasibility of the project, mainly by focusing on an analysis of economic and financial performance, as well as an analysis of environmental and social impacts. Recommendations from these analyses are later used to inform decision-making about whether or not the Bank should fund a particular project. The review of sample appraisal reports found that the AfDB does indeed conduct regular economic as well as environmental and social impact analyses during its project appraisals.

MI 10.2 – Milestones/targets set to monitor implementation

The majority of survey respondents (60 per cent of donors in-country and 88 per cent of clients) rated the AfDB as adequate or above in setting targets that enable monitoring of progress during project implementation. However, there were statistically significant differences between the views of donors at country level, who rated the organisation adequate, and those of clients, who gave it a rating of strong.

The document review provided a rating of adequate on this MI. The 2011 Staff Guidance on Implementation Progress and Results Reporting for Public Sector Operations of the Bank requires that information from the results-based logical framework (RLF) of appraisal reports be used to track progress of results achieved at the project level. These guidelines aim to overcome prior deficiencies in terms of systematic use and content of the logical frameworks.[61]

Most of the 2011 Project Appraisal Reports sampled for the MOPAN assessment included both targets and baselines to monitor progress of project/programme implementation. Moreover, data from the 2010 COMPAS report indicates that 64 per cent of the Bank’s projects (34 out of 53 public sector investment projects) had explicit baseline data monitoring indicators, and clearly defined outcomes to be reached.[62] The newly introduced project implementation progress and results (IPR) reporting tool is expected to help improve the monitoring of results achieved at the project and eventually at the country level. Yet the use of IPRs has not yet been fully mainstreamed and implemented at the country level.

KPI 11: Delegating Authority

Finding 11: Survey respondents and the document review rated the Bank inadequate overall in delegating authority to the country level.

The majority of clients and donors in country offices (the only groups surveyed for the assessment of this KPI) rated the organisation’s performance as inadequate overall. However, many donors in-country were not familiar with the Bank’s decentralisation processes (32 per cent answered ‘don’t know’ to the questions in this area).

While acknowledging the Bank’s efforts to increase its presence in RMCs in recent years, the document review indicated that its delegation of decision-making authority remains limited at the country level.

Figure 3.17 KPI 11: Delegating Authority, Ratings of Micro-Indicators

MI 11.1 – Aid reallocation decisions can be made locally

Although both clients and donors in country offices rated the Bank adequate for its ability to re- allocate resources within project budgets locally, 40 per cent of donors in country offices responded ‘don’t know’.

The document review rated the Bank inadequate on this MI. For many MDBs, major reallocations or restructuring of projects must be approved by the Board of Directors. The AfDB is no different in this regard and decision making remains highly centralised. One of the key findings of the 2011 Evaluation of Paris Declaration Implementation at the African Development Bank conducted by OPEV, related to the lack of delegation of decision making:

“Decentralisation has been the single most important factor affecting the Bank’s performance on AE [aid effectiveness] by staff and country partners. The reform is progressing at a slow pace and the Bank still has some way to go to build the presence and capacity that would enable it to become a major player in the aid effectiveness agenda at country level. Delegation of responsibilities to field offices is a key step. However the Delegation of Authority Matrix (DAM) has not been updated since 2008. (…) The current share of tasks that are managed by field offices is modest. In 2009, field offices were responsible for overseeing supervision of about 15 per cent of the project under implementation, but less than 10 per cent of new projects under preparation (Decentralisation Roadmap 2010, p. 9). Field offices remain reliant on decisions and “no objections” from HQ” [63]

According to interviews with Bank staff, the AfDB is experimenting with the possibility of reallocating more decision-making authority to regional levels. The Bank anticipates that the recent conversion of two field offices, in Nairobi and Pretoria, into Regional Resource Centres (RRCs) that cover the Eastern and Southern Africa regions will provide sufficient regional expertise to allow it to delegate authority for decision making on country-level matters. At the time of the assessment the Bank was in the process of revising its Delegation of Authority Matrix (DAM), and expected to make greater delegation of authority to Bank staff at regional and country levels a part of the DAM. According to Bank staff, when the new DAM is issued in October 2012 they anticipate that 90 per cent of procurement decisions will be processed and approved by the ADB Field Offices.

MI 11.2 – New programmes/projects can be approved locally within a budget cap

Donors in-country and clients were asked whether the AfDB has decentralised its project approval processes to local levels. Nearly half (48 per cent) rated the Bank inadequate or below, while 38 per cent rated it adequate or above.

The document review rated the Bank inadequate on this MI. A review of the Presidential Directive Concerning the Approval Authority for Operations indicates that all operations need approval from the Board of Directors with the exception of emergency operations of less than UA 1.0 million (which can be approved by the President). This practice is not unique to the AfDB and is reflected in other MDBs.

As noted in MI 11.1 above, the Bank is in the process of updating its Delegation of Authority Matrix, but the extent to which the Bank will decentralise processes or decisions related to loan operations or other products is unclear.

3.3.4 Relationship Management

The AfDB is seen as adequate in engaging with clients and development partners in ways that support the principles of aid effectiveness. There is room for improvement in the efficiency of its administrative procedures and use of country systems. The review of documents also suggests that the extent to which the AfDB participates in joint missions and supports government-led Programme-Based Approaches (PBA) could be improved.

Figure 3.18 below shows the overall survey review ratings for the five KPIs in the relationship management quadrant.

Overall the AfDB was perceived to perform adequately by survey respondents across all key performance indicators in Relationship Management. The document review noted the Bank’s commitment to the Paris Declaration and its efforts to integrate several PD indicators in its Results Measurement Framework.[64]

The review of documents drew on sources such as OECD’s bi-annual Survey on Monitoring the Paris Declaration, AfDB’s Roadmap for Improving Performance on Aid Effectiveness and Promoting Effective Development (2011), ADER 2012, as well as OPEV’s evaluation of the Paris Declaration Implementation in the AfDB.

Figure 3.18 Quadrant III: Relationship Management, Survey and Document Review Ratings

Figure 3.19 shows the mean scores for the five KPIs for all survey respondents, and by respondent groups.

Figure 3.19 Quadrant III: Relationship Management, Mean Scores by Respondent Group

KPI 12: Supporting National Plans

Finding 12: Survey respondents perceived the AfDB to be adequate in its support of national plans, due to its practice of funding project proposals that are designed and developed by its clients. The review of documents found that the Bank’s approach to conditionality is also linked to a government’s own indicators.

Survey respondent groups in general gave the AfDB a rating of adequate for its support of national plans. However, clients had a more positive view on this area of performance than in- country donors.

Figure 3.20 KPI 12: Supporting National Plans, Ratings of Micro-Indicators

MI 12.1 – Funding proposals developed with national government or clients

This MI was assessed by survey only. Donors in-country and clients were asked about the extent to which the AfDB supports funding proposals designed and developed by the national government or other clients. The majority of survey respondents (74 per cent) rated the AfDB as adequate or above. Clients were more positive than donors in-country and rated the bank as strong. The difference is statistically significant.

MI 12.2 – Conditionality draws on national benchmarks or indicators

Donors in-country and clients were asked about the extent to which AfDB’s conditionality draws on national benchmarks or indicators. The majority (66 per cent) rated the AfDB adequate or above.

The document review rated the Bank as adequate on this MI, based mainly on evidence related to policy-based operations.

OPEV’s recent Evaluation of Policy-Based Operations in the African Development Bank, 1999-2009 concluded that the design of Policy-Based Operations (PBOs) has been simplified, with a reduction in the scope and number of conditions. It also found that predictability of PBO disbursements had improved over the period evaluated, largely as a result of greater harmonisation and simplified conditions, but that in-year disbursements (i.e., disbursing as planned within the year designated for disbursement) and alignment with the budget cycles of Regional Member Countries (RMC) remains a problem.

The World Bank and AfDB have articulated common principles for conditionality in the context of budget support operations in fragile states.[65] The proposed approach suggests that core conditionalities in these contexts should be limited in number and drawn from national strategies – Poverty Reduction Strategies and/or National Development Strategies (PRS/NDS) – and national procurement programmes. The paper also states that conditionalities should be results-oriented, focused on the most critical elements of these national strategies, and realistic in light of the low implementation capacities of recipient countries. When they are drawn from a country’s national strategies, the use of limited and clear conditionalities can reinforce ownership and increase medium-term predictability by reducing uncertainty in the interpretation of conditions at the time of making disbursement decisions. According to the paper, the number of budget conditions per Budget Support Operation (BSO) in African Development Fund (ADF) countries dropped from an average of ten to seven between 1999 and 2009.[66]

KPI 13: Adjusting Procedures

Finding 13: The AfDB was perceived by respondents to be adequate in taking into account local conditions and capacities. However, responses indicate that the time it takes to complete procedures is too lengthy.

This KPI was assessed by the survey only. Clients and donors in-country rated the Bank adequate on three and inadequate on one MI.

Figure 3.21 KPI 13: Adjusting Procedures, Ratings of Micro-Indicators

MI 13.1 – Procedures easily understood and completed by partners

Asked whether the AfDB uses procedures that can be easily understood and completed by clients, the majority of clients and donors in-country (65 per cent) rated the organisation as adequate or above, and 29 per cent as inadequate or below.

MI 13.2 – Length of time for procedures does not affect implementation

Asked whether the length of time it takes to complete AfDB procedures affects implementation, the majority of clients and donors in-country (56 per cent) rated the organisation as inadequate or below, and 39 per cent as adequate or above.

MI 13.3 – Ability to respond quickly to changing circumstances

Asked whether the AfDB adjusts its work at country level quickly to respond to changing circumstances on the ground, only 48 per cent of clients and donors in-country rated the organisation as adequate or above, with 38 per cent rating it as inadequate or below.

MI 13.4 – Flexibility in implementation of projects/programmes

Donors in-country and clients were asked whether the AfDB flexibly adjusts its support to programmes and activities based on learning and budgetary issues. The majority of survey respondents (53 per cent) rated the AfDB as adequate or above, and 33 per cent as inadequate or below.

KPI 14: Using Country Systems

Finding 14: Survey respondents rated the AfDB as adequate overall in its use of country systems for operations and promotion of mutual accountability in partnerships. The document review provided mixed ratings.

Both MIs assessed by the survey under this KPI received ratings of adequate.

The document review of this KPI was based on various sources of data including ADER 2012, OPEV’s Evaluation of Paris Declaration Implementation at the African Development Bank (2011), AfDB’s Roadmap for Improving Performance on Aid Effectiveness and Promoting Effective Development (2011), and the Survey on Monitoring the Paris Declaration (indicators 3, 5a, 5b and 6). While the review of documents confirmed the Bank’s commitment to the Aid Effectiveness Agenda, it received moderate ratings due to its limited use of country systems. The Bank has maintained a cautious approach to fiduciary management due to specific challenges in many RMCs (e.g., corruption and lack of governmental capacity). The document review ratings should therefore be read in light of the contextual factors that the Bank faces in many of the countries where it operates.

Figure 3.22 KPI 14: Using Country Systems, Ratings of Micro-Indicators

MI 14.1 – ODA disbursements/ support recorded in annual budget

This MI was assessed by document review only.

The Bank was rated as adequate on this MI. The document review was based on various sources of data including the 2012 Annual Development Review (ADER), OPEV’s Evaluation of Paris Declaration Implementation at the African Development Bank (2011), AfDB’s Roadmap for Improving Performance on Aid Effectiveness and Promoting Effective Development (2011), the Survey on Monitoring the Paris Declaration (indicator 3) and the Quality of Official Development Assistance Assessment (QuODA).[67]

The document review indicates that the Bank has made progress in ensuring that aid flows are recorded in partner countries’ budgets. For example, the QuODA confirms that the AfDB ranks high among multilateral organisations with regard to percentages of aid recorded in partner countries’ budgets. ADER 2012 indicates that progress has been made in this area, although it notes that the target has not yet been met. More specifically, the report says that in 2011, 70 per cent of the Bank’s aid had been properly recorded in countries’ annual budgets, up from 57 per cent in 2009. However, the 2011 figure remains far below the 85 per cent level that the Bank targeted for the end of 2012.

MI 14.2 – Use of country systems for operations

Donors in-country and clients were asked whether the AfDB uses country systems (e.g., procurement systems and public financial management systems etc.) as a first option for its operations where appropriate. Slightly more than half of survey respondents (56 per cent) rated the AfDB as adequate or above, 24 per cent as inadequate or below, and 20 per cent answered ‘don’t know’.

The document review rated the Bank as inadequate on this MI. The review was based on the sources described above (see MI 14.1) and indicators 5A and 5B of the Survey on Monitoring the Paris Declaration. The documents reviewed gave mixed though generally negative assessments of the Bank’s use of country systems. While data from OECD/DAC did not record any progress in relation to the two relevant indicators, OPEV’s evaluation highlighted progress in the Bank’s use of country public management systems and slower progress in its use of country procurement systems. OPEV’s evaluation also mentioned that progress in the use of country systems was hampered by the lack of adequate systems meeting international fiduciary standards in RMCs. In this regard, it is important to note that the Bank has played a central role in strengthening audit capacities both at the regional level, through support provided to the African Organisation of Supreme Audit Institution (AFROSAI) and at the national level. It now has a joint strategy with the World Bank to strengthen existing national external audit systems.[68]

MI 14.3 – Parallel implementation structures avoided

This MI was assessed by document review only.

The Bank received an overall rating of adequate based on its progress with regard to indicator 6 of the Survey on Monitoring the Paris Declaration (strengthen capacity by avoiding parallel implementation structures) as reported in the document sources described above under MI 14.1. Both OECD/DAC data and ADER 2012 indicate that the Bank has made significant progress in reducing the use of parallel Project Implementation Units (PIUs). According to ADER 2012, parallel Project Implementation Units were initially measured at 113 in 2007 but fell to 38 in 2011, a figure that is below the Bank’s target of 40 for 2012. The recent evaluation of the Bank’s Implementation of the Paris Declaration (2011) seems to concur with the aforementioned findings, but also indicates that challenges remain in this area. According to the evaluation, incentives for management and staff to implement aid effectiveness principles are insufficient and indeed there are strong disincentives, such as stringent financial targets, that explain the persistence of non-aligned practices such as parallel PIUs. Furthermore, in some cases, for example in Fragile States, PIUS will continue to be preferred as a more effective way of supporting implementation. According to the evaluation, the parallel PIU issue “epitomizes the dilemma the Bank faces around the use of country systems.”

MI 14.4 – Promotion of mutual accountability in its partnerships

This MI was assessed by survey only. Donors in-country and clients were asked about the extent to which the AfDB encourages mutual accountability assessment of the Paris Declaration and subsequent Aid Effectiveness commitments (Accra Agenda for Action, Busan High Level Forum). More than half of the respondents (56 per cent) rated the Bank as adequate or above, but 32 per cent answered ‘don’t know’ to this question.

KPI 15: Contributing to Policy Dialogue

Finding 15: According to survey respondents, the AfDB makes an adequate contribution to policy dialogue while respecting the views of its partners in the process.

This KPI was assessed by the survey only and the AfDB was rated adequate. Overall, donors at headquarters held more favourable views of the AfDB in this area of performance than other respondent groups, and rated the organisation as strong in its ability to respect and take into account input from clients.

Figure 3.23 KPI 15: Contributing to Policy Dialogue, Ratings of Micro-Indicators

MI 15.1 – Reputation for valuable input to policy dialogue

All respondent groups were asked whether the AfDB provides valuable inputs to policy dialogue. The majority of survey respondents (67 per cent) rated the AfDB as adequate or above.

MI 15.2 – Policy dialogue respects partner views

All respondent groups were asked whether the AfDB respects the views of partners when it undertakes policy dialogue. The majority of survey respondents (77 per cent) rated the AfDB as adequate or above.

KPI 16: Harmonising Procedures

Finding 16: The AfDB was seen by survey respondents as adequate overall on the harmonisation of its procedures with other actors. The document review noted joint missions and participation in programme-based approaches as areas where the Bank had shown less progress.

Although the Bank received adequate ratings overall, clients were in general more positive about its participation in programme-based approaches (PBAs), and rated the organisation as strong on this MI.

The document review found that there is room for the Bank to improve its participation in joint missions and its level of support to Regional Member Countries through PBAs.

Figure 3.24 KPI 16: Harmonising Procedures, Ratings of Micro-Indicators

MI 16.1 – Participation in joint missions

When asked to assess the extent to which the Bank participates in joint missions, 59 per cent of the respondents rated the organisation as adequate or above while 24 per cent rated it as inadequate or below.

The document review rated the Bank as inadequate on this MI. According to the 2011 OECD/DAC survey, only around 9 per cent of the Bank’s missions were being jointly undertaken. External reviews, including the 2011 OPEV Evaluation and the QuODA confirm this rating. Indeed, according to the QuODA, the Bank’s proportion of co-ordinated joint missions is among the smallest of any of the multilateral development banks.

MI 16.2 – Technical cooperation disbursed through coordinated programmes

Donors in-country and clients were asked whether AfDB’s technical assistance is provided through coordinated programmes in support of capacity development. The majority of survey respondents (60 per cent) rated the AfDB as adequate or above.

The document review provided a rating of adequate on this MI. According to the OECD/DAC survey (2011), the Bank has made good progress in this area. In 2007, 31 per cent of its technical co-operation flows were being disbursed through co-ordinated programmes while by 2010, this had increased to 68 per cent. OPEV’s Evaluation of the Implementation of the Paris Declaration in the Bank notes that progress has been made in better aligning its support with other development partners. According to the evaluation, the increased presence of field offices has deepened the bank’s participation in development partner sector working groups, and many field offices are involved in several working groups. However, overall the evaluation considers the Bank’s progress towards greater harmonisation to be moderately satisfactory. The 2010 QuODA on the other hand indicated that the AfDB is among the MOs with the lowest share of co-ordinated technical cooperation.

MI 16.3 – ODA disbursements/support for government-led PBAs

Donors in-country and clients were asked whether the AfDB participates in programme-based approaches. The majority of survey respondents (70 per cent) rated the AfDB as adequate or above. Clients were more positive than donors in-country and rated the Bank as strong on this MI. The difference is statistically significant.

The document review rated the AfDB as inadequate on this MI. Data from the Survey on Monitoring the Paris Declaration indicates limited progress in the use of common arrangements or procedures. Indeed, the percentage of AfDB support channelled to countries through PBAs regressed in 2007 from 40 per cent to 32 per cent, and has stagnated since then. The 2011 OPEV evaluation of the implementation on the Paris Declaration notes that the Bank’s participation in pooled funding mechanisms has been limited and highlights a number of factors that limit the Bank’s participation in programme based approaches, including its application of procurement rules of origin, lack of guidance to staff about different aid modalities and the Bank’s generally cautious approach to fiduciary management. However, in 2008 the ADF rules of origin were amended to allow the Fund to participate in pooled funding mechanisms with other donors.

3.3.5 Knowledge Management

Survey respondents and the document review found AfDB’s evaluation of external results to be adequate overall. Several areas for improvement in its presentation of performance information were found.

Figure 3.25 below shows the overall survey and document review ratings for the three KPIs in the knowledge management quadrant.

According to survey results, the AfDB performs adequately in all areas of knowledge management. The document review ratings ranged from adequate to strong.

Figure 3.25 Quadrant IV: Knowledge Management, Survey and Document Review Ratings

Figure 3.26 shows the mean scores for the three KPIs for all survey respondents, and by respondent groups.

Figure 3.26 Quadrant IV: Knowledge Management, Mean Scores by Respondent Group

KPI 17: Evaluating External Results

Finding 17: The AfDB was perceived by survey respondents as performing adequately in the evaluation of external results, based on their assessment of use of evaluation findings and involvement of clients in evaluations. The document review rated the AfDB as strong in the independence of its evaluations but inadequate in its quality assurance mechanisms.

Three MIs were assessed by document review only and two by the survey only under this KPI. The AfDB was rated as adequate on both MIs assessed by the survey.

The document review provided ratings ranging from inadequate to strong. The AfDB ensures the structural independence of its evaluation unit and has strengthened the department with additional resources in recent years. However, according to the document review, OPEV’s quality assurance mechanisms could be strengthened.

Figure 3.27 KPI 17: Evaluating External Results, Ratings of Micro-Indicators

MI 17.1 – Independent evaluation unit

This MI was assessed by document review only. The review found the Bank to be strong on the MI. The Operations Evaluation Department (OPEV) is the independent evaluation unit of the AfDB. OPEV reports directly to the Board of Directors through the Bank’s Committee on Operations and Development Effectiveness (CODE). The Independent Evaluation Policy and Functional Responsibilities of the Operations Evaluation Department (OPEV) (2007) includes a number of guiding principles for conducting evaluations, and clearly describes how independence is assured.

MI 17.2 – Sufficient evaluation coverage of programming activities

This MI was assessed by document review only. The AfDB was rated as adequate for the coverage of its evaluations. The evaluation coverage is delineated in the Three-Year Rolling Work Programme and Budget developed by OPEV. The Work Programme provides an overview of the different evaluation products and dissemination activities that will be undertaken by the Bank during the three-year period. It also explains how these activities are linked to the Bank Group’s strategic orientation as set out in the Medium-Term Strategy 2008-2012. Comparatively speaking, the AfDB allocates approximately the same proportion of its resources to evaluation as other development banks. Thus in 2010, the AfDB allocated 1.5 per cent of its total administrative budget to OPEV, which is equivalent to the percentage allocated by the World Bank for similar activities but slightly less than that allocated by the Asian Development Bank (ADB).

Although OPEV has been strengthened with additional human resources in recent years, the Three-Year Rolling Work Program and Budget 2011-2013 notes that additional staff is required to effectively respond to the Bank’s evaluation needs. OPEV’s Budget and Work Program points to an “evaluation gap” at the Bank, which means that major areas of the portfolio are not adequately covered, most significantly the rapidly growing area of private sector operations, and that the number of Country Assistance Evaluations carried out annually may be insufficient (as mentioned under MI 8.2). In addition, the Bank has not been able to launch a programme of rigorous, well-resourced impact evaluations.

MI 17.3 – Quality of evaluations

This MI, which was assessed by document review only, focused on the existence and application of quality assurance mechanisms (i.e., it did not assess the quality of actual evaluations). It rated the AfDB as inadequate.

According to the Bank’s Evaluation Policy, OPEV operates according to internationally accepted principles for the evaluation of development assistance such as the OECD/DAC Evaluation Principles, the Development Banks’ Evaluation Cooperation Group (ECG)[69] standards, and the United Nations Evaluation Group’s (UNEG) good practice standards. However, the document review did not find evidence of a formal process for assessing the quality of evaluations. That being said, OPEV is currently conducting a self-assessment and will use the findings to update the 2007 Evaluation Policy.

MI 17.4 – Use of evaluation findings to inform decisions

This MI was assessed via survey of donors at headquarters only. When asked whether the Bank ensures that its evaluation findings inform decisions on programming, policies and strategies, 75 per cent of respondents rated the Bank as adequate or above.

MI 17.5 – Beneficiaries and clients involved in evaluations

This MI was assessed by survey of donors in-country and clients only. The majority of respondents (73 per cent) rated the Bank adequate or above. While donors in-country rated the Bank adequate, clients rated it strong, and the difference is statistically significant. It should be noted however that 37 per cent of donors in-country answered ‘don’t know’.

KPI 18: Presenting Performance Information

Finding 18: Survey respondents believe that the AfDB adequately presents performance information on its effectiveness, a finding that was also reflected in the document review.

The majority of donors at headquarters, the only respondent group asked, rated the Bank adequate in presenting performance information on its effectiveness.

The document review rated the Bank adequate on all MIs, noting the improvements made in corporate reporting on its strategy and results, as well as its Paris Declaration commitments, since the introduction of the ADER.

Figure 3.28 KPI 18: Presenting Performance Information, Ratings of Micro-Indicators

MI 18.1 – Reports on achievement of outcomes

Donors at headquarters were asked whether the Bank’s reports to the Board provide clear measures of achievement of outcomes. The majority (72 per cent) rated the Bank adequate or above.

The document review rated the Bank adequate on this MI. As noted in KPI 2, the AfDB has adopted the One Bank Results Measurement Framework and the Annual Development Effectiveness Review (ADER), which reports on the four levels of results articulated in the One Bank framework. Although ADER represents a major improvement in the Bank’s reporting on results at the organisation-wide level, the extent to which it provides evidence of AfDB’s contribution to development outcomes is still limited (see also Chapter 4).

MI 18.2 – Reports on performance using data obtained from measuring indicators

The document review provided a rating of adequate on this MI. AfDB’s One Bank Results Measurement Framework defines a set of common core performance indicators for the Bank. The Bank’s organisation-wide reporting on results in the ADER measures progress against indicators defined for the four levels of results. Targets and baselines are clearly established and the performance indicators generally respect internationally established criteria for quality.[70]

While the ADER has good potential, it was only introduced in 2011 and does not yet illustrate trends over time.

MI 18.3 – Reports against corporate strategy, including expected results

The majority of donors at headquarters (79 per cent) rated the AfDB as adequate or above on reporting against its corporate strategy.

The AfDB was rated as adequate for this MI by the document review. The Bank’s organisation- wide performance report ADER systematically reports against development and management priorities set in its Medium-Term Strategy. Reporting on results is based on the One Bank Results Measurement Framework, which is designed to monitor progress toward the vision described in the Medium-Term Strategy. Reporting on results under Level 1 is related to development outcomes to which the Bank is seeking to contribute, but that are too broad to be attributed to the Bank alone. Level 2 measures the AfDB’s contributions to key sector outputs and intermediate outcomes in priority sectors identified in the Strategy.

MI 18.4 – Reports on Paris Declaration commitments using indicators and country targets

A significant majority (73 per cent) of donors at headquarters rated the AfDB adequate and above on the way it reports on Paris Declaration commitments.

The AfDB was rated as adequate by the document review. AfDB’s recently prepared Roadmap for Improving Performance on Aid Effectiveness and Promoting Effective Development provides a coordinated framework to guide and coordinate its efforts to implement the Paris Declaration (PD) commitments. One of the purposes of this roadmap is to define how the monitoring of aid effectiveness should be mainstreamed. As part of this process, the Bank has identified four PD indicators[71] that it finds particularly central to development effectiveness. According to the Bank, recent international evaluations on aid effectiveness, including evaluations by the Organisation for Economic Co-operation and Development (OECD), suggest that these areas of intervention are the ones most likely to maximise development benefits for RMCs.

The four PD indicators have been integrated into Level 3 of the reporting in ADER. Targets are set and each year’s progress in implementing aid effectiveness principles are published in the ADER report. However, the Bank does not report on individual country targets, instead focusing on targets for “All African countries” and specifically for “ADF countries”. This being said, it bears noting that its external reporting on the PD indicators has improved significantly since the introduction of ADER.

MI 18.5 – Reports on adjustments to policies/strategies based on performance information

This MI was rated by document review only. It receives a rating of adequate for reporting on adjustments made to policies and strategies based on performance information. ADER 2012 for example reports on adjustments made in the area of human resources management based on performance information indicating that the Bank suffers from relatively high premature attrition rates. According to OPEV’s Work Program and Budget, recent evaluations have fed directly into the Bank’s decision-making on critical topics such as decentralisation, agriculture, and the Bank’s systems for ensuring quality at entry and supervision of the Bank’s investments.

In addition, the Bank’s Programming and Budget Department (COBS) prepares a set of annual reports for Management and the Boards including the Quarterly Budget & Performance Reports and the Annual Retrospective Review of the budget. These reports review operational and managerial issues based on a set of key performance indicators (e.g., number of projects managed by field offices, number of Project Completion Reports produced, amount of operations approved vs. targets, amount of lending vs. target, Problematic Projects Rate, Achievement on projects at risk, vacancy rate etc.).

MI 18.6 – Reports on programming adjustments based on performance information

This MI was assessed by document review only. The AfDB was rated as adequate in country- level reporting on programme adjustments being based on performance information. While no evidence has been found of an annual reporting mechanism used to systematically adjust strategies, standard Country Strategy Paper (CSP) review mechanisms were identified at the country level. The Bank’s CSPs are reviewed at mid-term and at completion. Both the Mid- Term Progress reports and the Completion Reviews are presented to the Boards of Directors and are made available to the public. The Mid-Term Progress Reports provide an assessment of the development of the country context (political, economic and social developments), results achieved during the first half of the CSP period and issues related to the portfolio management. Based on that assessment, the Mid-Term Progress reports lay out the strategy for the remaining period including adjustments made in terms of strategic and operational focus. According to interviews with the Bank, the AfDB is currently developing Annual Country Development Effectiveness Reviews, and specific Sector Reviews focusing on the Bank’s operations in fragile states. However, the introduction of these reports is still in the initial stages and very few Reviews have been completed to date.

KPI 19: Disseminating Lessons Learned

Finding 19: The Bank was rated adequate overall in disseminating lessons learned, and the document review noted that the AfDB has made improvements in reporting on lessons learned based on performance information.

In the survey for this KPI, which included donors at headquarters only, the AfDB was perceived to perform adequately in encouraging the identification, documentation and dissemination of lessons learned and/or best practices. The document review rated it as adequate in reporting on lessons learned based on performance information.

Figure 3.29 KPI 19: Disseminating Lessons Learned, Ratings of Micro-Indicators

MI 19.1 – Reports on lessons learned based on performance information

A majority of donors at headquarters (63 per cent) rated the Bank adequate or above on this MI. The document review also rated it as adequate, but also noted the Bank’s commitment to developing new strategies, projects and programmes based on lessons learned.

The Bank has begun to systematically disseminate and use lessons learned in its operations. For example, it has improved the quality and timeliness of Project Completion Reports (PCRs), thereby providing a stronger basis for using lessons learned in the development of new operations. The ADER 2012 refers to an OPEV review of a sample of PCRs produced in 2008-2009 that showed considerable improvement in compliance, timeliness and quality. According to the review, 78 per cent of PCRs were rated satisfactory or highly satisfactory, compared with 71 per cent of the 2003-2005 sample. The Bank has also been able to increase the percentage of completed projects that have PCRs.

In 2011, the Bank also introduced the Readiness Review System aimed at facilitating internal use of lessons learned during the design and appraisal processes of projects. Under this system, a team carries out desk reviews for each new project: first of the Project Concept Note, and then, following the design process, of the Project Appraisal Report. The review criteria include lessons learned from prior operations among other criteria (e.g., rationale and ownership, focus on results and risk assessment, implementation arrangements, financial criteria etc.).

The ADER 2012 and interviews with Bank staff at HQ indicate that the production of PCRs and the use of Readiness Reviews are still works in progress.

MI 19.2 – Lessons shared at all levels of the organisation

In this MI, which was assessed by survey only, donors at headquarters were asked whether the AfDB provides opportunities at all levels of the organisation to share lessons from practical experience. Only 38 per cent rated the AfDB adequate or above, while 33 per cent rated it inadequate or below, and 30 per cent answered ‘don’t know’.

4. Main Findings: Development Results Component

4.1 Overview

This section presents the results of the 2012 Common Approach assessment of the AfDB in the pilot component to assess the development results of multilateral organisations. It includes four key performance areas:

• Evidence of the extent of progress towards organisation-wide outcomes (KPI A)

• Assessment of country-level results and relevance:

- Evidence of the extent of contribution to country-level goals and priorities (KPI B)

- Evidence of the extent of contribution to relevant Millennium Development Goals (KPI C)

- Relevance of objectives and programme of work to stakeholders (KPI D)

Figure 4.1 provides a snapshot of the findings of this assessment. Please note that the pilot applied a simplified 4-point scale that uses the same “traffic light” colours used elsewhere in the report. As noted in Section 2.8 of this report, the scale was simplified to reflect the methodological approach used in the pilot of the development results component – in which various sources of data are considered together when rating the organisation’s performance on each KPI. The methodology is explained in Volume II, Appendix I.

Figure 4.1 Development Results Component – Overall Ratings*

*Across all KPIs in the development results component, respondents from the DRC provided lower ratings than respondents in other countries; the difference is statistically significant. This may reflect challenges that AfDB faces in operating in a fragile state.

4.2 Progress towards Organisation-wide Results

4.2.1 Overview

This section presents the findings of the assessment of organisation-wide results. KPI A suggests that an effective organisation should demonstrate progress towards organisation- wide, institutional outcomes. These are usually related to the organisation’s strategic objectives. The assessment draws on the evidence that the organisation has available on its results, particularly on its contributions to development outcomes.

AfDB’s results and reporting at organisation-wide level

As noted in Chapter 3 (KPI 2), AfDB’s work at the organisation-wide level is based on the Medium-Term Strategy (MTS) 2008-2012 and is structured in five-year cycles.

According to the Bank, “the overarching objective of the African Development Bank (AfDB) Group is to spur sustainable economic development and social progress in its regional member countries (RMCs), thus contributing to poverty reduction.” The MTS 2008-2012 emphasises four key sectors of operations (infrastructure, governance, private sector, and higher education) and puts specific operational focus on regional integration, support to fragile states, support to middle income countries (MICs), and development of the agricultural sector. In addition, the Bank seeks to better integrate gender considerations, environmental issues, climate change, and knowledge management throughout its operations.

As part of the Bank’s reform process, it introduced the “One Bank” results measurement framework in 2010. The results framework covers the entire Bank Group and is based on a set of core sector indicators organised in four levels. Level 1 measures Africa’s overall development progress in nine areas, including growth, human development, public service delivery, and governance. Level 2 presents the aggregate outputs and intermediate outcomes of the Bank’s operations in the same nine areas. Level 3 assesses how well the Bank manages its portfolio of operations, and Level 4 describes how well the organisation is managed. The assessment of KPI A mainly focuses on the Bank’s contributions to Level 2 results (i.e., aggregate outputs and intermediate outcomes achieved by projects supported by the Bank). However, the extent to which the Bank provides links between lower level results described under Level 2 and higher level regional results in Level 1 is also discussed.

As noted in Chapter 3 (MI 18.1), in 2011 the Bank introduced the Annual Development Effectiveness Review (ADER), an organisation-wide report that monitors the Bank’s contributions to development results in Africa, based on some of the indicators from the “One Bank” results measurement framework. ADER reports according to each of the four levels described above. Results reported under Level 2, which focuses on project results, are based on self-reported data from Project Completion Reports. For ADER 2012, this included data from on-going operations between 2009 and 2011 for which Project Completion Reports were available.

The AfDB communicates its organisation-wide results annually through the ADER and the Annual Report by the Boards of Directors. It also took stock of achievements and shortfalls in the 2010 Mid-Term Review of the 2008-2012 Strategy. Finally, OPEV, the Bank’s evaluation department, conducts periodic evaluations of the Bank’s performance in selected sectors and focus areas.

Data used for this assessment

The assessment of KPI A is based on survey data from donors at headquarters and a review of performance information provided from the most recent strategic cycle (2008-2012). This included the ADER 2011/2012, the Mid-Term Review of the Medium-Term Strategy 2008-2012, and available high level corporate and sector evaluations related to the areas assessed.

Attention was paid to the use of indicators, baselines and targets, and the nature of the data provided in these reports. As MOPAN is particularly interested in understanding contributions to development outcomes, the assessment analysed the extent to which the data substantiated the outputs achieved and plausible contributions to higher level results.

In late 2012, the AfDB will introduce a series of Development Effectiveness Reviews (DERs) aimed at analysing achieved results in particular areas of the Bank’s portfolio. The first set of DERs will cover the following thematic areas: governance, fragile states and conflict-affected countries, and regional operations.[72] Although the DERs were still works in progress at the time of the assessment, draft DERs on governance and fragile states were shared with the Assessment Team for the purpose of the assessment.[73]

4.2.2 KPI A: Evidence of the Extent of Progress towards Organisation-wide Outcomes

Overall assessment

Figure 4.2 shows the overall rating for this KPI based on the review of AfDB’s contributions to development outcomes in Africa – as expressed in AfDB reports and as indicated by surveyed donors at headquarters. The headings show the criteria that MOPAN used to assess each focus area and determine the overall rating (criteria met are indicated in blue). The final column shows the mean scores of the surveyed donors at headquarters, based on the same 6-point scale that was used in the assessment of AfDB’s practices and systems.

The assessment focuses on how the AfDB is substantiating the achievement of outputs and demonstrating its contributions to the higher level planned results (outcomes and goals) associated with focus areas as described in the Medium-Term Strategy 2008-2012.

Figure 4.2 KPI A: Evidence of the Extent of Progress towards Organisation-Wide Outcomes, Ratings

1 Baselines are not provided for Level 2 results. However, baselines are systematically used for Level 1, Level 3 and Level 4 results.
2 Most of the evidence of reported changes is derived from ADER which is based on aggregate results from Project Completion Reports. Evaluations and studies were available in relation to some of the areas assessed (WSS, health, agriculture, private sector) and these were taken into consideration. The assessment of the Bank’s performance in the Governance area was based primarily on the draft Development Effectiveness Review.
3 A review/evaluation of the 2009-2012 Strategy on Regional Integration is under way but the evaluation report was not available for the first draft of the MOPAN report. Therefore the assessment of organisation-wide contribution to results is based on ADER alone.

Evidence of AfDB’s Contributions

Finding 20: In the Annual Development Effectiveness Review, there is a significant gap between Level 2 results (outputs) and region-wide development results reported under Level 1 (outcomes). The Bank’s strategies and reports do not provide adequate explanations of the links between these two levels, also known as part of the theory of change.

Although the Bank’s introduction of the ADER in 2011 represents a key milestone in AfDB’s reporting on results, and puts it on par with other major development banks in this area, the Bank does not explain the intended links and assumptions that lead from the outputs identified under Level 2 to the regional outcomes reported under Level 1. As a consequence, the intermediate steps that would provide a more credible link between outputs directly supported by the Bank and its possible contributions to higher level development outcomes in the region are lacking. This is not unique to the AfDB; it is a challenge that most MDBs are facing.[74]

Changes in the wider development context and even to some extent outcomes and outputs described under Level 2 in the One Bank RMF cannot be attributed to Bank interventions alone. However, a basic theory of change (i.e., an explanation of how a set of low level results or outputs is intended to produce medium to long-term results) would be a useful tool to better explain how the AfDB is contributing to change and could provide a more unifying framework for strategic decision making, communicating and reporting. Developing a theory of change at the organisation-wide level for an organisation with a broad mandate (such as the AfDB) is admittedly challenging. Nevertheless, such a theory could define how the Bank’s business model (i.e., types of interventions) is intended to stimulate change at higher levels.

A preliminary review of the draft Development Effectiveness Reviews (DERs) indicates that the Bank is heading in a promising direction in reporting on its contributions in select areas of its portfolio. Similar to the ADER, the DERs reviewed are structured around the four levels of the Bank’s Results Measurement Framework and report, under Level 2, on the Bank’s contributions to select areas of focus. While this type of reporting cannot resolve the issue of attribution which all MDBs face, the AfDB anticipates that the DERs will provide clearer links between the Bank’s outputs and its contributions to higher-level development outcomes.

Finding 21: There is evidence that the Bank has made progress in areas where it is considered to have comparative advantages and, according to the 2012 ADER, the Bank met most of the targets identified for the 2009-2011 period.

The Medium-Term Strategy 2008-2012 emphasised the need for greater selectivity and focus at the sector level in order to leverage the Bank’s limited resources more effectively and provide leadership at the country level. In analysing the main areas of results achieved during the latest strategic planning period, the document review found that the Bank had an increasingly stronger focus on a limited number of areas. In the survey, donors at headquarters also seem to indicate that the Bank is contributing to development results in areas where it is seen to have comparative advantages (as shown in Figure 4.2 above, donors at headquarters provided ratings of adequate or above on 7 of the 10 focus areas).

According to the ADER 2012, overall, the Bank meets or exceeds most of the targets and only a few indicators show lower than expected performance. While this would seem to indicate that the Bank is performing strongly overall, it also raises the question of whether the targets are ambitious enough or if the Bank is avoiding risks by setting targets that are easy to reach.

Although ADER provides a good overview of results achieved in relation to certain indicators, the ADER indicators alone are not always sufficient to make a judgment about the Bank’s contributions overall. For example, its reporting on contributions to Regional Integration is based on two indicators only, but as illustrated in the Regional Integration Strategy, this is a multidimensional area that cannot be fully captured by the two indicators included in ADER.

Evidence of progress in AfDB’s focus areas

Finding 22: Among the ten focus areas of the Bank, the assessment found the greatest evidence of results in the area of infrastructure.

According to the 2010 Mid-Term Review of the MTS 2008-2012, AfDB’s investments in infrastructure operations during the period reviewed represented more than 60 per cent of the Bank’s total investment in the four core areas (infrastructure, governance, private sector, and higher education). Energy and power received the largest share (47 per cent), followed by transportation (39 per cent), water supply and sanitation (12 per cent), communications (1 per cent), and other infrastructure (2 per cent).

Transport

The ADER 2012 provides evidence of progress towards most of AfDB’s targets in the transport sector. Survey respondents largely confirmed this and provided a rating of strong for the Bank’s contributions in this area.

Considering the amount of funding and emphasis directed towards the transport sector, it is interesting that the Bank does not have a specific strategy for transport. The document review found no recent study or evaluation of the Bank’s contributions to the transport sector and the effects of those investments on people’s lives, however, OPEV is in the process of conducting an impact evaluation of the Bank’s operations in this sector.

Energy

Based on the evidence from ADER 2012, it also seems that the Bank has made contributions to the energy sector in Africa (although complementary documentary evidence is scarce). Survey respondents provided an adequate rating of the Bank’s contributions to the energy sector (85 per cent rated this adequate or above). Most of the targets established in ADER were delivered, apart from one area (people with a new electricity connection) in which only 60 per cent of the target was achieved (19,811 new connections as opposed to the 33,271 originally planned). ADER does not offer any explanation of variations in results achieved and overall there is little documentary evidence of results achieved at an organisation-wide level in this sector. Indeed, the Bank’s Energy Sector Policy and Energy Strategy only exist in draft form, but it is expected that both documents will be approved and implemented from 2012. The draft Energy Policy has a results measurement framework with outcomes, outcome indicators, output indicators, key activities of and assumptions/risks.

Water and Sanitation

One of the Bank’s main contributions in the Water and Sanitation sector has been through the Rural Water Supply and Sanitation Initiative (RWSSI) launched by the Bank in 2003 to mobilise investment in rural water supply and sanitation in Africa.[75] In 2005, the initiative was adopted by African governments and development partners as the common framework for resource mobilisation and investment for rural water and sanitation delivery in Africa. The RWSSI 2015 goals for water supply and sanitation are aligned to the respective MDG targets of 70 per cent and 62 per cent respectively (revised in 2011 from 80 per cent).

According to the 2011 Rural Water Supply and Sanitation Initiative Progress Report (September Edition), 16 of the 28 RWSS programmes approved have started to deliver water and provide sanitation facilities (see sidebar for details.) As of June 2011, the AfDB had provided 41 per cent of the financing for the initiative (€ 754 million) and other donors, African governments and local communities provided the rest of the funding.

In 2010, OPEV conducted a review of the Bank’s work in this sector. According to its synthesis note, Urban and Rural Water Supply and Sanitation Synthesis Note on Evaluation Results, total AfDB loan and grant approvals to urban and rural water supply and sanitation accounted for about 5 per cent of overall AfDB assistance during the 1968-2008 period. In the period 2003-2007, the share of AfDB’s assistance to the water and sanitation sector had grown to 10 per cent. The report concluded that the performance of Bank-funded water supply and sanitation projects was satisfactory with respect to meeting water supply objectives, but less satisfactory in sanitation and the environment.

Finding 23: Survey respondents rated the Bank strong for its contribution to regional integration, but documentary evidence of results achieved is scarce.

Despite the lack of documentary evidence of the Bank’s contributions to regional integration, the strong rating provided by survey respondents and the Bank’s increased focus on regional integration suggests that it is contributing to development in this area.

According to the Mid-Term Review of the MTS 2008-2012, investments in regional operations constituted 14 per cent of total Bank group lending during this period. The Bank’s investments in multinational or regional operations were concentrated in the infrastructure sector, particularly transport (40.3 per cent), energy (7.4 per cent), water and sanitation (3.6 per cent), and IT (2.7 per cent).

According to ADER 2012, there has been progress on the two indicators included in the report (see sidebar). However, it is difficult, on the basis of two indicators alone, to provide a judgment of results achieved at an organisation-wide level in this complex area.

According to information provided by the Bank, the 2009-2012 Strategy on Regional Integration is currently being evaluated and the report will be published later in 2012.

4.3 Evidence of the Extent of Contribution and Relevance at Country Level

4.3.1 Overview

This section presents the results of the assessment of evidence of AfDB’s contributions to country-level results and its relevance to stakeholders. By separating the KPI at the organisation-wide level from KPIs at the country level, MOPAN recognises the demand-driven nature of many of the activities of a multilateral organisation and the key role that is played by its country programming or strategy document, where expected results at the highest level (outcomes and impact) reflect a shared responsibility between the multilateral organisation and the partner country.

AfDB’s results and reporting at country level[76]

The Bank’s work at the country level is normally based on a five-year programming cycle. Country Strategy Papers (CSPs) are prepared in collaboration with each borrowing regional member country to define a medium-term to long-term development strategy. The CSPs are based on and aligned with a country's own development plan and poverty reduction goals.

The Bank communicates its results at country level through a number of key documents, including documents produced as part of its planning cycle such as the CSP, the Country Strategy Mid-Term Review, and CSP Completion Report at the end of the strategy cycle. In situations where the CSP is extended, results from the completed cycle are summarised in the CSP extension report. OPEV, the Bank’s independent evaluation department, also conducts a number of CSP evaluations each year.

Data used for this assessment

For this pilot test in 2012, the assessment is based on data from a sample of five of the six MOPAN survey countries which were proposed by the AfDB based on the availability of results data.

In the document review, all results information provided from the latest completed CSP period in the five focus countries was reviewed. The document review also reviewed the first draft Country Development Effectiveness Review (CDER) developed by the Bank for Rwanda.[77]

Attention was paid to the following elements: the quality of the evidence presented to substantiate the results achieved; the overall performance story; quality of the results statements; the baselines and targets; the strength of the link between result statements and results achieved.

In the survey, clients and MOPAN donors in-country were asked questions that were tailored to each of the five countries. Interviews with senior AfDB country office staff also informed the analysis of context and ensured that the assessment team had the right documentation.

The following sections on country-level KPIs present the overall results of this five-country review, with country-specific examples used to illustrate key findings.

4.3.2 KPI B: Evidence of the Extent of Contribution to Country-level Goals and Priorities

This KPI indicates that an effective organisation would demonstrate contributions to country- level goals and priorities. The assessment reviewed survey data from in-country stakeholders (donors and clients) and documentation provided by five AfDB country offices following interviews with senior country office staff.

Overall assessment

Figure 4.3 presents the overall rating for this KPI based on the review of AfDB’s contribution to country level goals and priorities as expressed in AfDB reports and as indicated by surveyed stakeholders. It also shows the criteria that MOPAN used to determine the overall rating (criteria met are indicated in blue).

Figure 4.3 KPI B: Evidence of the Extent of Contribution to Country-Level Goals and Priorities, Rating

1 This column is based on the total mean score (all respondents) in each country. A checkmark indicates the organisation received ratings of strong or higher on the majority of focus areas and limited, if any, ratings of inadequate.
2 Some baselines provided, but not used consistently, or not labelled as “baselines” but included in the results statements (e.g. “increase from XX per cent in 2005 to XX per cent in 2009).
3 Ibid.
4 Some baselines provided, but not for all indicators.

Evidence of AfDB’s contributions

Finding 24: The Bank’s Country Strategy Paper results statements are very closely linked and in some cases identical to national goals and priorities. While the AfDB reports on the country’s progress towards national goals, it provides little evidence of the extent to which it contributed to these results.

The Bank’s reports on results at the country level (e.g., CSP Completion Reports and CSP Mid- Term Reviews) do not usually explain the links between AfDB’s contributions and country level outputs and outcomes. Some CSP results frameworks are based on very high level national development outcomes (e.g., changes in inflation rates, growth in GDP, reduced maternal mortality at national level, etc.) and lack a description of concrete outputs or intermediate outcomes achieved by projects supported by the Bank (this is the case for the CSP of Ghana and to some extent Morocco). Also, while most of the results frameworks include monitorable indicators, baselines are not consistently used to measure progress. The lack of linkage between the Bank’s contributions and higher level outcomes at the country level makes it difficult to assess the extent to which the Bank contributed to national development results identified in the CSPs. Conversely, in other cases (e.g., DRC), the CSP results frameworks are closely linked to project objectives which gives a good sense of the Bank’s contribution at the project level, but makes it difficult to provide a judgment of the extent to which the organisation’s strategy and portfolio as a whole contributed to country-level development outcomes. According to the Bank such difficulties are however not unusual given the context of the DRC. The weak statistical capacity of the country, amongst other factors, presented challenges to developing quality results frameworks.

Some of these methodological issues were discussed in the Mid-Term Review of the 2005-2009 CSP in Ghana. The review highlighted major differences between results reported at the country level (included in the CSP Results Framework) and project-level results. According to the mid-term review, this difference constitutes a disconnect between “CSP outcomes” and actual outcomes from projects supported by the Bank. The assessment of project results (included in the Mid-Term Review) concludes that actual project outcomes in the social and water/sanitation sectors were very limited at mid-term, which raises the question of why good progress was reported with regard to the CSP outcomes in the same sectors. According to the mid-term review, the reason for this contradiction is that results defined in the CSP results framework are too global, covering all Ghana, and far too ambitious in relation to the size of Bank assistance (e.g., “reduce poverty from 40 per cent in 1999 to 32 per cent in 2007”).

For obvious reasons it is very difficult to establish a direct link between a comparatively small Bank investment and a precise quantitative reduction in poverty of millions of Ghanaians (within a relatively short period). While this example is taken from the Mid-Term Review of the CSP for Ghana, it also applies to other CSPs that tend to include very high-level results in their results frameworks, and which makes it difficult to discern AfDB’s concrete contributions.[78] This is a challenge for all MDBs and is not unique to the AfDB.

All this being said, the preliminary review of the draft Country Development Effectiveness Review developed for Rwanda indicates that the Bank is improving the reporting of results achieved at country levels. As of 2012, the CDER will form part of a series of country and thematic reviews that the Bank aims to publish as a complement to the ADER. Like the ADER, the CDERs will be structured around the four levels of the Bank’s RMF. It is anticipated that the publication of these reviews will provide a clearer picture of the Bank’s contributions to national development outcomes.

Finding 25: While the quality of the reviewed CSP results frameworks varies considerably, there are clear indications of improvement over time.

While none of the CSPs analysed provide explicit theories of change, they all build on a sound analysis of the country context, development challenges, and are linked to national development plans. However, the CSP results frameworks do not provide an explicit theory of the causal links that tie programme inputs to expected outputs and outcomes. The quality of results content in country-level documents varies considerably and in many cases the results described as either outputs or outcomes are often a mixture of the two types of results. Some of the results frameworks are based on very high level results statements that are difficult to correlate with the results of the Bank’s operations. Baselines are not systematically provided, and when they are, they are often national figures which make it difficult to assess AfDB’s contributions. However, some of the most recent CSPs reviewed (e.g., Morocco) include results frameworks that are generally of higher quality than earlier frameworks.

Most of the issues related to the Bank’s country results frameworks were noted in the Mid-Term Review of the CSP for Ghana.

• The original results framework included a number of results statements at the outcome level, but only a few of the indicators designed to track these outcomes were specific, measurable, relevant or time-bound (SMART). Only one of the indicators was quantified, and the relationship between some outputs and outcomes was not clear.

• The extent to which the expected results were achievable was questionable as some targets were very high level (e.g., ‘Proportion of people living below the poverty line’”).

• As with most of the Bank’s CSP results frameworks, there was a mixture of expected results at the output level (for which the AfDB would be directly responsible), and what could be seen as country outcomes, which would only be achieved with the contribution of a range of actors.

• Due to these weaknesses, the CSP’s results-based framework was revised following the Mid-Term Review in 2008. While there are no reports based on the updated results framework, it represents a clear improvement compared with the previous version.

• Staff members from the country office in Ghana indicated that a new CSP is being prepared and that it will include a more complete results framework based on lessons learned from the mid-term review of the 2005-2009 CSP.

Finding 26: The document review and survey data indicate that the Bank has made contributions to country-level outcomes in infrastructure development in Ghana and Morocco. Its funding for infrastructure development in DRC and Nigeria is significant, but there is less evidence of its contribution to development outcomes.

Based on analysis of results achieved in the five countries assessed and ratings from survey data, there is evidence that the Bank has contributed to infrastructure development results.

Given the amount of resources it invested in infrastructure in these countries, this might not be a great surprise. For example, more than 70 per cent of the Bank’s assistance to Morocco focused on infrastructure during the latest completed CSP period. Likewise, 44 per cent of the Bank’s resources in DRC were directed towards infrastructure development, while 40 per cent of the public sector portfolio in Nigeria and 26 per cent in Ghana was focused on infrastructure development (in Ghana however, funding to infrastructure was surpassed by allocations to projects in the agriculture sector that covered 30 per cent of the portfolio).

The CSP Completion Report from Morocco (2011) offers several concrete examples of outcomes achieved by major infrastructure projects to which the Bank contributed along with other major donors such as the World Bank and the European Union (EU) (See the section on Morocco below for more detail).

Ratings from survey respondents generally supported the positive document review assessment in this area. In all countries assessed, the Bank’s support to infrastructure development received the highest average ratings among the results areas identified for each country. This was especially the case in Morocco and Ghana where the Bank received strong ratings in all results areas related to infrastructure.

Evidence of Results Achieved, by Country

The following results by country are drawn from both the survey results and an analysis of documentation. Specific examples are given of results where the AfDB country office provided particularly strong documented evidence that a result was achieved.

DRC

When asked about AfDB’s contributions to its eight country-level goals and priorities for DRC for the period 2008 to 2012, survey respondents rated the Bank inadequate overall (41 per cent rated it adequate or above). The Bank’s contributions to infrastructure development (water and sanitation/rural and urban road networks) were rated as adequate but 67 per cent of donors in- country responded ‘don’t know’ to questions about the Bank’s contributions to water and sanitation and electrification.

Figure 4.4 AfDB Stakeholder Survey - DRC, Mean Scores by CSP Priorities

The document review focused on the 2008-2012 CSP for the DRC, which was revised in 2010 following a mid-term review. The indicators and results statements in the revised CSP Results Framework for the 2010-2012 period are very different from those in the original 2008 CSP. The original results framework was based mainly on indicators from the government’s Growth and Poverty Reduction and Strategy Paper (GPRSP) and was a mixture of outcome and output indicators (mainly high level outcome indicators). In the revised framework, progress is measured primarily in terms of outputs[79] achieved by the government with support from the Bank, which better illustrates the Bank’s contributions to development outcomes at the country level.

The 2010 mid-term review of the CSP noted that progress under the CSP’s Pillar 1 (support to good governance) was slow mainly due to the complexity of reforms, weak institutional capacity for their implementation, the security situation in the Eastern part of the country, and the international financial crisis at the end of 2008 and in 2009. With regard to CSP Pillar 2 (promoting pro-poor growth), it noted that infrastructure development in support of pro-poor growth was significant in the transport sector but less so in the energy, water/sanitation, and agricultural sectors.

According to documents reviewed and interviews with the DRC country office, the lack of progress on Pillar 1 was primarily due to a delay in the implementation of the Bank-supported Public Administration Reform Support Programme (PAREG) which should have been appraised in 2008. The approval of PAREG was based on the condition that the Government concluded a new programme supported by the IMF. However, due to implications of the Sino-Congolese Cooperation Agreement signed between DRC and a group of Chinese companies in 2008, the IMF-supported programme stalled and PAREG was not implemented. The Bank’s other governance projects were also delayed and re-scheduled for the 2010-2012 period.

According to the Bank, most of the on-going operations in DRC were started prior to the development of the 2008-2012 CSP and designed in a post-conflict situation, therefore focusing on capacity development activities. Hence, there is a time lag for the new operations started since the 2008-2012 CSP to bear fruit.

In interviews at the DRC country office, staff members noted that the Bank is mitigating the slow implementation (due in part to the political climate and capacity problems) by supervising its projects closely through two project supervision missions per year per project and three annual monitoring missions.

While progress towards the 2008-2012 CSP targets was generally slow, there is evidence of results of the Emergency Programme to Mitigate the Impact of the International Financial Crisis (PUAICF) approved in 2009. UA 65 million that had initially been earmarked for PAREG was allocated to this programme (see sidebar).[80]

Ghana

The majority of survey respondents (80 per cent) provided ratings of adequate or above when asked aboutAfDB’s contributions to country- level goals and priorities in Ghana for the period 2005-2011. As shown in Figure 4.5 below, respondents rated the Bank’s contributions to infrastructure development as strong. However, 33 per cent of donor respondents answered ‘don’t know’ regarding the Bank’s contributions to rural growth, education, health, and promotion of rural services.

Figure 4.5 AfDB Stakeholder Survey - Ghana, Mean Scores by CSP Priorities

The document review, which focused on AfDB Ghana’s 2005-2009 CSP (updated for 2010-2011) found it difficult to judge the Bank’s contribution to country-level outcomes due to the very high level of the results statements, as noted above in Finding 24.

The 2008 Mid-Term Review of CSP project level outputs/outcomes revealed that the Bank’s contribution to the attainment of the country’s development goals had been mixed (see sidebar). Some of this was due to the slow implementation of projects, and because some projects were implemented late in the period and, as such, could not be expected to have tangible impact by the time of the Mid- Term Review in 2008 (e.g., three projects were approved in late 2007, and five between mid-2005 and mid-2006).

According to interviews with Ghana country office staff and a review of the 2010 CSP Extension Report and various project level documents, delayed infrastructure projects are now being implemented as a result of collaborative efforts of the Bank and other donors.

There is also evidence from recent project completion reports and interviews with AfDB staff that the AfDB has invested considerable resources in the agriculture sector in Ghana (close to 30 per cent of its portfolio) and that roughly 50 per cent of their agriculture projects are concentrated in the poorer Northern parts of the country.

Nevertheless, the Bank’s performance story in Ghana is quite difficult to discern in the documentation reviewed. While there is evidence in project documentation that the Bank has contributed to specific results, the Bank’s country-level reports do not articulate what it has contributed to Ghana as a whole through its operations (projects), analytical services, and participation in policy dialogue.

Morocco

When asked about AfDB Morocco’s contributions to country-level goals and priorities, MOPAN donors in-country and clients rated the AfDB as adequate or strong in all of the Country Strategy Paper (CSP) priority areas for Morocco. In four out of the ten areas assessed, both respondent groups rated the Bank’s contributions as strong. However, 30 per cent answered ‘don’t know’ regarding AfDB’s contributions to increasing the investment rate and its contributions in the area of education.

Figure 4.6 AfDB Stakeholder Survey - Morocco, Mean Scores by CSP Priorities

The document review focused on the results achieved during AfDB’s 2007-2011 CSP cycle. In general, the CSP Completion Report for Morocco provides evidence of results achieved by the Bank (although the quality of the results framework could have been better).

During this period, the Bank’s assistance to Morocco focused on infrastructure (71 per cent), followed by governance (20 per cent) and the social sector (9 per cent). Its concentration in these sectors was reinforced after the March 2009 CSP mid-term review, which recommended that the Bank should put more emphasis on the first two pillars in order to support the priority thrusts of the Government’s agenda (2008-2012).

The document review largely confirmed the perceptions of survey respondents. See the sidebar for examples of contributions to infrastructure development. The results of the National Rural Roads Programme (PNRR2) launched in 2005 by the Moroccan government provide a sense of the higher level development outcomes brought about by interventions supported by the AfDB. However, as with most descriptions of results achieved at the country level, it can be difficult to discern the extent to which the Bank contributed to these outcomes. This is especially so as the PNRR2 was co-sponsored with other development partners such as the World Bank and the EU.

The country context in Morocco does, to some extent, facilitate reporting on results. According to a recent evaluation of AfDB’s policy-based operations in Africa, which used Morocco as a case study,[83] the quality of data is good in Morocco, which facilitates the assessment of progress. Moreover, the gradual establishment of RBM at the government level allows donors, including the AfDB, to better define the completion of objectives.

Niger

When asked about AfDB’s contributions to the CSP priorities for Niger during the 2005 – 2012 period, 51 per cent of respondents provided ratings of adequate or above. In Water and Sanitation, more than 70 per cent of respondents rated AfDB’s contributions as adequate or above; in the health sector, only 37 per cent of respondents rated its contributions adequate or above. There was a high level of ‘don’t know’ responses overall (32-34 per cent) on all questions apart from the Bank’s contributions to the water and sanitation sector.

Figure 4.7 AfDB Stakeholder Survey - Niger, Mean Scores by CSP Priorities

The document review focused on the results achieved during the Bank’s CSP 2005-2009 (extended to 2012) in Niger. Most of the data is derived from the CSP Extension Report.

By 2010, operations in the agriculture sector accounted for 43 per cent of the Bank’s sector commitments in Niger, the social sector 20 per cent, transport 14 per cent, multinational operations 11 per cent, and water and sanitation 9 per cent.

According to the review of documents, the results achieved in Niger during the strategic period were mixed.

Under Pillar I (rural development through water resource mobilisation) there is evidence of an increase in the irrigated areas and agricultural production (see sidebar).

According to the CSP Extension paper, the improvement of drinking water supply for rural households suffered from some delays in executing certain works under the Drinking Water Supply and Sanitation Project in three regions (Maradi, Tahoua and Tillabéry) and the Integrated National Water Resource Management Action Plan. While the Bank-financed projects did not perform well in this area due to delays, the Government’s own actions and those of the development partners helped achieve a drinking water supply rate of 62.3 per cent in 2008 which is on par with the CSP objectives.

Under Pillar II (infrastructure development), few results were recorded. Two road projects were approved during the period (Tibéri-Dakoro and Madaoua-Bouza Road Improvement Project and Dori-Tera-Burkina Faso Border Road Improvement Project). The first project started late owing to long delays in obtaining the commitment of the West African Development Bank (BOAD) to co-finance the project. The second project, which has a sub-regional dimension, also fell behind schedule, the Government having modified the structure of the roadway, thereby increasing the project cost by 125 per cent. According to the Results Matrix, it was expected that the road length would increase from 3,677 km in 2005 to 7,990 km in 2009. However, due to the delays described above, the length of asphalted roads only increased to 3,912 km in 2008 which is significantly less than expected.

The Bank also planned to provide budget support to sustain the consolidation of macro- economic stability and the improvement of public finance management. However, according to the Extension Paper, the budgetary support was not presented to the Board in 2009 because of the deterioration of the country’s political situation. An important contextual factor to keep in mind is that the Bank does not have a country office in Niger. According to the Bank, this situation affects reporting practices and evaluation process at the operational level because of limited interactions with the government, project managers and other development partners in the country. However, the Bank is making efforts to improve this situation through the on-going decentralisation process. As part of this process, the Country Economist for Niger (who until recently was based in Tunis) was transferred to Burkina Faso in July 2012 and will cover Niger from there, which should ensure a closer interaction with main stakeholders.

The Bank is currently preparing the CSP 2013-2017 which will be initiated in August 2012. In this context a review of the CSP 2005-2009, extended to 2012, and a review of the Bank’s portfolio in Niger will be conducted. It is expected that both reviews will contribute more information about development outputs and outcomes, provide recommendations for the new strategy, and set up an Action Plan for improving the quality of the portfolio and preparation of new operations for Niger.

Nigeria

When asked about AfDB’s contributions to the Bank’s five country-level goals and priorities identified for Nigeria for the period 2005 to 2011, 53 per cent of respondents provided ratings of adequate or above. On questions related to infrastructure development and private sector development, more than 70 per cent of respondents rated AfDB’s contributions as adequate or above, while on the question related to education, only 21 per cent rated the AfDB as adequate or above. There was a high level of ‘don’t know’ responses among clients (36-43 per cent) on all questions apart from the Bank’s contributions to water and sanitation facilities.

Figure 4.8 AfDB Stakeholder Survey - Nigeria, Mean Scores by CSP Priorities

The document review focused on the results achieved during the Bank’s CSP 2005-2009 (extended for 2010-2011) in Nigeria.

As of 2008 (during the Mid-Term Review of the CSP) AfDB Nigeria’s public sector portfolio was dominated by infrastructure (40 per cent), followed by the social sector (32 per cent) and the agriculture sector (28 per cent). The private sector accounted for 34 per cent of total net commitments.

According to the Mid-Term Review, the results of the Bank’s interventions were mixed during the 2005-2008 period. Under CSP Pillar I (development of human capital through improved service delivery in education and health), the Bank’s intervention contributed, overall, to some positive results in the health sub-sector while the anticipated progress in the education sector lagged behind.

Under Pillar II (stimulating private sector-led growth in the non-oil sector through enhanced infrastructure, agricultural and rural development), the sector outcomes were mixed. While the planned results for water supply and sanitation were achieved, most targets could not be attained in the agriculture and power sectors. In the power sector for example, the Bank reports that “unfinished projects absorbed significant fiscal resources without commensurate results”. Failure to meet planned results in both sectors was due to major delays, capacity gaps, and a deficient M&E system.

According to the extension of the CSP to 2011, however, there seems to be evidence of progress in the Bank’s contributions to infrastructure and agricultural development. The Bank for example contributed to the construction of 180 km of rural roads and trained close to 50 per cent of staff at the Federal Ministry of Agriculture in IT to improve sector statistics and management. Outputs were also recorded in the power sector. These improvements were largely confirmed by the first Quarterly Report on Activities from the Nigeria Field Office (2012).

The Field Office also indicated that a CSP Completion Report will be available later in 2012 and that a new CSP 2012-16 will be approved by the Board in November.

4.3.3 KPI C: Evidence of Contributions to Relevant MDGs

This KPI recognises that multilateral organisations have made commitments to the Millennium Development Goals (MDGs) and assumes that they explicitly articulate or make links to the MDGs that they contribute to at the country level. The MDGs are collective, global targets that have, in many cases, been adapted by partner countries in defining their priorities. While partner countries are responsible for making progress toward the MDGs, multilateral organisations ensure that their aid, knowledge, and other types of support facilitate achievement of these goals.

Given the extent of collaboration on MDGs at country level, the African Development Bank cannot take responsibility for the achievement of specific MDGs, but supports projects and programmes that contribute to the achievement of country-level MDGs. In the latest CSPs for four of the five countries sampled, there were attempts to link AfDB objectives to specific MDGs but reports explaining the Bank’s contributions to these were scarce (see figure 4.9).

Links between AfDB’s objectives and the MDGs

At an organisation-wide level, the AfDB has identified four focus sectors (Infrastructure, Governance, Private Sector and Higher Education) with specific applicability to regional integration, fragile states, middle income countries (MICs), and agriculture. Although the AfDB, like all other MDBs, has made commitments to the MDGs, it does not systematically link its development objectives at the organisation-wide level to specific MDGs.

However, the AfDB puts strong emphasis on economic growth as a basis for supporting progress towards the MDGs. From this perspective, it argues that by helping countries strengthen tax and non-tax revenues it helps to create additional fiscal space, fund essential infrastructure investments and social services that in the end will help reduce poverty.

Overall Assessment

Figure 4.9 shows the overall rating for this KPI based on the review of AfDB’s contribution to relevant MDGs – as expressed in AfDB reports and as indicated by stakeholders surveyed. It also shows the criteria that MOPAN used to assess each country and determine the overall rating.

Figure 4.9 KPI C: Evidence of Contributions to Relevant MDGs, Overall Rating and Criteria

1 A checkmark indicates that the organisation received survey ratings of strong or higher on the majority of MDGs and limited, if any, ratings of inadequate.

Evidence of AfDB’s contributions

Finding 27: Survey respondents provided mostly adequate ratings of the Bank’s contributions to MDGs. The document review found that while country level reports comment on progress towards national MDG targets, they rarely provide an explicit explanation of the Bank’s contribution or “facilitative” role in these areas.

Across all countries assessed, the survey ratings on this KPI were adequate but generally lower than ratings on the Bank’s contributions to country-level priorities (KPI B). This may be an indication that the AfDB focuses its support in areas that are less directly related to the MDGs than organisations with more specific, MDG-related, mandates.

All CSPs assessed make reference to the MDGs in general terms. One example is the CSP 2005-2009 for Ghana where it is stated that “the CSP results framework indicates the outcomes that the Bank expects to help the Government achieve over the CSP period. These outcomes were chosen in light of the MDGs, the priority indicators in the Government’s PAF matrix and the HIPC expenditure tracking indicators.” This is a typical statement and can be found in various versions in all the CSPs analysed. Most of the CSP results frameworks also link some, but not all, of their expected results to specific MDGs.

While all the country level reports reviewed comment on progress towards national MDG targets, they rarely provide an explanation of the Bank’s contributions to these. To some extent these links can be made by associating the results achieved with specific MDGs, but the links are rarely explicit. One could also argue that most of what the Bank supports contributes to MDG 1 – although the nature of this contribution has not been articulated.

In some cases, reports include explicit links between results achieved and progress towards the MDGs. In Morocco, for example, the CSP completion report links some of its CSP objectives to MDG 5 - Maternal Health. While it provides an overview of objectives and actual results achieved, the expected results seem to be identical to national MDG targets, which makes it challenging to assess the Bank’s contribution.

Overview of Survey and Document Review Data, by Country

DRC

When asked about AfDB DRC’s contributions to MDGs at the country level, 30 per cent of respondents rated the AfDB adequate or above, while 44 per cent provided a rating of inadequate or below. Although still low, the highest mean scores were recorded for AfDB contributions to DRC’s efforts in Goal 1 (Eradicate extreme poverty and hunger) and Goal 2 (Achieve universal primary education) where 39 per cent of the respondents rated the AfDB as adequate or above.

There were high levels of ‘don’t know’ responses (51 per cent overall) from in-country donors in all questions related to the MDGs. However, none of the clients answered “don’t know”.

AfDB’s CSP 2008-2012 is based on DRC’s Poverty Reduction and Growth Strategy Paper (PRGSP). While the PRGSP is designed with a clear focus on achieving the MDGs objectives, its results statements are not linked to specific MDGs. This is also the case for the Bank’s CSP 2008-2012 where none of the results statements are linked to specific MDGs. The Bank reported on progress towards the MDGs in DRC in the Mid-Term Review of the CSP, but it did not attempt to associate its contributions to specific MDGs. According to the Bank, the weakness of the national monitoring and evaluation systems in the DRC make it difficult to measure the Bank’s contribution.

Ghana

When asked about AfDB Ghana’s contributions to MDGs at the country level, 58 per cent rated the AfDB adequate or above. The highest mean scores were for AfDB’s contributions to Ghana’s efforts in Goal 1 (Eradicate extreme poverty and hunger) and Goal 3 (Promote gender equality and empower women) where 72 per cent and 79 per cent respectively provided ratings of adequate or above.

Among MOPAN donors in-country, 33 per cent responded ‘don’t know’ to all MDG questions apart from MDG 3 (Promote gender equality and empower women) where no one answered ‘don’t know’. Among clients there were particularly high ‘don’t know’ levels in relation to MDG 4 (Reduce child mortality) and MDG 5 (Improve maternal health). In both cases 47 per cent of the client respondents answered ‘don’t know’.

Most of AfDB Ghana’s CSP objectives are closely associated with MDGs but explicit links are only made in relation to education and health-related MDGs. Both the 2005-2009 CSP and the Mid-Term Review address Ghana’s progress towards the MDGs, but no attempts are made to explicitly link the Bank’s contributions to specific MDGs. Indeed, the bulk of AfDB’s work in Ghana is in the agriculture sector (30 per cent of the portfolio) and infrastructure (almost 26). These sector areas are not directly associated with the MDGs although links could be made to MDG 1.

Morocco

When asked about AfDB`s contributions to MDGs in Morocco, 47 per cent of respondents rated the AfDB adequate or above. The highest mean scores were for AfDB’s contributions to Morocco’s efforts in Goal 2 (Achieve universal primary education).

There were high levels of ‘don’t know’ responses on all questions related to the MDGs (30 to 45 per cent) apart from Goal 7 (Ensure environmental sustainability) where the level of ‘don’t know’ responses was 20 per cent. Among donors at country level between 40 and 60 per cent answered ‘don’t know’ to six out of the seven questions.

Niger

When asked about AfDB’s contributions to MDGs in Niger, only 41 per cent of the respondents rated the AfDB adequate or above. The highest mean score was for AfDB’s contributions to Niger’s efforts in Goal 1 (Eradicate extreme poverty and hunger), and the lowest mean score was for its contributions to Goal 4 (Reduce child mortality).

The overall level of ‘don’t know’ responses was high on all questions (between 34 and 47 per cent of the responses). Among donors at the country level, 50 per cent responded ‘don’t know’ on all questions.

The high level of ‘don’t know’ responses is interesting in light of the fact that some of the results statements from AfDB Niger’s CSP 2005-2009 are explicitly associated with specific MDGs – in particular Goal 1 (Eradicate extreme poverty and hunger) and Goal 7 (Environmental sustainability).

Nigeria

When asked about AfDB Nigeria’s contributions to MDGs at the country level, 48 per cent of respondents rated the AfDB adequate or above and 36 per cent rated it as inadequate or below. The highest mean scores were for AfDB contributions to the Nigeria’s efforts in Goal 4 (Reduce child mortality) and Goal 7 (Ensure environmental sustainability). There were high levels of ‘don’t know’ responses (35 per cent) in relation to Goal 5 (Improve maternal health). Generally the ‘don’t know’ levels were higher among clients than donors at country level.

AfDB Nigeria’s Strategic Framework Matrix from the CSP 2005-2009 links AfDB results statements with specific MDG goals more systematically than most other CSPs analysed. AfDB Nigeria reports on progress towards the MDGs, but less directly on AfDB’s contributions to this end. Indeed, since some results statements are very high level (e.g., “Infant and under five mortality rates per 1000 live births reduced to 18” or “Proportion of births attended by skilled health personnel increased to above 47 per cent”) it can be difficult to assess the extent of Bank contributions.

4.3.4 KPI D: Relevance of Objectives and Programme of Work to Stakeholders

For this KPI, MOPAN assessed relevance as a measure of the extent to which stakeholders perceive that the multilateral organisation supports country priorities and meets the changing needs of clients and the target population. The assessment is based exclusively on survey data gathered from clients and MOPAN donors in-country in the five countries selected for the test of this component focused on AfDB results.

Overall assessment

Across the five countries, the AfDB received mixed ratings from survey respondents but strong overall ratings for responding to client countries’ key development priorities. As noted in section 4.1, respondents from the DRC provided consistently lower ratings than respondents from other countries.

Figure 4.10 shows the overall assessment rating and the means scores on the three survey questions on which the assessment is based.

Figure 4.9 KPI D: Relevance of Objectives and Programme of Work to Stakeholders, Overall Rating and Survey Mean Scores by Country

Overall Assessment: Adequate

1 Detailed scores are shown in Volume II, Appendix VI.

5. Conclusion

This conclusion steps away from the specific ratings of the MOPAN assessment and looks at the major messages that can contribute to dialogue between MOPAN, the AfDB and its clients. It draws on the survey findings and principal observations of the assessment of AfDB’s practices and systems (Key Performance Indicators 1-19) and the assessment of development results component (Key Performance Indicators A-D).

AfDB stakeholders surveyed in 2009 and 2012 largely considered the Bank’s performance as adequate in most areas assessed. In both years, survey respondents indicated the need to increase its delegation of decision-making authority to the country level.

According to survey respondents, the Bank has made progress in human resource management practices as well as in its ability to adjust procedures to local conditions and capacities, both of which were rated inadequate in 2009 but adequate in 2012.

In human resource management, survey respondents particularly noted the Bank’s efforts to instil a culture that is transparent and meritocratic. In adjusting procedures, they noted some improvement in the Bank’s use of procedures that can be easily understood and completed by partners, its ability to quickly respond to changing circumstances, and its flexibility in the implementation of projects and programmes.

In both 2009 and 2012, surveyed stakeholders perceived the Bank’s procedures as too lengthy and noted the need for the Bank to increase country-level decision making.

The AfDB remains strongly committed to a results-focused reform agenda and is improving its capacity to manage for development results. However, the frameworks and systems that it uses to manage for and report on organisation-wide results need further improvement, particularly in formulating and presenting the intended links between lower level and higher level results.

As in previous MOPAN assessments, the AfDB continues to be recognised for its strong African identity and regional ownership, which has a positive impact on its legitimacy among African governments and its understanding of regional needs and priorities. Indeed, both survey respondents and the document review noted the strong link between AfDB’s strategy and its clear mandate. The Bank has been implementing an ambitious set of reforms in recent years – focusing, among other things, on instilling a results-oriented culture throughout the organisation. Although the Bank received relatively modest ratings from survey respondents on indicators related to its reform agenda, that agenda is starting to yield concrete results. With the establishment in 2008 of the Quality Assurance and Results Department (ORQR) – which is responsible for managing results reporting and implementing the Bank’s results agenda – the Bank has confirmed its commitment to, and has strengthened its capacity in, this area.

An important part of the Bank’s reform process included the revamping of its results measurement frameworks (RMFs), which since 2010 have been consolidated into one results framework known as the “One Bank” results measurement framework. The reform process also led to the introduction of the Annual Development Effectiveness Review (ADER), an organisation-wide report on development effectiveness. The Annual Development Effectiveness Review represents a major improvement in the Bank’s reporting on results and provides an annual overview of its performance according to core sector indicators. While this Review identifies some of the tangible outputs achieved by Bank-supported operations, evidence of contributions to higher level development outcomes in the region is still limited. There is a notable gap between aggregate outputs (and intermediate outcomes) achieved by Bank- supported projects and higher level outcomes in Africa. The Bank does not present a theory of change that could contribute to minimising the gap between these levels of results. This is a challenge that all multilateral development banks are facing.

The Bank is also in the process of developing Development Effectiveness Reviews (DERs), which are thematic and country reviews that analyse achieved results in particular areas of the Bank’s portfolio. It is anticipated that the publication of these reviews will provide a clearer indication of the Bank’s contributions to country level results as well as results achieved in several areas of focus.

The Bank is recognised for aligning its country strategies with national development priorities and its country strategies reflect a sound analysis of the country context. Assessing the extent to which the AfDB contributes to country level development outcomes remains a challenge.

Both survey respondents and the document review found that the Bank’s country strategies are clearly aligned with national Poverty Reduction Strategic Plans (or equivalent). Indeed, in all Country Strategy Papers (CSP) reviewed, there were clear and explicit links between AfDB’s expected results and those identified in national plans.

However, the quality and format of CSP results frameworks vary, and the use of results statements and indicators is inconsistent. Further, while most of these frameworks link expected outputs and outcomes to results at the country level, not all statements of results are appropriate to their results level, links between outputs and outcomes are not always clear, and some output statements are in fact outcome statements and vice versa. Therefore, it can be difficult to discern AfDB’s contribution to country level outcomes. In addition, none of the Country Strategy Papers analysed provide any explicit theories of change, and most of the results frameworks assessed are based on very high level results statements that are difficult to correlate with the results of the Bank’s operations. That being said, all Country Strategy Papers build on a solid analysis of the country context and more recent results frameworks have improved in quality and consistency as compared to the previous generation.

The AfDB is seen to have sound policies and processes for financial accountability.

The Bank received strong ratings for its policies and practices for audit and combating corruption. It is considered adequate in other areas, such as its risk management, procurement and contract management processes, as well as procedures undertaken to respond and follow up on irregularities.

AfDB’s human resource management is the focus of an on-going reform process. According to both document review and survey respondents, the Bank is still facing challenges in this area.

In 2009, the Bank adopted a new performance management system (PMS) and developed a handbook that outlines its practices for managing staff performance and performance review procedures for evaluating the performance of senior management in general (although it is not clear if this applies to the level of Vice Presidents).

There are still challenges related to the implementation of the performance management system, including delays in completing each stage of the performance cycle and a tendency towards overly high ratings. A recent review of this system noted several factors that hindered its full application and effectiveness, including: lack of management ownership, poor leadership, lack of people management skills among managers, and inadequate compliance mechanisms to enforce consistent use of the new system.

There is also a perceived lack of transparency in AfDB’s system to manage and reward staff performance. While the Bank has a performance-based pay system that rewards staff based on merit, the effectiveness of the system is dependent on the operationalisation of the performance management system.

The Bank has increased its country presence significantly in recent years through the establishment of additional field offices, but its delegation of decision making authority to the country level remains a work in progress.

As of 2012, the AfDB has a field presence in 34 out of 54 regional member countries and ensures its presence in other countries through its regional office and regional resource centres located in Dakar, Nairobi and Pretoria, respectively. Delegation of decision making is an operational area that has significant implications for the efficient and effective delivery of aid at the country level. While the Bank adopted a Roadmap[84] in 2011 that mapped out the its vision for transferring decision-making authority to Field Offices over the period 2011-2015, decision- making authority remains quite centralised, with very little authority having been delegated to country or regional levels. For example, project budget reallocations and project approvals cannot be made at the country level and both survey respondents and the document review considered the Bank inadequate in this area. However, it must be noted that for many multilateral development banks, the restructuring of projects must be approved by Boards of Directors.

The AfDB is currently considering allocating more decision-making authority to regional levels and has recently converted the Bank’s field offices in Nairobi and Pretoria to Regional Resource Centres (RRCs) that cover the eastern and southern Africa regions. It is also updating its Delegation of Authority Matrix and, according to OPEV, has made some progress in delegating authority for procurement to Field Offices.

The Bank has made progress on most of its Paris Declaration commitments. There is room for improvement in its use of country systems and its participation in joint missions and programme-based approaches.

According to the 2011 OECD/DAC Survey on Monitoring the Paris Declaration, the Bank has made progress in ensuring that aid flows are recorded in partner countries’ budgets, reducing the use of parallel project implementation units and disbursing technical cooperation through coordinated programmes.

However, less than 10 per cent of the Bank’s missions are being jointly undertaken, and the recent Quality of Official Development Assistance Assessment (QuODA) places the Bank among the multilateral development banks with the lowest share of coordinated joint missions. The 2011 OECD Survey and an AfDB evaluation from 2011 indicated that the Bank has made limited progress in the use of common arrangements or procedures, and the Bank received moderate ratings in the MOPAN assessment from both document review and survey respondents for its somewhat limited use of country systems.

According to the AfDB evaluation, the Bank’s participation in programme-based approaches is limited by its application of procurement rules of origin, lack of guidance to staff, and its generally cautious approach to financial management. However, the Bank’s cautious approach must also be considered in context as it has faced challenges in dealing with corruption and lack of government capacity in many Regional Member Countries.

Development Results Component

The Bank’s progress towards organisation-wide outcomes is not clear.

MOPAN donors at headquarters were positive about AfDB’s contributions to infrastructure development (in particular transport) and regional integration, but see its performance in other areas as adequate.

While AfDB’s data on results at the organisation-wide level is consolidated in the Annual Development Effectiveness Review, it does not give a complete picture of the Bank’s results achievement and some details are better presented in sector evaluations and studies. Reports provide evidence of progress towards planned lower level results in the One Bank results framework, but limited evidence of AfDB’s overall contribution to higher-level change.

Although the Bank’s country-level results are aligned with national priorities, the extent to which its operations contribute to the achievement of country-level goals or relevant Millennium Development Goals is not always clearly evident in its reporting.

At the country level, stakeholders gave the Bank generally strong ratings for responding to their countries’ key development priorities but mixed ratings on its overall relevance and its contributions to MDGs.

AfDB’s Country Strategy Paper results statements are aligned with national goals and priorities and the Bank reports on the country’s progress towards national goals. However, the Bank’s reports do not provide a clear picture of the nature, magnitude, or relative importance of AfDB contributions to these results and most reports do not describe the Bank’s role or contribution in support of its clients’ efforts to achieve the MDGs.

The Bank has been strengthening its country reporting tools and systems, which may lead to better evidence of its contributions to country development outcomes in the future.

---

[1] The AfDB, UNICEF, UNDP, and the World Bank

[2] African Development Bank Group. (2011). A Roadmap for Improving Performance on Aid Effectiveness and Promoting Effective Development - Turning Commitments Into Action.

[3] These organisations were selected because this is the second time that they have been assessed by MOPAN using the Common Approach Methodology. The 2009 assessment focused on organisational effectiveness and was based on survey data only.

[4] This section draws from information and documents provided on the AfDB website: http://www.afdb.org

[5] This component was tested in 2012 with the African Development Bank, UNICEF, UNDP, and the World Bank.

[6] The full methodology is presented in Volume II, Appendix I.

[7] MOPAN criteria for country selection include: multilateral organisation presence in-country, presence and availability of MOPAN members, no recent inclusion in the survey, the need for geographical spread, and a mix of low income and middle income countries (middle income countries being subdivided into lower middle and upper availability of MOPAN members, no recent inclusion in the survey, the need for geographical spread, and a mix of low income and middle income countries (middle income countries being subdivided into lower middle and upper middle).

[8] In the context of IFIs, these are referred to as “clients” and some organisations refer simply to “partners.”

[9] The normal convention for statistical significance was adopted (p≤.05).

[10] These organisations were selected because they were assessed by MOPAN in 2009. The 2009 assessment focused on organisational effectiveness and was based only on survey data.

[11] Different organisations use different terms to refer to their planned results – they may be called goals, objectives, outcomes, etc.

[12] The “best fit” approach is used in public sector institutions (see Ofsted, 2011: Criteria for making judgments)

[13] Respondents who wrote “no comment” or something similar were filtered out of the analysis.

[14] While most KPIs and micro-indicators were considered in the document review, not all were rated. See section 2.3.

[15] Where statistically significant differences among categories of respondents were found, these differences are noted.

[16] The African Development Bank (2011). Agreement Establishing the African Development Bank. Retrieved April 23 2012 from: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Legal- Documents/Agreement%20Establishing%20the%20ADB%20final%202011.pdf

[17] The African Development Bank Group (2011). Annual Development Effectiveness Review.(p.46) Retrieved 24 April 2012 from: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Project-and-Operations/ADER%20(En)%20-%20Websafe.pdf

[18] The African Development Group (2012). Annual Development Effectiveness Review. Retrieved from: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Project-and-Operations/ADER%202012%20(En).pdf

[19] At a basic level, a theory of change makes explicit the expected changes and how they are expected to come about. It provides a rationale for a certain way of working, the assumptions or beliefs about change, and can be articulated at different levels: sector, organisational, or project/programme level. This definition builds on definitions of programme theory in the evaluation literature (such as Weiss, 1997, p.55).

[20] However, it is our understanding that the Regional Integration Strategy is currently being evaluated and will be completed in 2012.

[21] The Gender and Social Development Monitoring Division, located in the quality assurance and results department (ORQR) of the Bank, was established in March 2010

[22] Operations Evaluation Department - African Development Bank Group (2012). Mainstreaming Gender Equality: A Road to Results or a Road to Nowhere? (p.32-34). Retrieved 28 May 2012 from: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Evaluation-Reports/Mainstreaming%20Gender%20Equality%20-%20An%20Evaluation%20Synthesis_www.pdf

[23] African Development Bank. (2011). Staff Guidance on the Implementation Progress and Results Reporting of Public Sector Operations

[24] African Development Bank. (March 2012). Integrated Safeguards System Working Progress : Strategic Choices Made in the Design of the Integrated Safeguards System (ISS. (p.12)

[25] African Development Bank Group (2010). The 2011-2013 Programme Budget. Retrieved 1 May 2012 from: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Boards-Documents/African%20Development%20Bank%20-%202011-2013%20Programme%20and%20Budget.pdf

[26] African Development Group (2010). The 2012-2014 Programme Budget. Retrieved 1 May 2012 from: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Budget-Documents/2012-2014%20Programme%20and%20Budgets.pdf

[27] African Development Bank Group (February 2010).Mid-Term Review of the 2008-2010 Business Plan for Private Sectors Operations. (p.9).

[28] Ibidem.(p.IV)

[29] African Development Fund (2010). Proposed Adjustments to the Enhanced Approach to Fragile States. Retrieved 3 May 2012 from: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Boards-Documents/DP%20Fragile%20States%20paper.FINAL3.EN.pdf

[30] African Development Bank Group – OPEV. (2012). Evaluation of the assistance of the African Development Bank to Fragile States.

[31] The document review acknowledged the Bank’s efforts to improve the measurement and reporting of results achieved in fragile states and conflict-afftected countries in the Development Effectiveness Review (DER). However, the DER for this particular area was still in draft form at the time of the assessment.

[32] The African Development Bank (2011). Agreement Establishing the African Development Bank. Retrieved April 23 2012 from: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Legal-Documents/Agreement%20Establishing%20the%20ADB%20final%202011.pdf

[33] The CSPs reviewed during the 2012 MOPAN Assessment covered the following years: DRC: 2008-2012; Ghana: 2005-2009 (extended to 2011); Morocco: 2007-2011 and 2012-2016; Niger: 2005-2009 (extended to 2012); Nigeria: 2005-2009 (extended to 2011)

[34] The Bank’s credit policy, which was developed in 1995, was built in alignment with IBRD criteria.

[35] African Development Bank Group. (2011). Transition Framework for Graduating and Reversing countries (p.6). Retrieved 22 May 2012 from: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Policy-Documents/-per cent20REV per cent20Transition per cent20Framework per cent20Graduating-Reversing per cent20Countries per cent20-.pdf

[36] Ibidem

[37] ADF Eligibility Criteria. http://www.afdb.org/en/about-us/african-development-fund-adf/adf-recipient-countries/

[38] Ibidem

[39] African Development Bank Group – OPEV. (2011). Evaluation of Paris Declaration Implementation at the African Development Bank Retrieved 10 May 2012 from: http://www.oecd.org/dataoecd/31/26/47832813.pdf

[40] African Development Bank (n.d) A Roadmap for Improving Performance on Aid Effectiveness and Promoting Effective Development - Turning Commitments Into Action: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Publications/Rev per cent20- per cent20Roadmap per cent20Aid per cent20Effectiveness.pdf

[41] Aid Effectiveness 2005-10: Progress in Implementing the Paris Declaration (2011). Retrieved from: http://www.oecd.org/dataoecd/25/30/48742718.pdf

[42] 2012 Annual Development Effectiveness Review (ADER): http://www.afdb.org/fileadmin/uploads/afdb/Documents/Project-and-Operations/ADER per cent202012 per cent20 per cent28En per cent29.pdf

[43] Ibidem. (p.36).

[44] This rating is based on the OECD/DAC definition of Results-Based Budgeting: “A budgeting method that links appropriations to outcome level performance targets. Expected results justify resource requirements and actual performance in achieving results is measured by predefined performance targets

[45] The 2012-2014 Programme and Budget.

[46] According the 2012-2014 Budget glossary of the Bank, ”UA budgeting is a set of budget reforms with expanded flexibility and responsibilities to Managers that aim to improve the efficiency and effectiveness of linking resource allocations to Work Programme planning and implementation. It emphasises the importance of well-defined and realistic Work Programmes as the basis of budget and staffing proposals and allocations. The starting point of the Work Programmes is the Bank Strategy, to which they must be aligned”

[47] African Development Bank. (2006). Guidelines for Financial management and Financial Analysis of Projects (p.105): http://www.afdb.org/fileadmin/uploads/afdb/Documents/Project-related-Procurement/GFA01_Guidelines per cent20for per cent20FG per cent20& per cent20FA per cent20of per cent20Proj.pdf

[48] African Development Bank. (2003).Guidelines for Preventing and Combating Corruption and Fraud.

[49] The African Development Bank. (2006). Uniform Framework for Preventing & Combating Fraud and Corruption (2006) - International Financial Institutions Anti-Corruption task Force: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Generic-Documents/30716700-EN-UNIFORM- FRAMEWORK-FOR-COMBATTING-FRAUD-V6.PDF

[50] The African Development Bank Group. (2006) The African Development Bank-Guidelines for Financial management and Financial Analysis of Projects, Chapter 5: Reporting and Auditing. (p.16)

[51] African Development Bank.(2012). Annual Development Effectiveness Review (ADER). (p.42) Retrieved from:

http://www.afdb.org/fileadmin/uploads/afdb/Documents/Project-and-Operations/ADER%202012%20%28En%29.pdf

[52] See the indicator “Lapse time for bidding completion” in Level 4 of ADER 2012.

[53] Ibidem. (p.43)

[54] African Development Bank. (2012) 2011 Annual Report. Retrieved from: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Publications/AfDB%202012%20EN_WEB.pdf

[55] African Development Bank Group. (2011). OPEV Proposed Three-year rolling work programme and budget 2012-

2014. (p.8). Retrieved from: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Evaluation-Reports/OPEV per cent20- per cent203-YrRolling per cent20Work per cent20Prog per cent20 per cent20Budget per cent202012-14 per cent20- per cent20Sept per cent20 per cent2013.pdf

[56] 61 The Three-Year Rolling Work Programme and Proposed Budget for 2012-2014 indicates that the current practice at the Bank is to conduct one CAE per year, but that the number of CAEs conducted should be increased to 2 or 4 given the scope of countries covered by the Bank.

[57] African Development Bank, Asian Development Bank, European Bank for Reconstruction and Development, et.al. (2011). Multilateral Development Bank’s Common Performance Assessment System: 2010 COMPAS Report. (p31). Retrieved from: http://www.mfdr.org/COMPAS/documents/2010_COMPAS-Report.pdf and 2009 COMPAS Report (p.24) Retrieved from: http://www.mfdr.org/COMPAS/documents/2009_COMPAS.pdf

[58] E.g., through the document “Staff guidance on Implementation Progress and Results reporting (IPR) for Public Sector Operations”, which includes detailed guidelines on project performance ratings and risk assessment and management.

[59] African Development Bank Group. (2011). OPEV Proposed Three-year rolling work programme and budget 2012-2014. (p.8)http://www.afdb.org/fileadmin/uploads/afdb/Documents/Evaluation-Reports/OPEV per cent20- per cent203-YrRolling per cent20Work per cent20Prog per cent20 per cent20Budget per cent202012-14 per cent20- per cent20Sept per cent20 per cent2013.pdf

[60] Guidelines for Financial Management and Financial Analysis of Projects. (p.56 -Chapter 3: Financial Analysis and Appraisal of Projects - section 3.6: project financial viability),): http://www.afdb.org/fileadmin/uploads/afdb/Documents/Project-related-Procurement/GFA01_Guidelines per cent20for per cent20FG per cent20 per cent26 per cent20FA per cent20of per cent20Proj.pdf

[61] For example, according to the Staff Guidance issued in 2011, prior to 2010 the logical frameworks often lacked a clear definition of project outputs and outcomes, baselines, and clear identification of targets to be met during the project implementation process.

[62] 2010 COMPAS Report.(p.27 - indicator 4) http://www.mfdr.org/COMPAS/documents/2010_COMPAS-Report.pdf

[63] African Development Bank Group. (2011) Evaluation of Paris Declaration Implementation at the African Development Bank (p.9-10): http://www.oecd.org/dataoecd/31/26/47832813.pdf

[64] The AfDB monitors the Paris Declaration indicators and reports annually on progress in four areas: alignment on country priorities (PD 3), predictability (PD 7), the use of country systems (PD 5a and 5b), and the use of parallel project implementation units (PD 6).

[65] World Bank – AfDB Common Approach Paper on “Providing Budget Aid in Situations Of Fragility.” It should be noted that this paper focuses on Budget Support Operations in fragile states only.

[66] A review of policy-based operations under ADF-11 was conducted as part of the ADF-12 replenishment process in 2010. According to this review, PBOs primarily consisted of budget support operations during the 2008-2010 time period (92 per cent of all PBOs by value). The remaining 8 per cent were provided for import support.

[67] Quality of Official Development Assistance Assessment. Birdsall & Kharas, 2010, Center for Global Development.

[68] African Development Bank, World Bank .(2010).Strengthening Country External Audit Systems in Africa A Joint Strategy of the Africa Development Bank and the World Bank: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Policy-Documents/24-EN-Audit_Systems_in_Africa_-_AfDB- WB.pdf

[69] The Evaluation Cooperation Group (ECG) of the MDBs has developed good-practice standards (GPS) and evaluation criteria for the evaluation of public and private investment operations, policy-based lending and non- lending operations, within the general framework of the OECD-DAC evaluation principles. Other good-practice standards are being prepared for the evaluation of country assistance strategies and technical assistance operations.

[70] For example, criteria such as SMART (specific, measurable, achievable, relevant and time bound) or CREAM (clear, relevant, economic, adequate and monitor-able).

[71] (i) Development resources recorded on budget; (ii) predictable disbursements; (iii) use of country systems; and (iv) parallel project implementation units.

[72] According to the Bank, the DER on Regional Operations will be published in November 2012.

[73] A draft of the first DER focusing on country programme, i.e., DER Rwanda, was also shared with the Assessment Team. Please see KPI B.

[74] AfDB participates in the Working Group on MfDR along with most other major MDBs. MDBs share and report on their MfDR practices through the annual COMPAS report. Most MDBs have adopted corporate results frameworks that aim to track results at four levels – from regional or global development progress to organisational

[75] In addition to RWSSI, the Bank is currently overseeing other initiatives in the water and sanitation area, including the African Water Facility (AWF) and the Water Partnership Program (WPP). The assessment of the Bank’s efforts in this section therefore does not undermine the significant efforts made by the Bank through these initiatives but rather places focus on one where the Bank’s contributions was observed the most. For more information on the AWF, please visit: http://www.africanwaterfacility.org/en/about-us/donors-and-funding/. Information on the WPP, can also be found here: http://water.worldbank.org/wpp

[76] Please refer to the sections in chapter 3 on KPIs 2, 4, 18 and 20 for the analysis of AfDB’s results-based systems and practices.

[77] Although Rwanda was not one of the five countries selected for the MOPAN assessment this year, the DER of the Rwanda country programme was shared with the Assessment Team in order to highlight the Bank’s on-going efforts to improve its reporting on country level results.

[78] The CSP results-based framework for Ghana was revised during the CSP Mid-Term Review due to these considerations. However, the assessment team has not been able to identify a performance report using the updated results-framework. According to the AfDB Country Office in Ghana, the new CSP, which is currently being prepared, will include a more complete results framework.

[79] Although the results statements are mostly formulated as outcomes, the indicators for measuring progress are a mixture of outputs and, to a lesser extent, outcomes.

[80] The Emergency Programme was not captured in the CSP 2008–2012 and respondents were not asked questions related to this initiative.

[81] Baselines are not presented in the CSP 2007-2011 which makes it difficult to assess the extent of the improvement in this area.

[82] Ibid

[83] Evaluation of Policy Based Operations in the African Development Bank, 1999-2009 – Country Case Study: Morocco

[84] African Development Bank Group. (2011). A Roadmap for Improving Performance on Aid Effectiveness and Promoting Effective Development - Turning Commitments Into Action.