Preface

The Multilateral Organisation Performance Assessment Network (MOPAN) is a network of donor countries with a common interest in assessing the organisational effectiveness of multilateral organisations and their measurement and reporting on development and/or humanitarian results. MOPAN was established in 2002 in response to international forums on aid effectiveness and calls for greater donor harmonisation and co-ordination.

Today, MOPAN is made up of 17 donor countries: Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Ireland, The Netherlands, Norway, Republic of Korea, Spain, Sweden, Switzerland, the United Kingdom, and the United States. For more information on MOPAN and to access previous MOPAN reports, please visit the MOPAN website (www.mopanonline.org).

Each year MOPAN carries out assessments of several multilateral organisations based on criteria agreed by MOPAN members. Its approach has evolved over the years, and since 2010 has been based on a survey of key stakeholders and a review of documents of multilateral organisations. MOPAN assessments provide a snapshot of four dimensions of organisational effectiveness (strategic management, operational management, relationship management, and knowledge management). In 2013, MOPAN has integrated a new component to examine the evidence of achievement of development and/or humanitarian results to complement the assessment of organisational effectiveness.

MOPAN 2013

In 2013, MOPAN assessed four multilateral organisations: the Asian Development Bank (ADB), the International Fund for Agricultural Development (IFAD), the World Food Programme (WFP) and the World Health Organization (WHO).

MOPAN Institutional Leads liaised with the multilateral organisations throughout the assessment and reporting process. MOPAN Country Leads monitored the process in each country and ensured the success of the survey.

Acknowledgements

We thank all participants in the MOPAN 2013 assessment of WFP. WFP’s senior management and staff made valuable contributions throughout the assessment and document review processes and provided lists of their partners to be surveyed. Survey respondents contributed useful insights and time to respond to the survey. The MOPAN Institutional Leads, Finland and Switzerland, liaised with WFP throughout the assessment and reporting process. The MOPAN Country Leads oversaw the survey planning process in the field and ensured the success of the survey. Consultants in each country provided vital in-country support by following up with country-level survey respondents to enhance survey response rates.

Roles of authors and the MOPAN Secretariat

The MOPAN Chair was held by Australia in 2013 and worked in close co-operation with the MOPAN Technical Working Group to launch and manage the survey. Once the MOPAN Secretariat was established in April 2013, the Head of the Secretariat oversaw all related tasks.

MOPAN developed the key performance and micro-indicators, designed the survey methodology, co- ordinated the preparation of the lists of survey respondents, and approved the final survey questionnaire for each agency. MOPAN also directed the approach to the document review and oversaw the design, structure, tone, and content of the institutional reports.

Universalia and Epinion developed the survey instrument and carried out the survey and analysis. Universalia carried out the document review, conducted the interviews with multilateral organisation staff at headquarters and country levels, and wrote the institutional reports.

Epinion is a consulting firm in Denmark that analyses and evaluates data to support decision making. It conducts specially designed studies for public and private organisations based on data collected among an organisation’s employees, members, customers, partners, and other sources. Epinion has 75 employees and 200 interviewers. Website: www.epinion.dk

Universalia Management Group is a Canadian consulting firm established in 1980 that specialises in evaluation and monitoring for international development. Universalia has made significant contributions to identifying best practices and developing tools in the fields of organisational assessment; planning, monitoring, and evaluation; results-based management; and capacity building. Website: www.universalia.com.

Acronyms

Executive summary

This report presents the results of an assessment of the World Food Programme (WFP) conducted by the Multilateral Organisation Performance Assessment Network (MOPAN). The MOPAN Common Approach examines organisational systems, practices, and behaviours that MOPAN believes are important for aid effectiveness. It also examines the extent to which there is evidence of an organisation’s contributions to development and/or humanitarian results, and relevance to stakeholders at the country level.

WFP, established in 1961, is the largest humanitarian agency fighting hunger worldwide. It is characterised by a dual mandate to avert starvation in humanitarian crises through food assistance delivered through emergency operations that fill food gaps in the short-term, and also via programmes that promote long- term development and thereby break the deeply rooted hunger-poverty cycle.

In response to the changing and evermore global nature of the hunger challenge, WFP’s Strategic Plan for the 2008-2013 period repositioned the organisation from a food aid to a food assistance agency, expanding the range of approaches and instruments WFP uses to help the food insecure, such as vouchers and cash transfers, new food products, and local purchase schemes.[1]

WFP relies entirely on voluntary contributions to carry out its programme of work, having no pre-existing budget generated by assessed contributions, dues, core funding or financial endowments (unlike most other UN funds and programmes). Other than for a small proportion of funding received through multi- year agreements, WFP must raise the bulk of the resources required to meet its projected needs on an annual basis.

Since 2012, WFP has been undergoing reforms to strengthen the organisation. Major change processes currently underway include a restructuring of the organisation to better align its design around strategic priorities, the strengthening of resource management at the country level, the comprehensive review of human resource core processes, development of new approaches to communication and learning, and implementation of a single, streamlined system to improve performance monitoring and reporting.

MOPAN Assessment

The MOPAN assessment framework was adapted for organisations with a humanitarian mandate in 2011, following tests conducted with WFP. However, this is the first time that WFP has been formally assessed by MOPAN.

In 2013, the MOPAN assessment of WFP was based on information collected through a survey of key stakeholders, a review of documents, and interviews with WFP staff. The survey targeted MOPAN donors at headquarters as well as MOPAN donors, recipient governments, direct partners and peer organisations based in five countries (i.e. Ethiopia, Guatemala, Indonesia, Mozambique and Pakistan).[2] A total of 298 respondents participated in the WFP survey (44 MOPAN donors based at headquarters, 41 MOPAN donors based in-country, 92 direct partners, 68 recipient government respondents and 53 peer organisation respondents). The document review examined close to 400 publicly available corporate documents as well as programming documents from the five surveyed countries. MOPAN also interviewed WFP staff members (45 from WFP’s headquarters and 9 from country offices). This information was not coded or used formally as part of the assessment process, but rather to gain a broader contextual understanding of the organisation’s systems, practices and results-related reporting.

MOPAN assessments provide a snapshot of four dimensions of organisational effectiveness (strategic management, operational management, relationship management, and knowledge management), and of contributions to development and/or humanitarian results. The main findings of the 2013 assessment of WFP in these performance areas are summarised below.

Key Findings

Strategic Management

In the area of strategic management, MOPAN established criteria to determine if a multilateral organisation has strategies and systems in place that reflect good practice in managing for results. Overall, the 2013 assessment found that:

• WFP’s commitment to results-based management spans nearly two decades; the organisation continues to strengthen its results focus, in particular with its ongoing emphasis on improving performance monitoring and reporting. The majority of surveyed stakeholders perceived senior management’s leadership and WFP’s value system as strongly supporting a results orientation.

• Although survey respondents rated WFP strong for having a corporate strategy based on a clear mandate, many of them also indicated in written comments that WFP’s dual mandate is a source of some confusion and/or concern. Both of these issues have also been noted in recent independent evaluations. The document review found that the organisation’s shift from food aid to food assistance has not yet been captured in WFP’s mission statement and has not been adequately communicated to stakeholders.

• WFP’s results frameworks at the corporate and country levels were found to be of sound design and to generally respect results-based management standards. The results chains could be strengthened, particularly in the corporate management results framework.

• Survey respondents rated WFP adequate or better in mainstreaming the seven cross-cutting issues assessed by MOPAN (gender equality, climate risk analysis and response, building capacities for good governance, human rights-based standards, emergency preparedness and response, protection, and HIV/AIDS) and considered its emergency preparedness and response an organisational strength. The document review concurred on most issues, but noted room for WFP to strengthen mainstreaming of gender equality in its work and the need for formal directives that require the consistent application of human rights-based standards in its development programming.

• WFP was considered strong in both the survey and documentary assessment for its focus on humanitarian and development results in-country. Respondents highlighted WFP’s contingency planning, needs assessments, and consultations with partners but noted room for improvement in the inclusion of beneficiaries in the design of humanitarian responses. The document review indicated that WFP’s approach to contingency planning has recently been redesigned but that it is too early to assess the effectiveness of these new measures.

Operational Management

In operational management, MOPAN established criteria to determine if a multilateral organisation manages its operations in a way that supports accountability for results and the use of information on performance.

Overall, the 2013 assessment found that:

• WFP’s resource allocation processes were rated adequate overall by respondents and the document review. The organisation has strengthened the allocation of un-earmarked funds and has improved the predictability of its assistance to countries through advance financing mechanisms.

• Survey respondents perceived WFP’s results-based budgeting practices as adequate; the document review found that the organisation does not yet have a system in place to track allocations and disbursements from activities through to outcomes, but that it is taking steps to align its programme and management results frameworks to its budget.

• WFP received consistently high ratings in the area of financial accountability given its strong internal and external audit functions, procurement and contract management processes, as well as strategies for managing risk. To further strengthen its performance, the organisation is developing a communication strategy that will inform staff of its anti-corruption policy and better communicate results of investigations of alleged fraud; it has also dedicated additional resources to improve timely implementation of audit recommendations.

• WFP’s use of performance information was considered adequate by surveyed stakeholders. The document review found WFP strong overall in its use of performance information; it may take some time to fully implement the organisation’s 2012 programme design guidelines on using performance information in planning new interventions.

• WFP’s human resources management was considered adequate or better in most areas, and the organisation was recognised for its strong measures to ensure staff security and enforce a code of conduct. WFP’s system to manage staff performance, assessed by document review only, was found to lack transparency, thus undermining the credibility of reassignment, promotion and recruitment decisions.

• WFP was considered adequate for subjecting new initiatives to risk and benefit/impact analyses before approval; the document review noted that a corporate directive on risks to be considered and guidelines on what to include in project proposals have been rolled out but have not yet been fully institutionalised.

• WFP does not have guidance in place to ensure that country office work plans or operational plans for projects/programmes consistently include the steps to be accomplished (with milestones, baselines and targets) to achieve the intended humanitarian/development results.

• Survey respondents and the document review both considered WFP as strong in delegating authority to local levels. The organisation is making additional improvements in this area by further strengthening regional bureaux and empowering country offices to perform their tasks.

• WFP’s approach to applying humanitarian principles in its operations was perceived as strong by survey respondents. The document review recognised the challenges in balancing a principled humanitarian stance with ensuring rapid food delivery, yet noted that further progress could be made by WFP to inform staff and provide training on implementation of the humanitarian principles during emergencies.

Relationship Management

In relationship management, MOPAN established criteria to determine if a multilateral organisation is engaging with its partners at the country level in ways that contribute to aid effectiveness. Overall, the 2013 assessment found that:

• WFP was considered adequate for taking local conditions and capacities into account during its operations. It was recognised in particular for its timely response to disasters and other rapid onset situations, for its sufficient use of local capacities and resources, and for seizing opportunities to procure food locally or regionally when cost-effective, timely and feasible. The document review observed room for improvement in WFP’s programming to enhance livelihoods, which is not yet leading to the desired changes in the long-term (i.e. to increased self-reliance of targeted beneficiaries in the recovery and development phases).

• WFP was seen as a strong contributor to humanitarian inter-agency plans and appeals by surveyed respondents and the document review found that WFP plays a key role in the Consolidated Appeals Process (CAP).

• WFP was seen as making strong contributions to policy dialogue and as respectful of partner views.

• WFP’s participation in the humanitarian cluster system was generally viewed as strong by survey respondents, though they noted room for improvement in the reliability of the organisation’s forecasts of financial needs for the clusters it leads or co-leads. The document review noted that the organisation is actively engaged in strengthening accountability mechanisms for cluster implementation.

• WFP was considered strong for harmonising its procedures with those of its programming partners. However, in the absence of documented corporate-level data for the Paris Declaration indicators, the document review could not effectively determine the extent to which WFP’s technical co-operation is disbursed through co-ordinated programmes.

Knowledge Management

In knowledge management, MOPAN established criteria to determine if a multilateral organisation has reporting mechanisms and learning strategies that facilitate the sharing of information inside the organisation and with the humanitarian and/or development community. Overall, the 2013 assessment found that:

• WFP was acknowledged for having strong processes in place to ensure the independence and quality of evaluations managed by the Office of Evaluation. In terms of decentralised evaluations, the document review found that evaluation coverage and quality assurance mechanisms are not yet sufficient, but that these areas are receiving attention from senior management.

• Survey respondents considered WFP strong in reporting on outcomes and against the results in its organisation-wide strategy. The document review commended WFP for reporting on how performance information is used to adjust its policies and strategies, but considered it inadequate in its reporting on outcomes and on its use of performance information to adjust programming at the country level. The organisation is making efforts to address some shortcomings by improving monitoring and evaluation capacities in the field.

• WFP was considered adequate for disseminating lessons learned. The document review acknowledged WFP’s recent initiatives to enhance the dissemination of lessons learned, but noted that the organisation does not yet have an integrated system for knowledge management.

• WFP was seen as a transparent organisation and rated very strong for making key documents readily available to the public.

Humanitarian/development results and relevance to stakeholders

In the 2013 humanitarian and development results component, WFP was rated adequate in providing evidence of progress towards organisation-wide results (KPI A), and for its contribution to country-level goals and priorities (KPI B). Survey respondents rated WFP adequate for the relevance of its objectives and programme of work to country level stakeholders (KPI C). These findings should be considered in conjunction with the findings above on WFP’s systems and practices for organisational effectiveness.

• Evidence of extent of progress towards organisation-wide results: WFP evaluations attest to WFP’s strong performance in achieving immediate, short-term results in emergencies, but highlight challenges in assessing achievement of medium and longer-term results. Lack of available data on outcome-level results is consistently mentioned in reports and evaluations as impeding assessment of WFP’s effectiveness; this finding was corroborated by the MOPAN analysis of WFP’s corporate data. Nevertheless, WFP is recognised for having made considerable improvements in its organisation-wide reporting (including on outcomes) over the 2008-2013 strategic plan, and for its continued commitment to monitoring and reporting. Demonstrating clear evidence of higher level results remains challenging for organisations across the humanitarian system.

MOPAN donors at headquarters considered WFP’s performance very strong in saving lives and protecting livelihoods but as adequate in its progress towards its other four strategic objectives.

• Evidence of extent of contribution to country-level goals and priorities: There is evidence across the sample of five countries that WFP is achieving its expected outputs. However, the document review noted room for improvement in WFP’s reporting at the outcome level, which makes it difficult to appreciate the overall effectiveness and contribution of its programming. Overall across the five sampled countries, WFP’s performance in delivering its intended results was rated adequate by survey respondents.

• Relevance of WFP’s objectives and programme of work to stakeholders: Surveyed country-level stakeholders perceived WFP to be strong in responding to the key development priorities of countries in which it has field operations. They considered it adequate in providing innovative solutions to help address these challenges and adapting its work to the changing needs of countries.

Conclusions on organisational effectiveness

Surveyed stakeholders reported different views on what constitutes WFP’s core mandate, on the agency’s recent shift to food assistance, and on the type of programming that WFP should focus on in the future.

Nearly a quarter of survey respondents cited the organisation’s mandate and its strategic shift to food assistance as areas of concern. Although WFP’s dual humanitarian and development mandate dates back to its establishment in 1961, the organisation is primarily regarded as a humanitarian agency. Donors, in particular, appear concerned that WFP’s recent shift to food assistance may extend the organisation further into development programming and result in a duplication of roles and responsibilities with other United Nations agencies. While some donors at headquarters suggested that WFP should concentrate nearly all efforts on its humanitarian mandate by providing assistance during emergencies, some recipient government and peer organisation respondents suggested that WFP should strengthen its development work and enhance activities in livelihood and resilience-building. Evaluations note that there is a need for WFP to clarify and better communicate its mandate in response to stakeholder concerns.

WFP has built a strong reputation for responding quickly to emergencies, and documented evidence confirms the strength of its practices and systems for launching and sustaining humanitarian interventions.

Stakeholders expressed confidence in WFP’s capacity to deliver assistance in emergencies; their comments highlighted the organisation’s logistical capabilities, expansive coverage and deep field presence as organisational strengths. The MOPAN assessment found that WFP is well-positioned to deliver assistance in emergencies and has a strong comparative advantage in humanitarian settings due to many noted strengths: WFP’s strong investment and focus on emergency preparedness and response across the organisation, the reliability of needs assessments and their use to inform programming, robust security measures to protect staff, effective procurement practices, strong risk management strategies, timely response to events and disasters, harmonised procedures with programming partners, and active contribution to inter-agency plans and appeals.

WFP’s performance in implementing the seven cross-cutting themes assessed by MOPAN was generally considered adequate. Areas requiring attention include gender mainstreaming throughout WFP’s work and the need for formal directives that require the consistent application of human rights-based standards in its development programming.

Although WFP formulated a policy and action plan to improve gender equality mainstreaming following an evaluation in 2008, in practice there have been insufficient dedicated financial and human resources to ensure their implementation. As a result, there are gaps in staff capacity to translate mainstreaming efforts into practice, with few projects qualifying as gender sensitive according to the Inter-Agency Standing Committee gender marker.

In its humanitarian work, which constitutes the bulk of WFP’s portfolio, WFP has developed a policy, implementation approach, guidelines and training, and defined roles and responsibilities on protection to ensure the application of humanitarian principles and standards. On the development side, however, WFP’s commitment to integrating human rights-based standards could be strengthened. The organisation indicates that its protection policy presents standards that are coherent with those found in the UN Common Statement of Understanding on Human Rights-Based Approaches to Development Co-operation and Programming (2003), and that its protection approach and guidelines are thus also applicable in development settings. While some offices have initiated measures to integrate protection standards into their development programming and have trained staff accordingly, the assessment team notes that there are currently no formal directives requiring country offices to mainstream the protection approach in their development work.

Results-based budgeting, which remains challenging for all multilateral organisations, was identified as an area requiring improvement.

Budgeting by results is intended to link expected or achieved results with allocations and disbursements to improve transparency, efficiency (value for money), and accountability with respect to diverse programmes.

WFP currently presents allocations and expenditures by strategic objective, but not yet by all outputs and outcomes. The organisation is committed to performance-informed budgeting and to aligning its performance planning and budgeting process fully in the coming years. In 2014, WFP will require each division or unit to articulate a results statement and to cost all contributing activities for the 2014 management plan.

WFP is committed to results-based management and has made considerable progress in improving the quality of its results frameworks and indicators. However, the organisation is not yet able to collect sufficient data to fully report on these frameworks due to gaps in staff and partner capacity and additional challenges on the ground.

The development of WFP’s strategic results framework and associated indicators has been an iterative learning process that has required country offices to make adjustments and adaptations to keep pace with changes and new demands. While the changes in indicators over time have been positive, this has also made it difficult to compare data across years. In addition, the organisation has been unable to report on the targets for outcome indicators presented in the corporate strategic results framework as the evidence from project reporting has been limited. The organisation is working to address these issues through its 2012-2014 strategy on monitoring, evaluation and reporting. It may take some time for results of this strategy to be achieved organisation-wide, especially if further changes are made to the results framework and indicators as part of the next strategic cycle (2014-2017).

As WFP moves to a new strategic cycle, the transition to the new strategic plan (2014-2017) will likely bring about substantial changes in WFP’s systems and practices.

The transition period, which will necessarily accompany the change from one cycle to the next, represents an opportunity for the organisation to revise and adjust procedures that go beyond the current MOPAN assessment. For example, the recently conducted rapid organisational assessment, the on-going change management process, and lessons learned from implementing the 2008-2013 strategic plan have made WFP quite aware of the weaknesses that it must address.

Conclusions on evidence of humanitarian and development results

WFP’s corporate performance reports do not yet provide a holistic account of the organisation’s results achievement, as the evidence base is often too limited to confidently demonstrate contributions to outcome-level change.

Both survey respondents and the document review considered the organisation’s ability to deliver immediate, short-term results during humanitarian emergencies (i.e. distributing food to those in need and saving lives) to be key strengths. However, limitations in the availability of data currently constrain the degree of confidence that can be given to conclusions on achievement of outcome and strategic objective-level change throughout WFP’s programming, Challenges in measuring and reporting on higher- level results are common among organisations in the humanitarian system. The 2012 report by the Accountability and Performance in Humanitarian Action (ALNAP) on the state of the humanitarian system indicated that there are very few examples of outcome/impact evaluations of humanitarian assistance, and that humanitarian organisations are primarily reporting evidence of outputs. Nevertheless, WFP’s evidence of higher-level results should improve in the coming years as the organisation continues to build capacity to monitor and report on outcomes in the field.

At the country level, there is evidence that WFP is achieving its expected outputs and making some contributions at the outcome level. However, a number of factors affect WFP’s measurement of results at the country level.

The document review found clear evidence of the achievement of expected outputs and some evidence of contribution to outcome achievement at the country level. Across the five sampled countries, WFP’s overall performance in delivering its intended results was rated adequate by surveyed stakeholders.

WFP generally operates in challenging and often volatile environments (e.g. conflict or war-torn regions and countries experiencing recurrent climatic shocks) where the delivery of food or non-food items is not simple and where needs may fluctuate significantly and rapidly. For instance, it is not uncommon for the number of beneficiaries within a targeted zone to increase midway through an operation, which can seemingly reduce evidence of earlier progress. Many other factors add to the complexity of WFP’s work both in delivering products and services and in measuring results: upsurge of violence limiting access to sites, disease outbreaks such as cholera, beneficiaries’ inability to travel to distribution sites (e.g. inability to attend school), short duration of WFP operations, absence of baseline data in emergencies, limited capacity of implementing partners to monitor results, etc. A key challenge in capturing outcomes as well as outputs is that food, which is the main commodity provided by WFP, is only one of many factors that affect food security and nutritional status.

Country offices often have limited resources and struggle to balance the humanitarian imperative to save the most lives possible with the need to dedicate resources to monitoring and evaluation to prove the effectiveness of its programming. Nevertheless, in order to demonstrate results achievement and attract continued funding, a strong monitoring and evaluation function at the country level is necessary. Though the sample of reviewed countries was very small, there is some evidence that efforts are being made at the country level to improve monitoring and evaluation systems.

Overall MOPAN Ratings of WFP

The two charts below show the ratings on the key performance indicators that MOPAN used to assess WFP in 2013. The first chart shows the ratings on 22 indicators designed to measure organisational effectiveness (practices and systems), and the second chart shows ratings on the three indicators designed to assess WFP measurement and reporting on humanitarian and development results. The indicators were adapted to reflect WFP’s dual mandate of humanitarian and development work.

Organisational effectiveness– overall ratings

Evidence of contribution to humanitarian and development results and relevance to stakeholders – overall ratings

1. Introduction

This report presents the results of an assessment of the World Food Programme (WFP) that was conducted in 2013 by the Multilateral Organisation Performance Assessment Network (MOPAN).

1.1 OVERVIEW

Background

MOPAN was established in 2002 in response to international fora on aid effectiveness and calls for greater donor harmonisation and co-ordination. The purpose of the network is to share information and experience in assessing the performance of multilateral organisations. MOPAN supports the commitments adopted by the international community to improve the impact and effectiveness of aid as reflected in the Paris Declaration on Aid Effectiveness, the Accra Agenda for Action, and the Busan High Level Forum. MOPAN’s processes and instruments embody the principles of local ownership, alignment and harmonisation of practices, and results-based management (RBM).

MOPAN provides a joint approach (known as the Common Approach) to assess the organisational effectiveness of multilateral organisations and their measurement of and reporting on humanitarian and development results. The approach was derived from existing bilateral assessment tools and complements and draws on other assessment processes for development organisations – such as the bi-annual Survey on Monitoring the Paris Declaration on Aid Effectiveness and annual reports of the Common Performance Assessment System (COMPAS) published by the multilateral development banks. In the long term, MOPAN hopes that this approach will replace or reduce the need for other assessment approaches by bilateral donors.

MOPAN assesses four dimensions of organisational effectiveness, and evidence of contribution to development and/or humanitarian results

MOPAN has defined organisational effectiveness as the extent to which a multilateral organisation is organised to contribute to development and/or humanitarian results in the countries or territories where it operates.

Based on a survey of stakeholders, a review of documents and interviews with multilateral organisation staff, MOPAN assessments provide a snapshot of a multilateral organisation’s effectiveness in four dimensions (see Chapter 3):

• Developing strategies and plans that reflect good practices in managing for humanitarian and development results (strategic management)

• Managing operations by results to support accountability for results and the use of information on performance (operational management)

• Engaging in relationships with direct partners/clients and donors at the country level in ways that contribute to aid effectiveness and that are aligned with the principles of the Paris Declaration and subsequent related agreements on aid effectiveness (relationship management)

• Developing reporting mechanisms and learning strategies that facilitate the sharing of knowledge and information inside the organisation and with the development community (knowledge management).

In 2012, MOPAN piloted a new component to assess a multilateral organisation’s reported contributions to development results. This component, which has become an integral component of all assessments in 2013, examines three areas of performance: evidence of the extent of the progress towards its institutional/organisation-wide results, evidence of contributions to country-level goals and priorities (including relevant MDGs), and stakeholder perceptions of the relevance of its objectives and programme of work. See Chapter 4.

Purpose of MOPAN assessments

MOPAN assessments are intended to:

• Generate relevant, credible and robust information MOPAN members can use to meet their domestic accountability requirements and fulfil their responsibilities and obligations as bilateral donors

• Provide an evidence base for MOPAN members, multilateral organisations and their partners/clients to discuss organisational effectiveness and reporting on development and/or humanitarian results

• Support dialogue between individual MOPAN members, multilateral organisations and their partners/clients to build understanding and improve organisational performance over time at both country and headquarters level.

The MOPAN methodology is evolving in response to what is being learned from year to year, and to accommodate multilateral organisations with different mandates. For example, it has adapted the indicators and approach for the review of global funds and organisations with a predominantly humanitarian mandate.

1.2 PROFILE OF WFP

WFP was established in 1961 by the Food and Agriculture Organization (FAO) and the United Nations General Assembly as an autonomous joint subsidiary programme.

Mission and mandate

As highlighted in its mission statement (1994), WFP is the food aid arm of the United Nations system. It provides food aid to eradicate hunger and poverty, with its ultimate objective being to eliminate the need for such aid.

WFP distinguishes itself from other UN agencies by its dual humanitarian and development mandate which dates back to its origins in 1962.[3] As stipulated in Article II of its General Regulations and Rules (2010), the organisation’s key functions are: a) to use food aid to support economic and social development; b) to meet refugee and other emergency and protracted relief food needs; and c) to promote world food security in accordance with the recommendations of the United Nations and FAO. Consistent with its mandate, WFP provides transport and logistical expertise to ensure that humanitarian aid is delivered both rapidly and effectively.

Strategy and objectives

WFP’s Strategic Plan for 2008-2013 marked a historical shift for the organisation. WFP repositioned itself and broadened its focus from solely delivering food aid to wider food assistance – characterised by an expanded range of approaches and instruments such as vouchers and cash transfers, new food products, and local purchase schemes. This shift to food assistance was made by WFP to respond to the changing and evermore global nature of the hunger challenge; new drivers of food insecurity, such as globalisation and climate change are combining with older obstacles, such as civil strife, inequality and weak governance, and threatening hard-won progress in the fight against hunger.[4] As part of its food assistance approach, WFP’s on-going strategic plan called for increased analysis of the causes of hunger to help define responses that are appropriately tailored to the local context. The shift is regarded as one of the most important internal changes for WFP since it was founded, and is requiring significant adaptations at all levels to implement.[5]

• WFP’s current strategic plan includes the following five programming objectives:

• Save lives and protect livelihoods in emergencies;

• Prevent acute hunger and invest in disaster preparedness and mitigation measures;

• Restore and rebuild lives and livelihoods in post-conflict, post-disaster or transition situations;

• Reduce chronic hunger and undernutrition; and

• Strengthen the capacities of countries to reduce hunger, including through hand-over strategies and local purchase.

The organisation’s next strategic plan, which will cover the 2014-2017 period in line with the Quadrennial Comprehensive Policy Review, was officially approved by WFP’s Executive Board at its Annual Session in June 2013.[6]

Structure and governance

Headquartered in Rome, WFP employs approximately 15 000 people, most of whom are deployed in the field. In 2012, WFP provided food assistance to 97.2 million people in more than 80 countries in Sub- Saharan Africa, Asia, Eastern Europe, Latin America, and the Middle East and North Africa.[7]

WFP is composed of two main organs: the Executive Board and the Secretariat. It is governed by its Executive Board (hereinafter referred to as ’the Board’), which operates under the authority of the Economic and Social Council of the United Nations (ECOSOC) and the Council of FAO. On a three year basis, the latter two entities elect the members of the Board, which is composed of 36 member states. The Board, which meets in Rome on a quarterly basis, is responsible for co-ordinating food aid policies, providing intergovernmental directions for the management of the organisation, approving budgets and monitoring the performance of programmes, projects, and activities, and reporting annually to ECOSOC and the Council of FAO. The Secretariat is headed by an Executive Director who is responsible for the overall administration of the organisation, as well as for the implementation of its programmes, projects, and activities. The Executive Director is appointed every five years by the UN Secretary General and the Director- General of FAO.[8]

Services

In order to fulfil its dual mandate, WFP provides food assistance to countries during and after emergencies (through emergency as well as protracted relief and recovery operations), and in support of economic and social development (through country programmes), offering for instance technical assistance to governments to help them establish or strengthen national disaster preparedness or food assistance programmes. WFP also carries out special operations to scale up infrastructure in support of existing emergency and protracted relief interventions.

In addition, WFP provides UN agencies, bilateral donors, and NGOs with support – mainly of a logistical nature – to help them fulfil their roles in the humanitarian assistance system. It offers the entire humanitarian community passenger and cargo air services through the United Nations Humanitarian Air Service (UNHAS), as well as logistic hubs for the storage of food and non-food items, through the United Nations Humanitarian Response Depot (UNHRD).[9]

Finances

WFP relies entirely on voluntary contributions to carry out its programme of work, having no pre-existing budget generated by assessed contributions, dues, core funding or financial endowments (unlike most other UN funds and programmes). Other than for a small proportion of funding received through multi- year agreements (see details below), the organisation must raise the bulk of the resources required to meet its projected needs on an annual basis, which has proved challenging.

In 2012, projected needs for WFP’s programme of work were estimated at USD 6.84 billion, but contributions received totalled USD 3.95 billion – this represents a 42% funding gap. A large proportion of WFP’s resources come from pooled funding mechanisms such as the Central Emergency Response Fund (CERF), the Common Humanitarian Funds (CHF), and other United Nations common funds. To palliate the unpredictability of its funding and diversify its donor base, WFP adopted a private partnership and fundraising strategy in 2008; in 2012 contributions from the private sector totalled USD 64.4 million, making it among WFP’s top fifteen donors. WFP has also sought strategic partnerships and multi-year arrangements with a number of bilateral donors and these efforts have resulted in increased un-earmarked contributions, which represented 10.6% of total contributions in 2012.[10]

Reform processes

In 2006, the Inter-agency Standing Committee (IASC) undertook humanitarian reforms to improve the leadership, accountability, co-ordination, and response capacity of the humanitarian community. In the wake of these reforms, WFP has assumed a new role in the humanitarian system, notably through its engagement in the global cluster system. In addition to actively participating in most clusters,[11] WFP leads the global logistics cluster and emergency telecommunications cluster, and co-leads the global food security cluster with FAO. WFP is also a central participant in the Transformative Agenda, an initiative undertaken by the IASC in an effort to further address gaps in leadership, co-ordination, and accountability.

Upon commencement of her tenure as Executive Director of WFP in April 2012, Ms Ertharin Cousin launched a process of organisational strengthening, which involved first a rapid organisational assessment and then the development of a framework for action identifying key corporate actions to enact transformational change. Major internal reforms currently underway include a restructuring of the organisation to better align its design around strategic priorities, the strengthening of resource management at the country level, a comprehensive review of human resource core processes, development of new approaches to communication and learning, and implementation of a single, streamlined system to improve performance monitoring and reporting.

This is the first time WFP has been assessed by MOPAN. For additional information, please consult the WFP website: www.wfp.org.

1.3 WFP SURVEY RESPONDENTS

WFP survey results reflect the views of 298 respondents who provided their views on WFP’s performance in the areas of organisational effectiveness and contribution to development and humanitarian results.[12]

Figure 1.1 | Number of survey respondents for WFP by country and respondent group

The distribution of responses on the six rating choices and ‘don’t know’ responses are summarised by respondent group across all survey questions in Figure 1.2.

Figure 1.2 | WFP – Distribution of responses (n=298) on all questions related to micro-indicators on organisational effectiveness

While there were responses in all six possible choices, relatively few responses overall were at the ‘weak’ end of the scale. There were higher percentages of ‘don’t know’ responses from in-country donors (30%) and peer organisations (23%). Recipient government respondents were slightly more positive than other groups overall (22% of responses were very strong).

2. MOPAN methodology 2013

2.1 OVERVIEW

The detailed MOPAN 2013 methodology, which includes rating scales, data analysis criteria, and strengths and limitations of the methodology, can be found in Volume II, Appendix I. The following is a brief summary.

MOPAN assessments examine:

• Organisational effectiveness: organisational systems, practices, and behaviours that MOPAN believes are important for aid effectiveness and that are likely to contribute to development and/or humanitarian results at the country level; and

• Humanitarian and development results: the extent to which there is evidence of an organisation’s contributions to development and/or humanitarian results.

As the MOPAN methodology has been refined each year since 2003, comparisons of this year’s assessments and any previous MOPAN assessments of an organisation should take this into consideration.

Data collection methods and sources

Over the years, MOPAN developed a mixed methods approach to generate relevant and credible information that MOPAN members can use to meet their domestic accountability requirements and support dialogue with multilateral organisations that they are funding.

The MOPAN approach uses multiple data sources and data collection methods to triangulate and validate findings. This helps eliminate bias and detect errors or anomalies.

In 2013, the sources of data included surveys of the multilateral organisation’s stakeholders and of MOPAN donors at headquarters and country level and a review of documents prepared by the organisations assessed and from other sources.

Assessment of organisational effectiveness

In the organisational effectiveness component, MOPAN examines performance in four areas: strategic management, operational management, relationship management, and knowledge management. Within each performance area, effectiveness is described using key performance indicators (KPIs) that are measured through a series of micro-indicators (MIs) using data from the survey and document review.

In this component, survey respondent ratings are shown as mean scores and are presented alongside document review ratings based on criteria defined for each micro-indicator. Not all micro-indicators are assessed by both the survey and the document review. The charts show survey scores and document review scores for the relevant KPIs or MIs. The full list of micro-indicators comprising this component is provided in Volume II, Appendix V.

Assessment of development and/or humanitarian results

In the development results component, MOPAN does not assess an organisation’s results on the ground, but examines how it measures and reports on its contributions to development and/or humanitarian results through three key performance indicators:

• Evidence of the extent of the organisation’s progress towards its stated institutional/organisation-wide results (i.e. goals, objectives, outcomes)

• Evidence of the extent of the organisation’s contributions to country-level goals and priorities, including relevant millennium development goals (MDGs)

• Relevance of the organisation’s objectives and programme of work to stakeholders.

In this component, a “best fit approach” is used in determining the ratings for the first two KPIs above. One of four qualitative ratings (strong, adequate, inadequate, or weak) is assigned following an analysis of data from all sources and confirmed in a consensus-based consultation (with of institutional advisors, a panel of experts, and MOPAN members). The rating of the KPI on relevance is based on stakeholder surveys only.

2.2 DATA SOURCES AND RATINGS

Survey

MOPAN gathers stakeholder perceptions through a survey of MOPAN members (at headquarters and in- country) and other key stakeholders of the multilateral organisation. Donor respondents are chosen by MOPAN member countries; other respondents are identified by the multilateral organisation being assessed.

The survey questions relate to both organisational effectiveness and to the achievement of development and/or humanitarian results. Survey respondents are presented with statements and are asked to rate the organisation’s performance on a six-point scale where a rating of 1 is considered “very weak” up to a rating of 6 which is considered “very strong.” A mean score is calculated for each respondent group (e.g. donors at headquarters).

MOPAN aims to achieve a 70 per cent response rate from donors at headquarters and a 50 per cent response rate among respondents in each of the survey countries (i.e. donors in-country and other respondent groups such as direct partners/clients).

Document review

The document review considers four types of documents: multilateral organisation documents; internal and external reviews of the organisation’s performance; external assessments such as the Survey on Monitoring the Paris Declaration, the Common Performance Assessment (COMPAS) report, and previous MOPAN surveys; and evaluations, either internal or external, of the achievement of results at various levels.

Document review ratings are based on a set of criteria that MOPAN considers to represent good practice in each area. The criteria are based on existing standards and guidelines (for example, UNEG or OECD-DAC guidelines), on MOPAN identification of key aspects to consider, and on the input of subject-matter specialists. The rating for each micro-indicator depends on the number of criteria met by the organisation.

Interviews

Interviews are conducted with staff based at headquarters and country offices of multilateral organisations who are knowledgeable in areas that relate to the MOPAN assessment. Interview data are not coded or used as a formal data source but rather to help ensure that the Assessment Team has all appropriate and necessary documents to enhance the triangulation of data from various sources and provide contextual information to assist in the analysis of the key performance indicators.

2.3 STRENGTHS AND LIMITATIONS OF THE MOPAN COMMON APPROACH

MOPAN continues to improve methodology based on experience each year. The following strengths and limitations should be considered when reading MOPAN reports.

Strengths

• The MOPAN Common Approach is based on existing bilateral assessment tools with the intent to replace or reduce the need for other assessment approaches by bilateral donors.

• In line with donor commitments to aid effectiveness, it seeks perceptual information from different stakeholder groups.

• It uses multiple sources of data to increase the validity of the assessment, enhance analysis, and provide a basis for discussion of agency effectiveness.

• MOPAN reports are validated and reviewed by MOPAN members and by the multilateral organisation being assessed.

Limitations

• The countries selected for MOPAN assessments comprise only a small proportion of each institution’s operations, thus limiting generalisations.

• Although MOPAN uses recognised standards and criteria for good practice, such criteria do not exist for all indicators. Many document review criteria were developed by MOPAN; these are a work in progress and not definitive standards.

• Survey rating choices may not be used consistently by all respondents. Some respondents may tend to avoid extremes on a scale and respondents in some cultures may be unwilling to criticise or too eager to praise.

• The survey covers a broad range of issues and individual respondents may not have the knowledge to respond to all the questions relating to a given organisation.

3. WFP’s organisational effectiveness

3.1 INTRODUCTION

This chapter presents the findings of the 2013 MOPAN assessment of WFP’s organisational effectiveness, that is its practices and systems that support the achievement of results. Findings are based on respondent survey data and document review.

• Section 3.2 presents overall ratings on the performance of WFP and summarises respondent views on its primary strengths and areas for improvement

• Section 3.3 provides findings on each of the four areas of performance (strategic, operational, relationship, and knowledge management).

3.2 OVERALL RATINGS

This section provides a summary of survey and document review ratings for all key performance indicators. It also presents survey respondent ratings of WFP’s overall organisational effectiveness and a summary of their written comments on WFP strengths and areas for improvement.

Overall ratings of key performance indicators

Figure 3.1 below shows scores from the document review and the survey on key performance indicators (KPIs) in the MOPAN 2013 assessment of WFP’s practices and systems. The white bar presents the survey score, while the black square presents the document review score. For example, on the first indicator, “providing direction for results”, WFP received a score of 4.56 (strong) in the survey and a score of 4 (adequate) in the document review.

In the overall ratings from the survey and document review, WFP was seen to perform adequately or better on the majority of key performance indicators. WFP received scores of adequate or better on all 20 KPIs assessed in the survey and on 18 of the 22 KPIs assessed by the document review.[13]

The survey and document review ratings differed on eight KPIs – seven of which were rated lower by the document review than by survey respondents, and the opposite for the remaining KPI. The reasons for these differences are discussed in the following sections.

Figure 3.1 | Overall ratings on key performance indicators of IFAD’s organisational effectiveness (survey mean scores and document review ratings)

Survey ratings of WFP’s organisational effectiveness

MOPAN has defined “organisational effectiveness” as the extent to which a multilateral organisation is organised to support partners and beneficiaries in producing and delivering expected results. Respondents were asked the question: “How would you rate the overall organisational effectiveness of WFP?” As shown in Figure 3.2, recipient government respondents provided the highest percentage of ratings on the high end of the scale, with nearly one-third (31%) rating WFP as very effective.

Figure 3.2 | Overall ratings of WFP organisational effectiveness by respondent group

Respondents’ views on WFP’s strengths and areas for improvement

The survey included two open-ended questions that asked respondents to identify WFP’s greatest strengths and areas for improvement. All 298 respondents provided comments on both questions.[14]

Survey respondents considered WFP’s greatest organisational strengths to be its capacity to respond to and deliver results during emergencies, and its expertise in logistics (including procurement). Respondents also commented positively on WFP’s expansive coverage and deep field presence.

The most frequently cited strength was WFP’s capacity for emergency response and delivery, noted by 39% of respondents (n=116). According to respondents, WFP has a well-established network and good relationships with local governments, supported by a solid reputation and extensive experience in emergency response. Respondents also highlighted WFP’s ‘can-do’ and action-oriented attitude as well as its co-operative and friendly staff. All of these factors are seen to contribute to its overall efficiency and rapid delivery of services.

WFP was also perceived by 23% of respondents (n=69) as having strong logistics. WFP’s many years of experience are seen as a key contributing factor to its planning and logistical capacity, as is the organisation’s ability to adapt quickly to difficult country contexts.

In addition, 17% of survey respondents (n=52) felt that WFP’s coverage, reach and field presence are a key strength. Respondents noted the organisation’s worldwide presence, as well as its ability to access unreachable areas, due in part to its strong ties with governments and other stakeholders.

In survey respondent comments on the areas most in need of improvement, the most frequently mentioned were WFP’s dual mandate, its transition from food aid to food assistance, its co-operation and co-ordination with partners, and its reporting on results.

The most frequently cited area for improvement concerned WFP’s dual mandate, which was commented on by 22% of surveyed stakeholders (n=67). Some donors at headquarters (n=14) felt that the organisation should better define and stick to its mandate, notably by concentrating or limiting its efforts to providing assistance during emergencies. In contrast, comments received from recipient government and peer organisation respondents (n=17) suggested that WFP should further engage in and improve its development work and enhance its resilience and livelihood-building activities.

One quarter of respondents (n=16) who commented on the organisation’s mandate indicated a need for WFP to strengthen its transition from food aid to food assistance. Donors at headquarters in particular (n=11) felt strongly about this issue, calling for the implementation of innovative practices and tools during the coming years.

WFP’s co-operation and co-ordination with partners was identified as an area for improvement by 18% of surveyed respondents (n=54), with more than half of comments received (n=31) from donors at headquarters or direct partners. Respondents perceived the need for WFP to increase and expand its collaboration with partners, most notably with other agencies in the UN. According to respondents, enhanced co-operation with implementing partners and sister UN agencies that have complementary areas of expertise could help WFP avoid duplication of efforts and allow it to stay focused on its mandate.

Finally, 9% of surveyed stakeholders (n=26) commented on the need for WFP to improve its monitoring and evaluation systems. Respondents indicated that WFP should improve results measure- ment at the outcome and impact level, rather than focusing solely on food distribution, and suggested that WFP could further improve its reporting on its contributions to results.

3.3 WFP’S PERFORMANCE IN STRATEGIC, OPERATIONAL, RELATIONSHIP, AND KNOWLEDGE MANAGEMENT

3.3.1 Overview

This section presents the results of the 2013 Common Approach assessment of WFP in four performance areas: strategic, operational, relationship, and knowledge management.

The following sections (3.3.2 to 3.3.5) provide the overall survey and document review ratings for the KPIs in each performance area, the mean scores by respondent group, and findings based on an analysis of survey and document review ratings in each performance area.

When there were notably divergent ratings between survey respondent groups or between the survey results and document review ratings, these are noted and the information gleaned from inter- views with staff is integrated when it has a bearing on the analysis. Where statistically significant differ- ences among categories of respondents were found, these differences are noted.

The survey data for each KPI and MI by performance area are presented in Volume II, Appendix V. The document review ratings are presented in Volume II, Appendix VI.

3.3.2 Strategic management

Survey respondents and the document review considered WFP adequate or strong in all areas of strategic management. The organisation’s results-oriented focus at the corporate and country levels was recognised as a strength in both its humanitarian and development programming. There is room for improvement in how WFP integrates gender equality in its overall work and a need for formal directives that require the consistent application of human rights-based standards in its development programming.

Figure 3.3 shows the overall survey and document review ratings for the five KPIs in the strategic management performance area.

Figure 3.3 | Performance area I: Strategic management, survey and document review ratings

Figure 3.4 shows the mean scores for the five KPIs for all survey respondents, and by category of respondent.

Figure 3.4 | Performance area I: Strategic management, mean scores by respondent group

KPI 1: Providing direction for results

Finding 1: Survey respondents considered WFP’s value system and leadership in providing direction for results to be strong. The document review noted that WFP has made considerable improvements in results-based management and that further efforts are underway to strengthen monitoring, evaluation and reporting systems.

Both perception data and the document review showed WFP to be strong in promoting results-based management. WFP’s strong commitment to managing for results has recently been reaffirmed within the context of the organisational realignment and change management processes. In particular, the Executive Director stated in 2012:

“My commitment to improving performance monitoring and reporting is absolute – significant improvements can and will be made with respect to monitoring, reporting and evaluation, and to the systems supporting performance management results and accountability.”[15]

Figure 3.5 | KPI 1: Providing direction for results, ratings of micro-indicators

MI 1.1 – Value system supports results-orientation and direct partner and beneficiary focus

For this MI, all survey respondent groups were asked four questions regarding WFP’s institutional culture: whether it reinforces a focus on results, is focused on humanitarian partners, is focused on direct beneficiaries, and emphasises respect for humanitarian principles. WFP was rated strong on all counts, but highest for integrating humanitarian principles within its value system; 70% of respondents rated the organisation strong or very strong on this point.

MI 1.2 – Leadership on results management

MOPAN donors at headquarters and recipient governments were the only respondent groups asked about the extent to which senior management demonstrates leadership on managing for results. More than half of respondents (52%) perceived WFP strong or very strong, 28% rated it adequate, and 9% inadequate or below.

MI 1.3 – Organisation-wide policy on results management

In the survey, donors at headquarters were consulted on the extent to which WFP ensures the application of results-based management across the organisation. The majority of respondents (82%) rated WFP adequate or above.

Based on documentary evidence, WFP was rated strong on this MI. The organisation first committed to implementing results-based management in 1997. Since that time, key milestones have included: the introduction of corporate strategic and financial plans; the development of results frameworks to ensure appropriate delivery of programming and efficient management; the roll out of annual performance plans to all offices to measure and report on key management dimensions; and the adoption of a performance management framework that considers performance and risk management holistically. The organisation has also developed an e-learning course on managing for results that is open to staff in all functional roles and locations and has established Performance and Risk Champions (PARCs) in all regional bureaux and country offices who are being trained on conceptual and practical aspects of performance and risk management.

Nevertheless, WFP recognises that certain gaps remain in the capacities of staff and partners in the field to collect and report programming outputs and outcomes. WFP’s 2012-2014 strategy on monitoring, evaluation and reporting is focused on strengthening staff capacity to: implement quality monitoring and evaluation within operations; ensure improved reporting on outcome indicators across the spectrum of WFP’s programme interventions; undertake robust self-evaluations of country office work; and implement the corporate monitoring and evaluation tool (COMET), which will provide a comprehensive results-based management platform for programme officers to design, plan, monitor, evaluate and report on the performance of operations throughout their life-cycle. Interviewees indicated that the 2012-2014 strategy, unlike its predecessor, is being properly resourced with earmarked funding and the basic module of COMET is scheduled to be rolled out by the end of 2013. In addition, WFP has placed monitoring and evaluation (M&E) advisors in all regional bureaux to create a network promoting a common understanding of M&E roles and functions, results frameworks, and strategy at the regional and headquarters level. The organisation intends to replicate this at the country level.

KPI 2: Corporate strategy and mandate

Finding 2: WFP was rated adequate overall for having a strategic plan that is based on its mandate and that includes sustainable solutions for beneficiaries. Key documents such as the mission statement do not yet reflect WFP’s repositioning as a food assistance agency.

Although survey respondents rated WFP strong on most questions related to WFP’s strategy and mandate (MI 2.1), in their comments on open-ended questions some respondents expressed confusion about WFP’s mandate and also some concerns about potential mission creep. Both of these issues were also raised in recent independent WFP evaluations.

Figure 3.6 | KPI 2: Corporate strategy and mandate, ratings of micro-indicators

MI 2.1 – Organisation-wide strategy based on clear mandate

Overall, WFP was rated strong or very strong on this MI by the majority of survey respondents (60%). Donors at headquarters as well as peer organisation and recipient government respondents were asked to rate WFP on four questions in this MI: whether it has a clearly defined mandate, its organisation-wide strategy[16] is aligned with its mandate, its strategy rests upon a clear understanding of the organisation’s capacity to deliver, and its strategy is clearly based on humanitarian principles. On the last question, peer organisations rated WFP more positively than other respondent groups, and the differences are statistically significant. In contrast, WFP was rated adequate by the document review.

WFP was established by the UN General Assembly (UNGA) in 1961 as an experimental programme falling under the purview of the United Nations and the Food and Agriculture Organization (FAO), with the main aim of providing food aid during emergencies and within development projects. Four years later, the programme was renewed indefinitely based on its achievements and potentialities in a resolution which defined additional terms of operation.[17] In 1994, the organisation also expressed its purpose and functions further within a mission statement based on a review of WFP’s policies, objectives and strategies that involved consultation with WFP member states, non-governmental organisations, United Nations and other agencies, academics and staff members. Building on this document, WFP’s General Rules and Regulations (last revised in 2010) delineate with more specificity the organisation’s roles and rules of procedures for delivering its mandate.

Although the mission statement is described as a ‘living document’ on the WFP website, it has not yet been modified despite substantial changes in the work of the agency in recent years. In fact, as highlighted in the strategic evaluation “How WFP Country Offices Adapt to Change” (2011), WFP’s repositioning from a food aid to a food assistance agency in WFP’s Strategic Plan for 2008-2013 represents perhaps the most significant strategic shift since the organisation was founded, and certainly “more than merely the adoption of new tools or incremental programme adjustments”. This evaluation as well as the strategic evaluation “WFP’s Role in Ending Long-term Hunger “(2011) noted that donors and partners are unclear on WFP’s mandate and that WFP should address these ambiguities and better articulate its comparative advantage. Nearly one-quarter of survey respondents (22%) expressed concerns regarding WFP’s mandate and identified this area as the one where WFP most needs to improve (see respondents views on WFP’s strengths and weaknesses in section 3.2).

MI 2.2 – Organisation-wide strategy includes appropriate mechanisms for sustainable solutions

In the survey, all respondent groups other than direct partners were asked whether WFP’s organisation- wide strategy includes appropriate mechanisms to facilitate sustainable solutions. While nearly two-thirds (65%) of respondents perceived WFP adequate or above, 23% considered its performance inadequate or below. Recipient government and peer organisation respondents rated WFP more positively than donors at headquarters or in-country, and these differences are statistically significant.

KPI 3: Corporate focus on results

Finding 3: Results statements and performance indicators in WFP’s corporate results frameworks were generally observed to respect results-based management standards.

WFP’s corporate results frameworks were assessed by document review only, and WFP was rated strong overall.

Figure 3.7 | KPI 3: Corporate focus on results, ratings of micro-indicators

MI 3.1 – Plans and strategies contain results frameworks

WFP was rated strong in the document review for having corporate-wide frameworks in place defining the organisation’s expected management and programme results.

WFP’s corporate level results framework to track programming achievements for the current strategic plan, known as WFP’s strategic results framework (SRF), was endorsed by the Executive Board in February 2009. The following year, in response to a recommendation from WFP’s External Auditor, the organisation also introduced a management results framework (MRF) outlining process results to ensure efficiency of services delivered. Both of these frameworks have been revised since, particularly their indicators.[18]

The SRF includes 5 high level strategic objectives (considered to correspond to impact level statements), 20 outcomes, and 6 outputs. The structure of WFP’s SRF is thus unusual, as results frameworks typically follow a pyramidal structure (i.e. a large number of outputs contribute to fewer outcomes, which in turn contribute to even fewer impacts). However, the inclusion of only a few output statements in WFP’s SRF can be explained. The organisation has a few key products and services, which it delivers using different approaches and in varying contexts and programming areas, thus leading to different and multiple outcomes.

In order to make the framework lighter and more manageable, WFP chose to regroup all distribution activities of food, cash/vouchers and non-food items into a single generic output statement that is repeated across four of its five strategic objectives[19] (as opposed to breaking down distribution into various outputs for each relevant outcome). This choice is understandable from an administrative standpoint, and provides for greater flexibility in programming. However, the common output is very broad and hides some of the specificity of WFP’s work across strategic objectives. Nevertheless, results statements in the SRF are all appropriate to their results level.

WFP’s management results framework (MRF) presents 17 results statements classified according to 5 management results dimensions or themes (securing resources, stewardship, learning and innovation, internal business processes and operational efficiency). Though not made explicit by WFP, these results statements appear to function as expected outputs.

MI 3.2 – Results frameworks link inputs to outputs/outcomes

WFP was rated adequate on this MI by document review for the quality of results chains in its strategic and management results frameworks. While the strategic results framework (SRF) includes three results levels (outputs, outcomes, and strategic objectives), there is only one tier of results in the management results framework (MRF). As there is no results chain in the MRF, it is not possible to determine what higher level change the management outputs are meant to contribute to.

In the SRF, the linkages between outputs and outcomes are logical; it is plausible that the products and services that WFP intends to deliver will contribute to the identified change at the outcome level, and that if achieved, these will lead to positive results in WFP’s strategic objectives. However, a possible improvement would be to further clarify the links between outputs and outcomes. In particular, the single output statement discussed in the previous MI could be broken down to more specifically reflect the populations targeted by the different outcomes.

MI 3.3 – Plans and strategies contain performance indicators

The quality of performance indicators in WFP’s corporate results frameworks was assessed as strong based on the documents reviewed.

All results statements in the management results framework and strategic results framework are associated with well-constructed indicators. The latter are relevant, clearly formulated, and measurable, and provide a sufficient basis to monitor the achievement of results.

Within the SRF, targets for indicators are atypical, but well-suited to the work of the organisation; they require that a substantial proportion of projects (often around 80%) meet their own project-specific target. Given that the bulk of WFP’s operations is in the humanitarian sector – a sector in which needs are hardly foreseeable – it is understandable, if not preferred, for the organisation to have the flexibility to determine its targets as a function of local conditions, rather than as an absolute numerical value for achievement organisation-wide.[20] Nevertheless, the timeframe in which WFP expects to reach identified targets is not clear as no dates for achievement are provided.

KPI 4: Focus on cross-cutting priorities

Finding 4: Survey respondents rated WFP adequate or better in mainstreaming the seven cross- cutting issues assessed by MOPAN and considered its emergency preparedness and response an organisational strength. The document review concurred on most issues, but noted gaps in how WFP integrates gender equality throughout its work and the need for formal directives that require the consistent application of human rights- based standards in its development programming.

The MOPAN assessment examined WFP’s practices and systems related to the integration of seven cross- cutting thematic priorities: gender equality, climate risk analysis and response, building capacities for good governance, human rights-based standards, emergency preparedness and response, protection, and HIV/AIDS.[21] The first four were assessed across all organisations by MOPAN in 2013,[22] and the last three were added by MOPAN in dialogue with WFP.

Figure 3.8 | KPI 4: Focus on cross-cutting priorities, ratings of micro-indicators

MI 4.1 – Gender equality

Donors at headquarters and in-country, as well as recipient government and peer organisation respondents were asked to rate WFP’s performance in mainstreaming gender equality (including gender- based violence) in its work. Overall, nearly three-quarters (74%) rated WFP adequate or above, and 15% rated it inadequate or below.

Strictly adhering to the MOPAN criteria, the document review rated WFP inadequate on this MI. Although WFP updated its Gender Policy in 2009 and adopted a corporate action plan to ensure its implementation over the 2010-2013 period, gender mainstreaming has been under-resourced. The 2012 “Update on the Implementation of the WFP Gender Policy” indicated that funding for the implementation of the corporate action plan over the first two years was less than half of the USD 7 million estimated as necessary. The 2012 Mid-Term Review of the Strategic Plan also highlighted a gap in staff’s capacity to create an enabling environment for gender mainstreaming. In fact, in its annual session in June 2012, WFP’s Executive Board:

“…noted with concern the lack of resources to support the corporate action plan and requested WFP to increase its investments and allocate appropriate human and financial resources to adequately support gender equality in WFP programming, including the development of gender equality results and indicators across all programming areas.”[23]

Nevertheless, there is evidence that in recent years WFP has reinvigorated its efforts to improve mainstreaming in the organisation. In the autumn of 2012, it finalised a gender mainstreaming accountability framework to strengthen accountability systems, tools and leadership at all levels within the organisation. During the same year, it also adopted the Inter-Agency Standing Committee’s (IASC) gender marker to assess the extent to which projects are gender sensitive, and began reporting against the performance indicators in the UN System-Wide Action Plan (UN SWAP) on Gender Equality and the Empowerment of Women. [24] WFP’s Gender Unit at headquarters, which has increased in size in the last year (gaining two professional staff posts, two short-term professional posts, and a general service staff post), has supported the integration of four new gender indicators in standard reporting documents for all WFP operations. As of 2013, the Gender Unit reports directly to the Deputy Executive Director and Chief Operating Officer “to ensure that gender receives the highest level of attention in WFP’s operational management”.[25] Given that these commendable actions are recent, it will likely take some time before they yield appreciable results.

MI 4.2 – Climate risk analysis and response

All respondent groups other than direct partners were surveyed for this MI. The majority of surveyed stakeholders (72%) considered WFP adequate or above in integrating climate risk analysis and response in its work. Peer organisation respondents rated WFP more positively than other groups, and these differences are statistically significant.

Based on documentary evidence reviewed, WFP was rated adequate on this MI. Since 2011, the organisation has engaged in consultations with the Executive Board to refine its strategy and approach to mainstreaming climate change within programmes and policies, supporting these discussions with documents outlining the expected climate impacts on hunger, WFP’s comparative advantages and experience to date, as well as an initial corporate agenda for action. In addition, climate risk analysis and response is embedded in several dimensions of WFP’s work. For instance, the 2011 Policy on Disaster Risk Reduction and Management commits WFP to monitoring climate risks through food security and vulnerability analysis, hazard analysis, and early warning. Notably, WFP’s Early Warning Analysis and Crisis Support Team (ODEP) is tasked with monitoring natural hazards that could affect the organisation’s on- going operations or trigger new interventions, and, as highlighted in WFP’s Emergency Preparedness and Response EPR Framework (2003), understanding climate risks is an integral part of preparing and responding to emergencies. The Emergency Preparedness and Response Package (EPRP) rolled out in 2012 specifies that the EPRP focal point at the country level is responsible for liaising with ODEP on potential climate risks identified, and country offices are strongly encouraged to establish early warning matrices. Moreover, in line with objective 2 of the strategic plan which relates to preventing acute hunger and investing in disaster preparedness and mitigation, WFP has been supporting national partners to develop early warning systems and other tools to monitor the effects of climate risks on food security. WFP supported the development or improvement of 59 early warning systems in 2012.[26]

MI 4.3 – Building capacities for good governance[27]

All respondent groups other than direct partners were consulted on WFP’s performance in building capacity for good governance to reduce hunger or promote food security. The majority (62%) rated WFP adequate or above and 23% rated it inadequate or below. Recipient government and peer organisation respondents rated WFP more positively than donors at headquarters or in-country and these differences are statistically significant.

In the document review, WFP was rated adequate on this MI. Following an evaluation in 2008 that demonstrated strengths and successes but also gaps and limitations in the implementation of WFP’s policy on Building National and Regional Capacities (2004), the organisation revised the policy to better define a vision, objectives, outcomes, outputs and activities to guide capacity-development investments and work, and framed these in the context of WFP’s shift to food assistance. In 2010, it also developed an action plan to implement the policy that defines priority points of engagement, maps out roles and responsibilities, and includes a matrix identifying key activities and timeframes for implementation. However, there has been no formal reporting on the action plan to track the progress of its implementation.

Capacity development has featured in WFP’s strategic plans since 1994. In the 2008-2013 strategic plan, it is the primary focus of WFP’s fifth strategic objective, which is to “strengthen the capacities of countries to reduce hunger through handover strategies and local purchase”. However, funding for this objective between 2009 and 2012 has paled in comparison to that allocated to the four other corporate objectives. The 2012 strategic evaluation “Working in Partnership” notes that funding, among other factors, hinders achievement of results:

“… limited resources for capacity development, work planning systems that are too short-term and project- based and other factors affect WFP’s ability to partner in a manner that increases government ownership [...] The evaluators found several examples in which governments asked for more support for capacity development but WFP‘s ability to respond was limited.”

In order to ramp up its focus on capacity development in its 2014-2017 strategic plan,[28] WFP has created a Country Capacity Strengthening Unit at headquarters that will be rolling out implementation guidance, a needs assessment, and a results framework to support country offices in implementing capacity development. WFP also indicates that systematic briefings and staff training will be undertaken to improve implementation capacity and reporting on results, and that it is working on a corporate approach to increase funding for capacity development functions.

MI 4.4 – Human rights-based standards

All respondent groups other than recipient governments were asked to rate WFP on how well it promotes human rights-based standards in its work. The majority of respondents (75%) rated WFP adequate or above.

The document review examined WFP’s mainstreaming of human rights-based standards in its humanitarian and development work, and rated WFP adequate based on the MOPAN criteria. The document review assessed whether WFP has policies and/or strategies, clearly defined roles and responsibilities, guidelines and training, integrated institutional systems and capacities (e.g. human resources, budgeting, etc.), as well as accountability mechanisms in place for applying human rights-based standards.

With respect to WFP’s humanitarian work, which constitutes the bulk of its portfolio, WFP has developed a policy, implementation approach, guidelines and training, and defined roles and responsibilities on humanitarian protection to ensure application of humanitarian principles and standards (related practices and systems are discussed further in MI 4.6 below).

On the development side, however, WFP’s commitment to applying human rights-based standards could be strengthened. The organisation indicates that its protection policy outlined above presents standards that are coherent with those found in the UN Common Statement of Understanding on Human Rights- Based Approaches to Development Co-operation and Programming (2003), and that its protection approach and guidelines are thus also applicable in development settings. “Rather than adopting two separate yet similar approaches to integrating human rights-based standards, the organisation seeks to build on its approach to integrate such standards in humanitarian settings, [and] in development contexts as well.” While some offices have initiated measures to integrate protection standards into their development programming and have trained staff accordingly, the assessment team notes that there are currently no formal directives requiring country offices to mainstream the protection approach in their development work.

MI 4.5 – Emergency preparedness and response

All respondent groups other than direct partners were asked whether WFP sufficiently integrates emergency preparedness and response in its work. Nearly two-thirds (65%) rated WFP as strong or very strong, 21% as adequate, and 7% as inadequate or below. The document review rated WFP strong on this MI.

Emergency preparedness and response (EPR) is central to WFP’s work, and identified within the organisation’s current strategic plan as crucial to saving lives and reducing malnutrition –WFP’s core business. Since the early 2000s, the organisation has invested more than USD 160 million in professionalising its EPR approach through directed programming, focusing notably on improving vulnerability assessment mapping (VAM) and geographic information systems (GIS), contingency planning, cluster telecommunications, logistics as well as avian flu preparedness. In fact, a formal trust fund has been established to help institutionalise EPR.

In 2011, WFP also launched an organisation-wide programme to develop and implement a faster, more efficient and effective response model to contend with the increasingly unpredictable environments in which WFP operates and adapt to global challenges (e.g. climate change, volatile food and fuel prices, expanding urbanisation, and more difficult security conditions), and to properly embed new tools and approaches promoted by WFP’s transition to food assistance. The thrust of the programme is to further strengthen response capacities, improve WFP’s internal accountability and normative framework, and strengthen partnerships with national authorities and the international community. The organisation also developed a Generic Response Capability Model (GRCM), a corporate tool for comprehensive emergency readiness planning that details response scenarios, outlines a realistic set of targets, and then maps out the capabilities required to meet the scenario parameters and targets.

WFP is continuously monitoring its progress and the impact of its work through performance reviews of corporate emergency operations, and the final evaluation of the programme is planned for 2014. In addition, a new EPR policy is scheduled to be presented to WFP’s Executive Board in 2015.

MI 4.6 – Protection

All respondent groups other than recipient governments were asked to assess this MI. The majority of respondents (69%) perceived WFP to be performing adequately or better in mainstreaming protection issues in its work, while 10% rated WFP as inadequate or below. There was a high proportion of ‘don’t know’ answers (21%) for this MI.

The document review rated WFP adequate on this MI. In 2012, the organisation adopted its first policy on humanitarian protection and the Humanitarian Protection Policy Implementation Approach as a companion document. Although the adoption of the policy is fairly recent, it follows and is informed by a pilot project initiated in 2005 which sought to operationalise humanitarian principles and standards, and allowed WFP to better understand: the meaning and application of the humanitarian protection concept for WFP; the impact of protection problems on its food assistance mandate; the scope for improved programming by adopting a protection lens; and the required skills for implementation. Since the launch of the pilot project, more than 2 500 WFP staff and partners have received training to improve their awareness and comprehension of beneficiaries’ protection concerns, the linkages with food security, and how to implement a protection approach.[29] In 2009, the organisation developed an enhanced Protection Training Manual and the 2012 policy requires that all WFP staff follow this training.

WFP’s commitment to protection represents an important step, but it will take some years to see how the policy and approach are implemented throughout the organisation. WFP is scheduled to provide an informal update on the implementation of its protection policy to its Executive Board in June 2014.

MI 4.7 – HIV/AIDS

MOPAN donors at headquarters were the only respondent group surveyed for this MI. While 64% rated WFP adequate or above for integrating HIV/AIDS in its work, 25% responded ‘don’t know’.

WFP received a strong rating for integrating HIV/AIDS as a thematic priority in its work based on the documentary evidence reviewed. Following a 2008 evaluation that highlighted weaknesses, WFP updated its HIV and AIDS Policy to better delineate its mandate in HIV/AIDS programming, as well as its related policy objectives. The policy is in line with WFP’s comparative advantage as determined by the UNAIDS division of labour, and also aligns with the UNAIDS 2011-2015 strategy. In response to an Executive Board recommendation, WFP also developed an e-learning strategy on HIV, food and nutrition for regional and country-level staff and partners.[30] The organisation also designed and launched additional guidance materials, including a Monitoring and Evaluation Guide for Food-Assisted HIV Programming to help programme staff design their own monitoring and evaluation (M&E) systems in line with national M&E systems.[31]

The Annual Performance Report for 2012 notes that there has been a decrease in the number of countries and beneficiaries receiving WFP assistance through HIV/AIDS programming since 2009.[32] This is due to several factors: food assistance in WFP programmes is now fixed for specific periods and usually tied to strict beneficiary entry and exit criteria, governments in some countries are successfully taking over HIV/AIDS response and programming, and donors are pushing for integration of HIV work into broader health interventions (which has positive aspects in general but some negative implications for organisations like WFP that do not have an overall health mandate).[33] While the global decline in funding for HIV/AIDS has forced some country offices to terminate implementation of HIV-specific work, WFP is increasingly building government capacity to deliver such programmes, and it plays an important role in facilitating country dialogue and collaborating with country stakeholders, country coordinating mechanisms and national disease programmes to ensure that food and nutrition support is included in national HIV strategies and programmes.

In recognition of WFP’s good HIV/AIDS programming results in 2010-2011, the organisation was one of the few UNAIDS co-sponsors to receive a 15% increase in its allocations from the UNAIDS Unified Budget, Results and Accountability Framework (UBRAF) for the 2012–2013 period.

KPI 5: Country focus on results

Finding 5: WFP was considered strong in both the survey and documentary assessment for its focus on humanitarian and development results in-country, though surveyed stakeholders noted room for improvement in WFP’s inclusion of beneficiaries in the design of its humanitarian responses.

Figure 3.9 | KPI 5: Country focus on results, ratings of micro-indicators

MI 5.1 – Country-level project/programme results aligned with corporate results framework

All respondent groups other than direct partners were consulted on this MI. While the majority of respondents (62%) considered WFP strong for aligning results statements in its country-level strategies[34] with those in the corporate strategic results framework, 34% responded ‘don’t know’.

The document review rated WFP very strong on this MI. WFP’s results frameworks at the country level for both its humanitarian and development operations include outputs and outcomes that are organised according to the five corporate strategic objectives, and that closely mirror the results statements in the corporate strategic framework. This close correspondence is corporately organised to ensure tracking and aggregation of results at the organisation-wide level.

In the logical frameworks of operations reviewed for the assessment, the outputs and outcomes were appropriate to their level and generally well formulated. In addition, due to the specificity of the outputs in these frameworks, results chains between outputs and outcomes were clearer than those observed at the corporate level.

MI 5.2 – Project/programme results frameworks include indicators

All respondent groups other than direct partners were asked about this MI and rated WFP strong overall for including indicators in the results frameworks of its operations at the country level. However, a high proportion of respondents (34%) responded‘don’t know’. The document review rated WFP strong on this MI.

In the logical frameworks of operations reviewed, all output and outcome results statements had indicators that were generally clearly articulated and relevant, and that provided a sufficient basis to assess performance. However, few indicators had targets for dates of achievement. Detailed guidance on the methodology to measure indicators was found in the organisation’s Indicator Compendium.[35]

MI 5.3 – Results for cross-cutting priorities included in country level frameworks

All respondent groups other than direct partners were asked whether WFP’s strategies at the country level include results for cross-cutting priorities. The majority (61%) rated WFP adequate or above, 9% rated it inadequate or below, and 30% answered ‘don’t know’.

The document review rated WFP adequate on this MI. All humanitarian strategies[36] as well as development strategies[37] sampled at the country level mentioned at least two of the seven cross-cutting themes assessed by MOPAN (gender equality, climate risk analysis and response, building capacities for good governance, human rights-based standards, emergency preparedness and response, protection and HIV/AIDS). However, less than half of the humanitarian strategies included related outputs or outcomes within their results frameworks. Of the seven thematic issues, gender equality was cited most frequently, while little to no mention was made of protection or human rights-based standards.

MI 5.4 – Design for humanitarian response developed in consultation with partners and beneficiaries

Country-level respondents were asked two questions regarding the design of WFP’s humanitarian responses: whether WFP ensures sufficient involvement of beneficiaries (except in some cases of armed conflict), and whether it sufficiently involves partners (including governments) whenever feasible and appropriate. Overall, respondents rated WFP strong for including partners in design, and adequate for consulting beneficiaries.

MI 5.5 – Humanitarian operations based on reliable assessments of food and nutrition needs

Respondents in the assessed countries considered WFP strong for basing its humanitarian programmes on reliable assessments of food and nutrition needs; more than half (58%) rated the organisation as strong or very strong, 25% as adequate, and 9% as inadequate or below.

In line with the survey, the document review rated WFP strong for its assessments of food and nutrition needs. Subsequent to adopting its Policy on Emergency Needs Assessments in 2004, the organisation initiated a three-year project to improve methodologies, guidance and training – and ultimately accuracy, timeliness and impartiality – of its needs assessments in emergencies. This was followed by the Enhanced Capacity in Food Security and Response Analysis Project (ENCAP) from 2008 to 2010, which was carried out to further expand capacities and tools as WFP shifted from a food aid to food assistance agency, and which focused in particular on developing analyses to inform decision making on the most appropriate transfer mechanisms (food, cash, and/or vouchers). Over the years, WFP has developed a range of tools and guidance notes for its staff and partners, and has also instituted an internal quality control mechanism for emergency needs assessment reports, some of which are subjected to an external peer review.

The 2011 ENCAP evaluation observed that most operations were adequately based on assessments and analyses and that programme activities were generally relevant to identified needs. The Annual Evaluation Report for 2012 confirmed that country portfolio evaluations completed during the year had identified vulnerability assessment mapping as a strength of WFP, but that they also noted room for improvement in: staffing to complete such analyses in highly complex environments; engagement to support development of national systems; and consistent analyses of factors that shape locally appropriate programme design (e.g. conflict, political environment, livelihoods, gender, and partner capacity).

MI 5.6 – Contingency plans in place for humanitarian action

All respondent groups other than direct partners were surveyed for this MI. Although the majority (72%) rated WFP adequate or above for having contingency plans in place to considerably increase or scale up humanitarian action in the case of need, a high proportion (21%) answered ‘don’t know’.

Based on the documentary evidence reviewed, WFP received a rating of adequate on this MI. An evaluation in 2009 observed that contingency planning had had:

“(…) limited impact on preparedness and response in many cases, particularly with regard to concrete preparedness enhancements such as pre-positioning of stocks, logistical arrangements, improved access to sources of information and pre-approved agreements with partners or authorities.”[38]

Nevertheless, the evaluation noted that these plans had potential and that impacts could be improved with a re-conceptualised approach, revised guidelines, and increased organisational commitment. At the time of the evaluation, contingency planning was being carried out in a stand-alone fashion. WFP has since revised this approach and integrated contingency planning more holistically with risk assessment, security management, and business continuity planning under the broader umbrella of emergency preparedness and response. WFP’s Emergency Preparedness and Response Package (EPRP), rolled out in mid-2012, replaced the organisation’s 2002 Contingency Planning Guidelines. Although it will take some time for WFP to demonstrate the effectiveness of these recent measures, WFP is considered a front runner on contingency planning among the UN agencies.

As co-chair of the Inter-agency Task Force on Preparedness and Contingency Planning, WFP participates actively in inter-agency contingency planning exercises and is currently revising related inter-agency guidelines with the United Nations Office for the Coordination of Humanitarian Affairs (OCHA) and the Food and Agriculture Organization (FAO). WFP has also supported the development or enhancement of government contingency plans as a way of strengthening national capacities to prepare for, assess and respond to acute hunger arising from disasters. As reported in the Annual Performance Report for 2012, WFP supported the establishment or improvement of 23 inter-agency or government contingency plans in 2012.

3.3.3 Operational management

In operational management, WFP was commended for its sound systems and practices for financial accountability, delegation of authority, and use of performance information. While surveyed stakeholders considered WFP adequate or better in all areas assessed, the document review ratings suggest it could improve its results-based budgeting by better linking allocations and expenditures to results.

Figure 3.10 below presents the overall survey and document review ratings for the eight KPIs in the operational management performance area.

Figure 3.10 | Performance area II: Operational management, survey and document review ratings

Figure 3.11 shows the mean scores for the KPIs for all survey respondents, and by respondent group.

Figure 3.11 | Performance area II: Operational management, mean scores by respondent group

KPI 6: Resource allocation decisions

Finding 6: Overall, both the survey and document review rated WFP adequate for its resource allocation processes.

Survey respondents considered WFP adequate overall in managing the allocation and disbursement of resources. The document review concurred but could not assess WFP on whether its financial estimates of demand and donor support are reasonable in the absence of clear standards in this area.

Figure 3.12 | KPI 6: Resource allocation decisions, ratings of micro-indicators*

*The document review was designed to draw data from the 2010 Survey on Monitoring the Paris Declaration.

MI 6.1 – Criteria for allocating un-earmarked resources publicly available

All respondent groups other than donors in-country were asked whether WFP makes readily available its criteria for allocating resources. More than half of respondents (56%) rated WFP adequate or above, 16% as inadequate or below, and a large proportion of respondents (28%) answered ‘don’t know’.

Based on the review of documents, WFP was rated adequate on this MI. In 1986 the Committee on Food Aid (the predecessor to the WFP Executive Board) established broad criteria to guide the Executive Director in making decisions on if and when to intervene in emergencies, and in 1994 established guidelines to determine resource levels for development operations. WFP’s General Regulations and General Rules (2010) further specify the terms defining countries’ eligibility for assistance, the processes to follow to request assistance, and the content of assistance agreements. In addition, the 2004-2007 Strategic Plan stipulated additional conditions for WFP to follow in allocating undirected multilateral resources received from donors. In 2009, in response to Executive Board concerns about prioritising allocation of resources, WFP established a Strategic Resource Allocation Committee (SRAC). Over the 2010-2011 period, WFP’s Secretariat organised three seminars to share information with member states on how WFP plans and prioritises its resources. One seminar provided detailed information on the quantitative and qualitative analyses the SRAC uses to determine which projects experiencing funding shortages receive multilateral contributions.

While WFP has clear criteria for allocating its resources, not all information is publicly available (for instance regarding the SRAC prioritisation process) or consolidated in a single document for ease of reference.

MI 6.2 – Resource allocations conform to criteria

MOPAN donors at headquarters, the only respondent group surveyed for this MI, were asked whether WFP allocates un-earmarked resources according to established criteria. Nearly two-thirds (66%) rated WFP as adequate or above, 11% rated it inadequate or below, and 23% responded ‘don’t know’.

MI 6.3 – Resources released as needed and available

All respondent groups other than direct partners were asked whether WFP manages aid flows on the basis of needs and its own strategic priorities. Overall, 48% rated WFP as strong or very strong, 26% as adequate, and 11% as inadequate or weak.

The document review rated WFP adequate on this MI. WFP is funded solely by voluntary contributions (i.e. it receives no assessed contributions),[39] and its budget is prepared primarily on the basis of assessed needs rather than on levels of anticipated donor support. These conditions, as noted in numerous audits, evaluations and other reports, challenge WFP’s ability to ensure the predictability of its assistance to developing countries. However, the organisation has taken numerous steps to improve the delivery of its scheduled aid. WFP has adopted a three-year rolling management plan/budget to revise appropriations annually, and has made significant efforts to secure resources by diversifying donor contributions (e.g. using innovative approaches such as twinning and debt swaps) and increasing multi-year agreements. Multi-year contributions to WFP doubled in 2012, representing 10% of all contributions in that year. In addition, WFP has expanded its use of advance financing facilities such as the Immediate Response Account, the Working Capital Financing, and the Forward Purchase Facility, which allow funds for operational purposes to be released before donor contributions arrive. In 2012, use of advance financing mechanisms reduced lead times in the supply chain of operations by 70%.

MI 6.4 – Reasonable financial estimates of demand and support for its humanitarian action

MOPAN donors at headquarters were asked two questions for this MI. They rated WFP strong for producing reasonable financial estimates of demand for its humanitarian action, and adequate for the accuracy of its projections of expected donor support. The document review did not rate WFP on this MI as there is no consensus or agreed standards related to estimates and forecasting in humanitarian response.

KPI 7: Results-based budgeting

Finding 7: Although survey respondents perceived WFP’s results-based budgeting practices to be adequate, documentary evidence revealed considerable room for improvement in how the organisation links its allocations and expenditures to results.

Only MOPAN donors were surveyed for this KPI. They rated WFP strong for linking allocations to expected results and adequate for linking disbursements to results. The document review, which noted that the organisation has yet to implement a system to track allocations and disbursements from activities through to outcomes, rated WFP weak. It should be noted, however, that linking resource allocations and/or expenditures to planned humanitarian or development results remains a challenge for all multilateral organisations.

Figure 3.13 | KPI 7: Results-based budgeting, ratings of micro-indicators

MI 7.1 – Allocations linked to expected results

MOPAN donors at headquarters were asked whether WFP links its budget allocations to its expected humanitarian results. More than half (55%) rated WFP as strong or very strong, 34% as adequate, and 7% as inadequate.

In contrast to the survey data, the document review rated WFP weak on this MI. While WFP has improved its organisation-wide budget over the last two years and now presents allocations by strategic objective and management result dimension, only one output is costed[40] and no outcome results statements are costed in the “WFP Management Plan (2013-2015).[41] The organisation’s resource management system does not yet allow the organisation to track allocations from activities through to outcomes. Nevertheless, the organisation is strongly committed to performance-informed budgeting and to aligning its performance planning and budgeting processes: WFP has taken steps to align its programme and management results frameworks to the annual management plan through a revised budget template that will inform the Strategic Resource Allocation Committee (SRAC) budget allocations as well as programme support and administrative costs. In 2014, WFP will require each division or unit to articulate a results statement and to cost all contributing activities for the 2014 management plan.

MI 7.2 – Expenditures linked to results

MOPAN donors at headquarters and in-country were asked if WFP’s reports on results include the amounts disbursed to achieve those results. They rated WFP adequate overall and the survey results were statistically significant.

WFP was rated weak on this MI on the basis of the documents reviewed. Although WFP’s Annual Performance Report for 2012 presents expenditure data for WFP’s strategic objectives, financial disbursements are presented only for one output and not aligned with programme outcomes achieved. The report also includes and discusses an indicator on the deviation between planned versus actual expenditures, but the information is calculated at the strategic objective level. The report does not explain variances between expected and achieved organisation-wide programming results.

KPI 8: Financial accountability

Finding 8: WFP’s processes for financial accountability are considered strong.

This KPI examined eight aspects of financial accountability, all of which were assessed by document review and two by MOPAN donors at headquarters; WFP was consistently rated strong or very strong.

The document review found WFP’s practices and systems for financial accountability to be strong. Although timely implementation of internal and external audit recommendations could be improved, the internal and external audit functions are independent and respect international standards. WFP has considerably strengthened its internal controls and management of risks across the organisation.

Figure 3.14 | KPI 8: Financial accountability, ratings of micro-indicators

MI 8.1 – External financial audits performed across the organisation

This MI was assessed through document review only, which rated WFP as very strong. As specified in the organisation’s Financial Regulations (2010), an external auditor is appointed for a six-year term by WFP’s Executive Board through a competitive process, and must be the Auditor-General (or equivalent) of a member state of the United Nations or of FAO. The external auditor’s purview concerns the efficiency of WFP’s financial procedures, accounting system, internal financial controls, and administration and management.

Since 2009, WFP’s financial statements have been prepared in accordance with the International Public Sector Accounting Standards (IPSAS) and reviewed by WFP’s external auditor annually. The external auditor’s report, submitted with the financial statements as a single document to the Executive Board, has each year certified that the audit was conducted in conformity with the International Auditing Standards (ISA) and WFP’s Financial Regulations.

MI 8.2 – External financial audits performed at the regional, country, or project level

This MI, which was assessed through document review only, was rated strong. WFP’s Financial Regulations do not explicitly indicate that the external auditor must conduct financial audits at the regional, country or project levels, but that he or she may “perform such audit of the accounts of WFP, including trust funds and special accounts, as deemed necessary in order to satisfy himself or herself.” Each of the external auditor’s work plans over the 2010-2013 period provided a list of 10 or 11 country and regional offices that were to be audited (based on risk criteria such as total active projects, number of beneficiaries, projected tonnage needed, operational budget/outlay and previous audit findings). The reports to the Executive Board on externally audited financial statements incorporate information from the regional and country office audits, and confirm that letters were sent to the management of country offices and regional bureaux to inform them more fully of each audit’s findings. In addition, when country or regional audits are requested by the Executive Board (as was the case for Somalia in 2012), a full report and management response are shared at a Board session and made publicly available.

MI 8.3 – Policy on anti-corruption

Based on the review of documentary evidence, WFP was rated strong on this MI. In 2010, WFP adopted its first comprehensive Anti-Fraud and Anti-Corruption Policy, which is founded on and reflects principles from existing WFP regulations and rules, standards, directives and manuals, as well as the United Nations Convention against Corruption. The policy encourages the denunciation of fraudulent activities through whistle blowing mechanisms and commits the organisation to promptly investigate alleged fraud cases. The policy also delineates roles and responsibilities to ensure implementation. Since the policy was approved in 2011, the Office of the Inspector General has provided training on the subject to all Office of the Investigator staff, all procurement staff in headquarters, all human resources officers, participants in regional bureau meetings, and some members of the senior management team. Investigators have also been training staff in country offices they visit.

Nevertheless, the report submitted to the Board by WFP’s Inspector General in 2013 indicates that further improvements could be made to increase staff awareness of the policy and ensure that the results of investigations of alleged fraudulent behaviour are better communicated to staff. In 2013, in response to the Inspector General’s report, WFP’s Executive Director reiterated the organisation’s commitment to maintaining zero tolerance against fraud and stated that WFP’s internal justice system would be reviewed to enhance its efficiency and effectiveness, and that efforts would be made to improve reporting practices to address perceptions that WFP’s policy has not been applied rigorously. The Office of the Inspector General has started developing a formal communication strategy, including the use of the intranet, to give staff more information on its activities.

MI 8.4 – Systems for immediate measures against irregularities

WFP’s systems for taking immediate measures against financial irregularities were rated strong by document review. The organisation’s Anti-Fraud and Anti-Corruption Policy (2010) explains the measures WFP has in place to prevent fraudulent, corrupt or collusive practices, and its Financial Regulations (2010) detail the irregularities to be investigated by both the internal and external audit functions. In addition, WFP has adopted a legal framework which is in accordance with the United Nations Standard Code of Conduct and which specifies the disciplinary measures to be followed in cases of non-compliance with the code. On an annual basis, a minimum of three reports are presented to WFP’s Executive Board to alert it, as necessary, to financial irregularities and to track implementation of internal and external audit recommendations: the Audited Annual Accounts, the Annual Report of the Inspector General, and the Report on the Implementation of the External Auditor Recommendations. WFP has also introduced a system whereby senior managers provide the Executive Director with signed statements of assurance on the effectiveness of internal controls within their office, which highlight actions taken to address risks and to implement recommendations of oversight bodies.

While there is room to improve the timeliness of action on internal and external audit recommendations,[42] the Annual Performance Report for 2012 indicates that the number of outstanding medium-risk and high- risk internal audit recommendations decreased by 20% between 2011 and 2012. Moreover, the Office of the Inspector General has allocated increased resources to assess information received from WFP management on actions taken to implement audit recommendations; it will report on the results of this exercise to the Audit Committee in December 2013 and include the outcome in the Annual Report of the Inspector General in 2014.

MI 8.5 – Internal financial audits processes provide objective information

MOPAN donors at headquarters, the only respondent group consulted on this MI, considered WFP’s internal audit processes to be strong in providing objective information to its governing body (57% rated WFP strong or very strong, 32% as adequate, and none rated it lower).

WFP’s internal audit processes were assessed as very strong through document review. In accordance with WFP Financial Regulation 12.1, the Oversight Office was established by the Executive Director to carry out internal audits and investigations across the organisation. The Oversight Office’s charter, which was revised in 2012, emphasises the structural independence of the internal audit function and further defines the office’s authority, scope of work, and responsibilities. The charter also stipulates that the office’s director must report to the Executive Board annually on activities, and must include a summary of significant oversight findings, the status of implementation of internal audit recommendations, and a confirmation that the office had full organisational independence in conducting its activities. In 2011, an external quality assessment conducted by the Institute of Internal Auditors confirmed that significant improvements had been made since the previous review in 2006 and that WFP’s internal audit function operates in conformity with international standards; WFP was awarded the highest rating in 88% of assessed areas.

MI 8.6 – Effective procurement and contract management processes

MOPAN donors at headquarters perceived WFP’s procurement and contract management processes as effective: 55% rated WFP as strong or very strong, 23% as adequate, and only 5% as inadequate.

The document review rated WFP as very strong for its procurement and contract management processes. By virtue of its mandate as a provider of food assistance in emergencies, procurement of food, non-food (including contracts), and transport is at the very heart of the organisation’s business processes.[43] WFP has adopted a number of policies, directives, and manuals to ensure the timeliness and cost-effectiveness of its procurement practices, which are overseen by the Food Purchase Committee, the Bid Opening Committee, and the Committee on Commodities, Transport, and Insurance (CCTI). In accordance with WFP Financial Rule 112.14, food purchases are to be made on the basis of international competition, but procedures are in place to waive these requirements in case of sudden emergencies. To encourage local economies, preference is given to vendors from developing countries, provided that such procurement represents the best value and meets operational requirements; in 2012, almost 80% of food was purchased from developing countries.[44] In addition, WFP uses advance financing mechanisms and prepositioning of food before emergencies occur, which substantially reduce lead times in the delivery of food. WFP is also rolling out a system to ensure the quality of the food it delivers. The organisation reports annually to its Board on its procurement activities, which is a rare practice in the United Nations.

Nevertheless, the document review noted that WFP experiences challenges in accounting for food in the custody of implementing partners, due in part to deficiencies in its current commodity tracking system,[45] lack of timely reporting by some partners, and limited number of field monitors. Monitoring is further hampered by travel restrictions imposed by the United Nations Department of Safety and Security (UNDSS), which can limit WFP monitors’ access to partners’ warehouses and distribution sites in high-risk zones. WFP indicates however that its Logistics Execution Support System (LESS), which is being rolled- out in 2013, will improve supply-chain management, inventory accounting, and real-time tracking of procured commodities.

MI 8.7 – Strategies for risk management

WFP was rated strong by document review for its systems and practices for managing risks. Although WFP’s 2005 policy on enterprise risk management took many years to implement,[46] the organisation has taken multiple steps to strengthen its controls to manage risks in recent years. Notably, the organisation developed a risk management framework based on the Committee of Sponsoring Organizations of the Treadway Commission (COSO), an accepted international standard. In addition, WFP has been implementing risk registers across the organisation to measure risk and identify mitigation actions at headquarter, regional and country levels. Contextual, programmatic or institutional risks that are considered to pose a significant threat to the organisation are escalated in line with managerial structures, included in the corporate risk register, and presented to WFP’s Executive Management Group (EMG). Since the beginning of 2011, WFP’s Audit Committee and its Executive Board, which provide oversight on risk management, have also been briefed on risk on a quarterly basis. Moreover, in 2012, WFP approved its Risk Appetite Statement, which acknowledges that risk is an inevitable part of WFP operations, provides the basis for setting acceptable levels of risk tolerance and thresholds (to be communicated with partners and stakeholders), and contributes to the identification and implementation of mitigation actions.

Notwithstanding progress made by the organisation, internal audit reports point to further opportunities to strengthen risk management within WFP (for instance, by mapping risks related to fundraising from private sector donors, to ensure alignment between the risk profile and controls and to enable the risks to be escalated to the corporate level).[47] Additionally, further efforts may be required to ensure that risk registers are implemented across all business units (to meet WFP’s target of 100% of country offices with risk registers). While only 84% of offices currently have risk registers in place, these offices cover 98% of all operational resources managed.[48]

KPI 9: Using performance information

Finding 9: WFP’s use of performance information was perceived as adequate overall by surveyed stakeholders. The document review found WFP strong overall in using performance information but noted that it may take some time to fully implement its 2012 guidelines to use performance information in planning new interventions.

Figure 3.15 | KPI 9: Using performance information, ratings of micro-indicators

MI 9.1 – Using information to revise and adjust policies

The majority of donors (82%) at headquarters surveyed rated WFP adequate or above for using performance information from rapid onset situations, projects, programmes, sectors and countries to revise its organisation-wide policies. Only 5% rated it inadequate or below.

In the document review, WFP was rated very strong for using performance information to revise and adjust its policies. In 2011, a policy on WFP Policy Formulation was approved by the Executive Board. This document emphasises the importance of reviewing policies on a cyclical basis, stipulating that all WFP policies must be evaluated by the Office of Evaluation through a strategic or policy evaluation within four to six years of adoption. It also states that policies are to be updated in light of such performance information.

Although the adoption of the WFP Policy Formulation is fairly recent, revising and adjusting policies based on performance information is not a new practice for WFP. Indeed, many of the policies reviewed by the assessment team clearly highlight and explain how they have been updated following recommendations made in an evaluation. A few examples include: the WFP Private Sector Partnerships and Fundraising Strategy 2013-2017 (2013); the WFP HIV and AIDS Policy (2010); the WFP Gender Policy (2009); and the WFP Policy on Capacity Development (2009).

MI 9.2 – Using information to plan new interventions

All respondent groups other than direct partners were asked whether WFP uses performance information to plan new interventions: 76% rated WFP adequate or above, 1% rated it inadequate or below, and 23% responded ‘don’t know’.

The document review rated WFP adequate on this MI as not all project documents reviewed reported evaluative information, and for the ones that did, it was often quite minimal.

In 2012, WFP issued a programme design framework to guide all staff engaged in the formulation of new operations and programmes. The document instructs staff to consider information generated from a number of processes and data sources to develop good quality programming: vulnerability assessments (e.g. on food security, nutrition, market surveys, etc.); trends regarding shocks/risks; alignment along seasonal timelines and complementarity with partners; positioning according to strategies, policies and strategic frameworks; implementation capacity at all levels (WFP, government, co-operating partners); consultation with stakeholders; and performance information from evaluations, monitoring and best practices. With respect to performance information, the framework also indicates that project proposal documents for protracted relief and recovery operations and country programmes submitted for approval to the Programme Review Committee (PRC) must explain how the proposed intervention conforms to recently completed evaluations. In fact, the terms of reference for the PRC (2005) indicate that the Director of Evaluation who sits on the committee is responsible for reviewing the extent to which recommendations from previous project evaluations or relevant thematic and external evaluations have been taken into account. As the 2012 framework is relatively new it may take a while for these guidelines to be implemented in new proposals.

MI 9.3 – Proactive management of poorly performing initiatives

All survey respondent groups other than direct partners were asked whether WFP addresses poorly performing interventions proactively in order to improve their performance. Although more than half of respondents (53%) rated WFP adequate or above and only 14% rated it inadequate or below, 33% responded ‘don’t know’.

WFP was rated strong on this MI based on the documentary evidence reviewed. In 2004, WFP issued a directive requiring all country offices, regional bureaus and headquarter divisions to conduct reviews of their performance at least every six months. According to the most current procedures for performance reviews, offices and divisions must: assess achievements for each result and indicator against set targets; assess progress made in implementing activities against planned timelines; identify reasons, challenges, constraints, and successes which led to unexpected performance; identify lessons learned and best practices; and agree about future actions to improve expected results. The actions and associated risks are recorded in annual performance plans, which also include clear lines of accountability and timelines.

MI 9.4 – Evaluation recommendations are acted upon

MOPAN donors at headquarters, the only respondent group surveyed for this MI, rated WFP adequate for tracking implementation of evaluation recommendations reported to its governing body.

In the document review, WFP was rated strong on this MI. WFP’s Evaluation Policy (2008) stipulates that the organisation’s Executive Director is responsible for developing a mechanism that ensures follow-up actions to accepted evaluation recommendations are implemented, and that tracking information on implementation is reported to the Board. The policy also indicates that the Board is tasked with holding management responsible for implementing follow-up actions. Accordingly, each year, WFP’s Office of Evaluation submits the strategic, policy, impact, country portfolio and/or operation evaluations it has completed to the Board. These are all accompanied by a response from management specifying the actions they have agreed to take, along with the corresponding implementation deadlines. Since 2008, WFP’s Performance Management and Monitoring Division has reported to the Board annually on the implementation status of evaluation recommendations. As of June 2013, 79% of evaluation recommendations for the 2008-2012 period had been implemented.

KPI 10: Managing human resources

Finding 10: Survey respondents considered WFP’s human resource practices to be strong, and the ratings assigned by document review ranged from inadequate to very strong.

Surveyed stakeholders rated WFP strong on the aspects of WFP’s management of human resources they were asked to consider. However, ‘don’t know’ responses ranged from 23% to 43% for each MI.

The document review ratings were mixed for this KPI. Strengths observed included WFP’s clear procedures for enforcing its code of conduct, as well as its systems to ensure staff security. However, further improvements are required to increase the transparency and staff confidence in reassignment, promotion and recruitment decision-making processes.

Figure 3.16 | KPI 10: Managing human resources, ratings of micro-indicators

MI 10.1 – Results-focused performance assessment systems for senior staff

Based on a review of documents, WFP was rated adequate on this MI. In 2004, WFP adopted its Performance and Competency Enhancement Programme (PACE) to appraise the performance of its staff, including those at senior level. This system assesses each individual’s work outputs achieved and competencies demonstrated at the end of a given year, based on a workplan and indicator targets established by the individual and his/her supervisor at the beginning of the year and revised as necessary at the midway mark. The performance evaluation process occurs at two levels: an individual is first assessed by his/her supervisor, and this assessment is then reviewed by a higher ranking supervisor. Although past audits noted poor compliance with the PACE process, concerted efforts by management to respect deadlines resulted in a compliance rate of 88% in 2012, which represents an 11% increase from the preceding year. Two indicators to track WFP’s compliance with the PACE system have recently been added to the organisation’s Management Results Framework: one focuses on compliance rates achieved by all staff and one on compliance of senior staff. In both cases, the target for 2013 is to achieve full compliance.

In addition to the PACE system, WFP uses a 360° feedback mechanism to assess the strengths and weaknesses of senior managers as part of a leadership exercise offered by the Management Assessment Centre.[49] A senior manager’s supervisors as well as invited staff members are asked to provide comments on the manager’s performance. Following the assessment, the evaluated individual is offered six coaching sessions by the centre to build on strengths and improve upon weaknesses. The ratings of these assessments are also considered during promotion and reassignment decisions. Since 2006, nearly all senior staff members have participated once in the 360° review. The assessment team was informed that it is envisioned that senior managers will undergo this assessment every five years (provided sufficient funding is available), but the 2012 “Report of the External Auditor on the Management of Human Resources” recommended that such a mechanism, involving feedback from both supervisors and staff, be performed annually.[50]

MI 10.2 – Transparent system to manage staff performance

The document review rated WFP inadequate for the transparency of its system to manage staff performance. In 2011, WFP’s Human Resources Division revised the organisation’s career framework and adopted a directive for the promotion of professional staff based on past performance and future potential. However, the 2012 “Report of the External Auditor on the Management of Human Resources” noted that only 52% of eligible international professional staff were considered for promotion by the Professional Promotion Panel (PPP) based on recommendations by their managers. No records were found documenting the reasons why the remaining 48% had not been deemed suitable candidates by their managers, or whether the decisions to not propose them for consideration had been discussed with them. The report emphasises that lack of transparency within the current system undermines the credibility of decisions made:

“Our review of staffing decisions – reassignment, promotions and recruitment showed that the decisions were not adequately supported by documented justification and feel that a system of structured feedback to staff on reassignment/promotion decisions would increase transparency in the processes, and enhance among the staff, faith in the processes.”

MI 10.3 – Measures to ensure staff security

All respondent groups other than direct partners were asked whether WFP has appropriate measures in place to ensure staff security. Half of respondents rated WFP strong or very strong, 23% as adequate, and 3% as inadequate or weak.

In the document review, WFP was rated strong on this MI. WFP operates in high risk environments and the security of its staff is a major concern. WFP follows overall policy guidance provided for the entire United Nations Security Management System (UNSMS)[51] and has also developed specific directives to ensure strong accountability for implementing security measures within the organisation. For instance, WFP established a Security Management Policy in October 2011 which clearly defines responsibilities and accountabilities of WFP officials and personnel for managing security. In addition, in January 2013, WFP issued an Executive Director circular specifying delegation of authority for deciding whether to initiate or pursue operational activities, institute alternate work modalities, approve establishment, relocation, and security upgrade of premises, or approve travel of WFP personnel, and this depending on the level of risk involved. However, as noted in WFP’s Annual Performance Report for 2012, implementation of recommendations from security assessments in the field could be improved; at the end of 2012, a third of recommendations were outstanding.

MI 10.4 – Staff deployment adequate for the development of effective partnerships

All respondent groups other than direct partners were asked whether WFP keeps deployed international staff in post for a sufficient period to maintain effective partnerships at the country level. Overall, 68% rated WFP adequate or above and 9% as inadequate or below.

MI 10.5 – Code of conduct followed by staff

The survey included three questions in relation to this MI. The first asked all country-level respondent groups about staff compliance with the organisation’s code of conduct; overall, two-thirds of survey respondents rated WFP adequate or above and only 4% rated it inadequate or below. The second and third questions asked MOPAN donors at headquarters to rate WFP’s systems for monitoring compliance with the code of conduct and whether it reports transparently on this data. They rated WFP strong on both questions.

On the basis of the documentary evidence reviewed, WFP was rated very strong for having a code of conduct that is followed by its staff. All WFP staff are expected to abide by the United Nations’ Standard of Conduct for the International Civil Service and the Secretary-General’s Bulletin on Sexual Abuse and Exploitation in Humanitarian Crises and Other Operations. In addition, WFP adopted a Policy on Harassment, Sexual Harassment and Abuse of Authority (2007) as well as an Anti-Fraud and Anti- Corruption Policy (2010) which further detail roles, responsibilities and modalities for implementing appropriate conduct within WFP. Furthermore, in 2008, WFP established its Ethics Office, the main objective of which is to ensure that staff comply with the “highest standards of integrity”, in conformity with the aforementioned codes. The Ethics Office provides training to all WFP staff on the standards of conduct, and reports regularly to the Executive Director on its activities and on patterns and trends identified in its evaluations and assessments. WFP’s Oversight Office is responsible for investigating alleged cases of misconduct, and reports to the Executive Director and Director of the Human Resources Division and the latter determines and applies disciplinary measures as necessary in line with staff rules.

MI 10.6 – Rapid personnel deployment or surge mechanisms in place

For this MI on rapid deployment, all respondent groups other than direct partners were asked two questions: the first focusing on WFP’s use of in-house personnel, and the second on WFP’s use of externally accessible personnel. In both instances, WFP was rated strong.

The document review rated WFP as adequate for rapidly deploying personnel and having surge mechanisms in place, and noted that numerous changes to WFP’s related practices and systems are underway. In 2012, the organisation adopted a revised activation protocol indicating that mobilisation of necessary staff and resources is to be ordered within 24 hours following the activation of emergencies that require regional co-ordination or corporate capabilities (i.e. level 2 or level 3 emergencies). The document also notes that WFP follows a ‘no-regrets’ approach to emergency activation and deployment, whereby it prefers to mobilise excess capacity and then withdraw it, rather than send insufficient resources and fail to meet urgent humanitarian needs. Additionally, the document highlights that WFP’s deployment of staff during humanitarian system-wide emergencies is conducted in conformity with the Inter-Agency Rapid Response Mechanism (IARRM). WFP has also drafted an emergency preparedness and response training and deployment strategy and established a related working group, and is in the process of reinstating its emergency response roster (previously dismantled due to a number of inefficiencies) following recommendations made at WFP’s 2010 Global Meeting in Madrid and 2011 Global Meeting in Montreux.

While WFP establishes stand-by agreements for the rapid deployment of external partners should internal resources be insufficient, the “Annual Report of the WFP Inspector General” presented to the Board in 2012 noted that the use of external staff in emergencies is inadequate. The Inspector General’s 2013 report indicates that country offices have been receiving guidance to improve the recruitment of external staff.

KPI 11: Performance-oriented programming

Finding 11: WFP was rated adequate by survey respondents and document review for subjecting new initiatives to risk and benefit/impact analyses. The document review found that WFP does not have guidance in place to ensure that country office work plans or operational plans for projects/programmes consistently include the steps to be accomplished (with milestones, baselines and targets) to achieve or to monitor the intended humanitarian/development results.

Surveyed stakeholders recognised WFP as strong for conducting risk assessments prior to emergencies, and rated it adequate for risk and benefit/impact analyses performed for development interventions. The document review noted that many recently approved documents provide a frame to strengthen risk and benefit/impact analysis across the organisation. However, time is needed to determine whether these guidelines will be followed systematically.

Figure 3.17 | KPI 11: Performance-oriented programming, ratings of micro-indicators

MI 11.1 – New initiatives subject to risk and benefit/impact analysis

Survey respondents, which included donors at headquarters as well as respondents from recipient governments and peer organisations, rated WFP as strong for subjecting new humanitarian operations to risk analysis, but as adequate for conducting either risk or impact analyses for new development interventions.

In the document review, WFP was rated adequate on this MI. As noted in MI 8.7 regarding WFP’s strategies for risk management, country offices must maintain risk registers. A corporate directive[52] issued in October 2012 indicates that the risks to be considered and included in the register are categorised as programmatic, institutional and contextual – WFP defines programmatic risks as those related not only to the failure of reaching programme objectives, but also of potentially causing harm to others (e.g. by drawing beneficiaries into a conflict zone or by hurting fragile economies). Guidelines for vulnerability and food security analysis further specify that WFP should conduct an analysis of strengths, weaknesses, opportunities, and threats (SWOT) for appraising different programming options and give due consideration to negative impacts on the local social, environmental or economic situation (in essence a benefit/impact analysis). In line with these documents, WFP also requires, since 2012, that project proposal documents submitted for approval to the Programme Review Committee (PRC) specify programmatic risks and explicitly indicate in the case of emergency and protracted relief operations any potential negative impact on beneficiaries. Taken together, these guiding documents suggest that WFP has the necessary framework to ensure that risk and benefit/impact analysis are undertaken prior to initiation of projects and that these analyses are being looked at to inform decision-making. However, given that many of these guiding documents are recent, the ensuing practices cannot yet be considered to be fully institutionalised.

MI 11.2 – Milestones/targets set to monitor implementation

This MI was assessed by document review only and examined the use of work plans to track project or programme implementation; WFP was rated inadequate.

Although country offices are required to prepare work plans/operational plans for their projects/ programmes, there is no agreed format for these documents and the quality of the plans sampled for this assessment varied considerably. In general, they lacked detail on the steps to be accomplished (including milestones, baselines and targets) to achieve the intended humanitarian/development results. As noted by the OECD, such plans are important for gauging whether progress is being made in the project’s implementation and help “orient and motivate project staff and managers to the tasks at hand”.[53] Each country office does however also produce an Annual Performance Plan that clearly articulates the management-type results required to improve office performance and that can benefit project/ programme implementation.

KPI 12: Delegating authority

Finding 12: WFP was considered strong for delegating decision-making authority.

Survey respondents perceived WFP as strong in delegating authority to local levels, though country-level respondents indicated having limited knowledge of WFP’s performance in this area.

The document review found clear and very detailed information on the delegation of powers to the field level, and noted that the organisation is making additional improvements in this area through recent and on-going reforms.

Figure 3.18 | KPI 12: Delegating authority, ratings of micro-indicators

MI 12.1 – Key operational and management decisions made locally

All respondent groups other than direct partners were asked whether key operational and management decisions within WFP are delegated and decentralised in a manner appropriate to the context. Overall, 60% rated WFP adequate or above, but there was a high level of ‘don’t know’ responses from country level respondents (52% of donors in-country, 30% of recipient government respondents, and 36% of peer organisation respondents).

The document review rated WFP strong for its delegation of authority regarding key operational and management decisions. Over the 1996-2002 period, WFP shifted authority, responsibility and personnel to field offices closer to programme beneficiaries. Since then, audits and evaluations have noted that ensuring decisions are taken as close as possible to front line operations remains an on-going priority for WFP. In fact, as part of wider organisational reforms, WFP recently undertook a realignment exercise to rethink and redesign its organisational structure. As a result, regional bureaux are being strengthened so that they can better manage, oversee and support country offices, and country offices are being further empowered to act as ‘centres of gravity’ for the organisation. While clear delegations of authority are currently specified in the basic texts of the Programme Guidance Manual, the Financial Resource Management Manual, Executive Director circulars, and departmental, divisional and office directives, these are all in the process of being revised by WFP’s Legal Office as the new organisational design is being implemented.

KPI 13: Humanitarian principles

Finding 13: WFP’s approach to applying humanitarian principles was perceived as strong by survey respondents. The document review, which rated WFP adequate, recognised the challenges in balancing a principled humanitarian stance with ensuring rapid food delivery and noted some room for improvement in informing and training staff on the application of humanitarian principles during delivery of emergency food assistance.

Figure 3.19 | KPI 13: Humanitarian principles, ratings of micro-indicators

MI 13.1 – Clear procedures for in-house co-ordination

Donors at headquarters and in-country perceived WFP as strong overall for its in-house procedures to co- ordinate the various functions and roles of its mandate. However, 35% of respondents indicated having insufficient knowledge to respond.

Based on documentary evidence, WFP was rated adequate for co-ordinating the multiple functions and roles related to its dual mandate. WFP has established several tools and systems to co-ordinate and address its dual mandate, as described below.

In 2008, WFP developed the Country Strategy Document (CSD), a programming tool intended to help country offices better delineate their humanitarian and development activities and reduce overlap between operations in a given country. By 2011, however, CSDs had been adopted in only 21 of the 80 countries in which WFP operates and the Office of Evaluation recommended in its Annual Evaluation Report for 2012 that management reaffirm the importance of these documents and clarify their role in WFP’s governance, partnership, strategic and operational frameworks. In response, WFP’s management stated that it was committed to formalising CSDs as the primary governance tool for country portfolios as well as to improving mechanisms for their approval over the course of the 2014-2017 strategic plan.

WFP has established a corporate body to co-ordinate and transfer resources to emergency responses when needed: the Operational Task Force (OTF) begins mobilisation of human and financial resources, emergency stock, and equipment within 24 hours of activation of a Level 2 or Level 3 emergency response.

In 2009, WFP also updated its enterprise resource planning system (WINGS II) to transform it into a fully “integrated set of processes, systems, and data.” WINGS II has contributed to enhancing co-ordination for emergency response and rapid deployment by creating a global human resource system containing information on WFP’s 15,000 employees. It has also increased co-ordination in procurement and logistics activities as managers from country offices now have access to an integrated system with updated information on stocks and commodities.

MI 13.2 – On-going policy dialogue with partners on observing humanitarian principles

All survey respondent groups were asked whether WFP maintains on-going policy dialogue with partners on the importance of observing humanitarian principles in the delivery of humanitarian assistance, particularly in cases of conflict. Overall, 52% rated WFP as strong or very strong, 22% as adequate, and 11% as inadequate or below.

MI 13.3 – Humanitarian principles respected while delivering assistance

Survey respondents in all categories were asked two questions on this MI: whether WFP respects humanitarian principles while delivering humanitarian assistance, and whether it takes relevant corrective action when unable to fully implement these principles in its emergency and relief operations. WFP was rated strong on both questions. Overall, 56% of respondents considered WFP strong or very strong, 25% adequate, and 6% inadequate or below.

The document review rated WFP adequate on this MI. WFP officially endorsed the humanitarian principles of humanity, impartiality and neutrality in a policy in 2004 and reaffirmed its commitment to these in another policy on protection in 2012. The organisation also formalised its commitment to the principle of operational independence in its strategic plan for 2014-2017, a document which was endorsed by its Executive Board in June 2013.[54] WFP also provides written guidance on the application of the principles. However, the organisation has not established clear accountabilities for action and reports have indicated that staff are unclear about their responsibilities and accountabilities for applying the humanitarian principles.[55] While WFP’s Protection Training Manual (2009) includes a module on humanitarian principles, this training is not yet provided to all staff.

The assessment team acknowledges that applying humanitarian principles in practice is no small feat. There is an inherent programmatic tension between adhering to humanitarian principles and serving the humanitarian imperative of food delivery; neutrality is especially difficult to maintain when delivering food to remote insecure areas which require military escort; and maintaining a principled stance is perhaps that much more difficult for WFP given its complementary development mandate, which encourages it to engage in extensive partnership with government.[56]

3.3.4 Relationship management

WFP was seen to perform adequately or better in all areas of relationship management and was commended in particular for harmonising procedures with programming partners and supporting inter-agency plans and appeals. Programming to enhance livelihoods during the recovery and development phases was noted as an area for improvement.

Figure 3.20 below presents the overall survey and document review ratings for the five KPIs in the relationship management performance area.

Figure 3.20 | Performance area III: Relationship management, survey and document review ratings

Figure 3.21 shows the mean scores for the five KPIs for all survey respondents, and by respondent groups.

Figure 3.21 | Performance area III: Relationship management, mean scores by respondent group

KPI 14: Procedures consider local conditions and capacities

Finding 14: WFP was perceived to be adequate overall by survey respondents for ensuring that its procedures take local conditions and capacities into account. The document review noted that WFP performs better in protecting livelihoods during emergencies than in enhancing them during the recovery and development phases.

Figure 3.22 | KPI 14: Procedures consider local conditions and capacities, ratings of micro-indicators

Unless otherwise noted, all survey respondent groups were asked about the following micro-indicators.

MI 14.1 – Procedures easily understood and completed by partners

WFP was rated adequate overall for using procedures (for instance with respect to funding and reporting requirements) that can be easily understood and followed by its humanitarian partners.

MI 14.2 – Time for procedures does not affect implementation

When asked whether the length of time it takes to complete WFP’s procedures affects implementation, the majority of respondents (66%) rated the organisation as adequate or above and 15% as inadequate or below.

MI 14.3 – Ability to respond quickly to changing circumstances

Survey respondents rated WFP adequate overall for adjusting its country portfolio quickly to respond to changing circumstances.

MI 14.4 – Flexibility in implementation of projects/programmes

Nearly three quarters of respondents (72%) rated WFP as adequate or above for adjusting its implementation of individual projects/programmes as learning occurs.

MI 14.5 – Timely response to events/disaster situations

WFP’s ability to respond to rapid onset situations in a timely manner was perceived as strong by survey respondents: 63% rated WFP as strong or very strong, 24% as adequate, and 7% as inadequate or below.

MI 14.6 – Timely transfer of funds to organisations

Survey respondents were asked whether WFP transfers funds to its humanitarian partners in a timely manner. The majority (52%) rated WFP adequate or above, 10% rated it inadequate or below, and 38% responded ‘don’t know’.

MI 14.7 – Use of local capacities and resources

Country-level respondent groups were asked two questions for this MI: whether WFP ensures that it sufficiently uses local capacities, and whether it seizes opportunities to procure food locally or regionally (when cost-effective, timely and feasible). On both counts, WFP was perceived as strong.

MI 14.8 – Capacity development of local partners

Respondents based in-country held mixed views on WFP’s performance in ensuring that capacity development of local partners is undertaken. While 66% rated WFP as adequate or above, 21% rated it inadequate or below.

MI 14.9 – Protects and enhances livelihoods in the transition to development

On this MI all respondent groups were asked two survey questions: whether WFP engages with local partners and communities to protect and enhance livelihoods, and whether it uses appropriate modalities to promote government ownership of livelihood programmes in the transition to development. Although both questions were rated adequate overall, the second yielded a higher proportion of negative ratings (23% of respondents answered inadequate or below).

In the document review, WFP was rated adequate on this MI. Striving to protect and enhance livelihoods is covered by two WFP corporate level strategic objectives: SO1 focuses on saving lives and protecting livelihoods during emergencies, while SO3 focuses on restoring and rebuilding lives and livelihoods in post-conflict, post-disaster or transition situations. A strategy on Food Aid and Livelihoods in Emergencies established by WFP in 2003 provides the guiding framework for SO1. SO3 is currently guided by WFP’s From Crisis to Recovery policy (1998) and a 2004 note on transition from relief to development. According to the 2012 Programme of Work, these will be replaced by a Policy on Transition.[57]

In recent years, efforts have been made to strengthen staff capacity to design, implement and monitor livelihood programmes. Notably, WFP prepared a manual in 2011 on use of food assistance to create assets, which it updated in 2013 with context-specific guidance. The organisation is supporting the roll-out of this guidance through in-country training, but it may take some time to cover all the countries implementing food-for-asset programming. WFP has also been providing training and planning tools to governments on alignment between food for asset and other multi-sectoral programmes in order to enhance national ownership of the planning processes and coordinate implementing partners to deliver harmonised programmes.

While evaluations indicate that WFP has been effective in helping protect people’s livelihoods during emergencies and that its work on social protection is relevant and has potential for building resilience and promoting livelihoods, they also note that programmes are not yet leading to the desired changes in the longer term (i.e. to increased self-reliance of targeted beneficiaries in the recovery and development phases). Challenges faced by WFP include its short-term project modalities, funding shortfalls, and pipeline breaks. It has been suggested that stronger contingency planning could help the organisation better address these constraints and WFP is addressing this issue. Its Seasonal Livelihood Programming (SLP) consultative tool helps identify the programmatic responses required (when, where, and for whom) and the partner that can deliver such responses in both typical and crisis situations.

WFP reports that it is increasingly being called upon by governments and communities to move towards livelihood programming in support of resilience building efforts against shocks and hazards. While longer- term funding remains a challenge, the organisation indicates that its planning approaches for livelihood programming increasingly apply a theory of change and a scaling-up approach, thus allowing programmes to: (i) be sequenced in ways that ensure each activity safeguards the resilience gains being made and laying foundations for subsequent programmes; and (ii) expand so as to incorporate pre-identified areas where longer-term programmes are required as new funding becomes available.

KPI 15: Supporting inter-agency plans and appeals

Finding 15: All respondent groups rated WFP as strong for contributing actively to inter-agency plans and appeals. The document review rated it very strong.

Figure 3.23 | KPI 15: Supporting inter-agency plans and appeals, ratings of micro-indicators

MI 15.1 – Active contribution to inter-agency plans and appeals

All respondent groups were asked two questions in relation to this MI and rated WFP as strong on both questions: for contributing to inter-agency plans such as consolidated appeals and annual programming exercises in a timely manner, and for engaging fully in Common Humanitarian Action Plans (CHAP) in countries where this programming process is being used. However, the second question generated a high proportion of ‘don’t know’ answers (32%).

WFP was rated very strong on this MI through document review. There is evidence that WFP is actively engaged in the inter-agency process of needs mapping (often in a leadership capacity), and in working with cluster partners to develop a joint strategy, which in turn informs its individual operational plans. The organisation also plays a substantial role in the Consolidated Appeals Process (CAP); in 2012, the organisation participated in all 27 appeals, hosted the meeting of the heads of all Inter-Agency Standing Committee (IASC) member agencies or their representatives,[58] and launched the 2013 Consolidated Appeals Process. In fact, WFP is the largest appealing agency under the CAP, which gives an indication of WFP’s commitment to the process. Moreover, WFP is an active member of the IASC Sub Working Group on Consolidated Appeals as evidenced by its contribution to the publication of 52 appeal documents in 2012 (Mid-Term Reviews, flash appeals, 2012 appeals, emergency revisions, etc.), to the strategic review of 2011 and 2012 guidance appeals, and to the preparation of new common guidelines. In 2013, WFP also issued internal guidelines that describe WFP’s roles, responsibilities and accountabilities in IASC cluster activation, leadership, management, and deactivation, and that clarify the role and responsibilities of WFP Country Directors in the Humanitarian Country Team (the centrepiece of the humanitarian coordination architecture established by IASC Humanitarian Reform).

KPI 16: Contributing to policy dialogue

Finding 16: WFP’s contributions to policy dialogue and its respect for the views of partners were acknowledged as strengths.

As made explicit in its mission statement, WFP advocates policies, strategies and operations that directly benefit the poor and hungry in its dialogue with recipient governments and the aid community. Survey respondents considered WFP’s performance in this area as strong.

Figure 3.24 | KPI 16: Contributing to policy dialogue, ratings of micro-indicators*

* White diamonds indicate that the data required for the assessment was unavailable for UNICEF.

MI 16.1 – Reputation for valuable input to policy dialogue

Survey respondents rated WFP strong overall for providing valuable inputs to policy dialogue: 83% provided ratings of adequate or above.

MI 16.2 – Policy dialogue respects partner views

WFP was perceived as strong for respecting the views of partners during policy dialogue: 79% of survey respondents rated WFP adequate or above, and only 9% as inadequate or below.

KPI 17: Participation in the cluster system

Finding 17: Survey respondents rated WFP as strong overall for its participation in the cluster system, but noted room for improvement in the reliability of its financial forecasts of need. The document review rated WFP adequate in cluster leadership and noted that the organisation is actively engaged in strengthening accountability mechanisms for the clusters it leads or co-leads.

WFP received strong ratings on six of the seven MIs in this KPI from survey respondents, but there were high proportions of ‘don’t know’ answers (ranging on average from 22% to 29% on all micro-indicators). Country-level respondents were asked about WFP’s participation in the cluster system in four of the five countries assessed (Ethiopia, Indonesia, Mozambique and Pakistan), as Guatemala has not participated in the cluster system. The document review rated WFP adequate for its practices and systems for cluster leadership (the only MI in this KPI assessed by the document review).

Figure 3.25 | KPI 17: Participation in the cluster system, ratings of micro-indicators

MI 17.1 – Programmes reflect strategic priorities agreed to by the cluster

All respondent groups other than recipient governments were surveyed on this MI. Asked whether WFP adjusts its programmes to reflect the strategic priorities agreed to by the cluster, 72% rated WFP adequate or above, and only 7% inadequate or below.

MI 17.2 – Programme implementation compliant with technical recommendations of the cluster

All respondent groups other than recipient governments were asked whether WFP implements its programmes in compliance with the technical recommendations of relevant clusters. More than two-thirds (69%) rated WFP as adequate or above, and only 7% as inadequate or below.

MI 17.3 – Sufficient analytical resources and policy-level engagement dedicated to the cluster

Country-level respondents rated WFP strong overall for dedicating sufficient analytical resources and policy-level engagement to strategic activities in the clusters it leads or co-leads: 66% rated WFP adequate or above, and 10% as inadequate or below.

MI 17.4 – Dedicated staff for co-ordination of the cluster

When asked whether WFP provides dedicated staff to co-ordinate the clusters it leads or co-leads, the majority of country-level respondents (64%) rated WFP adequate or above, and 9% as inadequate or below.

MI 17.5 – Pertinent information circulated within the cluster

The majority of respondents (65%) based in the four countries assessed considered WFP adequate in ensuring that pertinent information is circulated within the clusters it leads or co-leads. Only 8% perceived WFP’s performance as inadequate or below.

MI 17.6 – Reliable forecasting of financial needs for the cluster

Country-level respondents rated WFP adequate overall for generating reliable forecasts of financial need for the clusters it leads or co-leads. The majority of respondents (59%) rated WFP adequate or above, and 11% as inadequate or below.

MI 17.7 – Effective practices and systems for cluster leadership

All respondent groups were asked whether WFP has effective practices and systems in place to act as cluster lead or co-lead. The majority of respondents (71%) rated WFP as adequate or above, with only 7% rating it inadequate or below.

WFP was rated adequate on this MI by document review. WFP leads two clusters (logistics and emergency telecommunications) and co-leads with FAO the Global Food Security Cluster. The depth of information available regarding the practices and systems WFP has in place varies for each of these three clusters. The evidence suggests that WFP is generally well-positioned to play its leadership roles, though there is room for improvement.

For each of the three clusters, WFP manages a support cell, developing norms and guidance and identifying best practices, providing training, creating partnerships, and deploying resources and staff to the field when needed. WFP also has a cluster co-ordinator in Geneva to engage in interagency forums on behalf of all three clusters. However, available evaluative evidence such as the 2012 Joint Evaluation of the Global Logistics Cluster indicates that WFP does not yet have systematic operational and programmatic accountability mechanisms in place to ensure appropriate monitoring and continuous improvement of its leadership in the clusters. It appears that WFP’s effectiveness in co-ordinating key cluster actors varies at the country level depending on the context (e.g. whether there is formal activation of the cluster, or based on the managers’ attitude and approach).

To remedy these issues, WFP has continued its practice of gathering regular feedback from cluster partners through surveys that ask them about their satisfaction with WFP’s co-ordination role and is now bolstering this with additional measures. For example, in August 2013 WFP approved a directive for staff that outlines roles, responsibilities and accountabilities regarding cluster activation, management, and deactivation, and the roll-out of this directive is being backed up with training. In addition, the Global Logistics Cluster’s strategy for 2013-2015 has made accountability one of its three key objectives and the Performance Review Project undertaken in 2013 should also help to establish a more systematic approach to engaging partners and staff in meaningful lessons learned exercises for all cluster operations.

The Global Food Security Cluster, for which there is at present the least available information, is scheduled to be assessed by WFP’s Office of Evaluation over the 2013-2014 period.

KPI 18: Harmonising procedures

Finding 18: Overall, WFP was rated strong by surveyed stakeholders and document review for harmonising its procedures with those of its programming partners. In the absence of documented corporate level data for the Paris Declaration indicators, the document review could not rate WFP on the extent to which technical co-operation is disbursed through co-ordinated programmes.

Figure 3.26 | KPI 18: Harmonising procedures, ratings of micro-indicators

MI 18.1 – Participation in joint missions

All survey respondent groups were asked to consider the extent of WFP’s participation in joint missions. More than half (51%) rated WFP’s performance strong or very strong, 32% as adequate, and 7% as inadequate or below.

Based on documentary evidence reviewed, WFP was rated strong for its efforts to conduct joint work, for instance with regard to needs assessments, vulnerability analysis and mapping, research studies, evaluations, field visits, meetings, and programming. As reported in its Annual Performance Report for 2012, WFP took part in 74 United Nations joint programmes in 47 countries in 2012, with its main partners being UNICEF, FAO, UNDP, WHO and UNHCR. As further testament to WFP’s strong commitment to streamlining processes and conducting its work jointly, an output indicator in WFP’s corporate strategic results framework that spans all five of the organisation’s strategic objectives seeks to measure the number of joint UN programmes and activities in which WFP is engaged.

MI 18.2 – Relevant information shared with partners

All respondent groups were surveyed on the extent to which WFP shares relevant information, for instance on needs and logistics, with humanitarian and other partners. More than half (56%) rated WFP strong or very strong, 27% adequate, and 10% inadequate or below.

MI 18.3 – Technical co-operation disbursed through co-ordinated programmes

All survey respondent groups other than direct partners were asked to rate WFP’s performance in providing technical assistance through co-ordinated programmes. More than three-quarters (76%) rated WFP adequate or above, and only 11% inadequate or below.

The document review had insufficient data to rate this MI. This MI is informed by Paris Declaration Indicator 4, which is compiled as a single value for the United Nations at the country level by the United Nations Country Team (UNCT). While WFP participated in the 2011 OECD Survey on Monitoring the Paris Declaration, WFP data are not available at the corporate level.

MI 18.4 – Promotion and implementation of UN system-wide reforms

All respondent groups other than direct partners were consulted on this MI. They rated WFP adequate overall for promoting and engaging in the implementation of UN reforms (i.e. Delivering as One and the Humanitarian Reform/Transformative Agenda). Although two-thirds of respondents rated WFP adequate or above and only 12% as inadequate or below, 21% responded ‘don’t know’.

The document review rated WFP strong on this MI. Since 2005, the organisation has actively engaged in restructuring the humanitarian system, as evidenced by its participation in the clusters at both the global and country levels. Since 2011, it has also demonstrated its commitment to improving leadership, co- ordination, and accountability within the humanitarian system, in line with the Transformative Agenda of the Inter-Agency Standing Committee (IASC): WFP is co-chair of the IASC Needs Assessment Task Force, of its Sub-Group on Accountability to Affected Populations, and of its Sub-Working Group on Preparedness. Internally, WFP has created a task force co-led by its Deputy Executive Directors for External Relations and Operations to ensure implementation of the Transformative Agenda. WFP has also played a leadership role in the development of inter-agency guidance and protocols for system-wide emergency response and co-ordination.

3.3.5 Knowledge management

WFP was commended for the independence of its evaluation unit and its transparency in making documents available to the public but was perceived as adequate in most areas of knowledge management. There is room for improvement in evaluation coverage of programming and in reporting on outcomes and programme adjustments based on performance information. The organisation has undertaken several initiatives to improve its performance in knowledge management.

Figure 3.27 shows the overall survey and document review ratings for the KPIs in the knowledge management performance area.

Figure 3.27 | Performance area IV: Knowledge management, survey and document review ratings

Figure 3.28 shows the mean scores for the four KPIs for all survey respondents, and by respondent groups.

Figure 3.28 | Performance area IV: Knowledge management, mean scores by respondent group

KPI 19: Evaluating results

Finding 19: Survey respondents considered WFP adequate in using evaluation findings and involving stakeholders in evaluations, the only aspects of evaluation they were asked about. The document review ratings varied, with the independence of WFP’s evaluation unit recognised as strong, and evaluation coverage of operations noted as an area for improvement.

This KPI examined whether WFP has the required systems and practices in place to conduct high quality evaluations of its delivery of results and make use of reported findings and recommendations to inform decision making.

Figure 3.29 | KPI 19: Evaluating results, ratings of micro-indicators

MI 19.1 – Independent evaluation unit

On the basis of documentary evidence reviewed, WFP was rated very strong for ensuring the independence of its evaluation unit. In accordance with recommendations of a 2007 UNEG-DAC peer review of its evaluation function, the organisation revised its evaluation policy in 2008, establishing within this document a framework to safeguard the structural, institutional and behavioural independence of evaluation at WFP.

WFP’s Office of Evaluation is structurally autonomous from management functions responsible for designing, implementing or monitoring policies, strategies and operations. Its Director has full discretion in preparing the office’s work programme (which is then presented to the Executive Board for approval), in managing the human and financial resources for evaluation, and in supervising and reporting on evaluations. The Office of Evaluation’s reports are submitted directly to the Executive Board, without prior clearance by WFP management. The Executive Board, which plays an oversight role, is responsible for ensuring that evaluations are carried out independently across the organisation. WFP requires that staff and evaluation consultants abide by the code of conduct for evaluators in the United Nations system, as specified in their job profiles or contracts, to ensure behavioural independence and avoid conflicts of interests.

MI 19.2 – Sufficient evaluation coverage of programming activities

WFP’s evaluation coverage of programming activities, which was assessed by document review only, was rated inadequate.

WFP’s evaluation policy (2008) highlights the highly variable nature of WFP’s portfolio of work, and stresses the importance of focusing evaluations on a representative, unbiased sample of operations to avoid misrepresentation. Four criteria are identified to guide selection of the operations to be evaluated in each region: the timing at which an operation is to be completed; the funding level it received; its size and programme category (e.g. emergency operation, protracted relief and recovery operation, or country programme); and the date when the last evaluation took place. In addition, the policy specifies the minimum coverage expected annually to ensure a statistically meaningful sample size. On a yearly basis, 30 operations are to be evaluated: of these, 10 are to be conducted by the Office of Evaluation, and the remaining 20 are to be managed by regional bureaux and country offices.

In practice, however, the number of Office of Evaluation-led operation evaluations dropped from 10 in 2009 to zero in 2012. The Office of Evaluation indicates in its Annual Evaluation Report 2011 (issued in 2012) that, in light of financial constraints, it chose to focus its work on higher-level evaluations (i.e. global in scope or covering multiple operations in one country) as “this approach promises greater value-added to WFP by providing evidence to inform strategic-level decisions regarding policies, country strategies or corporate strategies”.

In the absence of a management information system to track decentralised evaluations, it is unclear whether the annual target of 20 operation evaluations at the decentralised level has ever been reached.[59]

Nevertheless, WFP’s evaluation coverage is set to improve. In late 2012, the Executive Director approved a new funding mechanism resourced from direct support costs to finance a series of single-operation evaluations. The Office of Evaluation’s work programme for 2012-2015 indicates that 12 operation evaluations are to be conducted in 2013, a number that will increase to 24 in 2014 and to 30 from 2015 onward. In line with WFP’s new organisational design, the series of single-operation evaluations will be handed over to regional bureaux and country offices by 2016.

In addition to operation evaluations, the Office of Evaluation introduced country portfolio evaluations (CPEs) in 2008-2009 that consider all WFP operations and activities in a given country over a specific period of time to inform WFP’s local strategic positioning. In the last three years, there have been three or four CPEs conducted annually.

Since 2010, the organisation has also conducted two to four impact evaluations per year. According to WFP, “these are evaluations conducted in series, on selected programming activities across multiple countries [that] probe deeper into the outcomes and impacts of specific programme activities covering multiple operations in each country”.

MI 19.3 – Quality of evaluations

This MI was assessed by document review only, and examined the existence and application of evaluation quality assurance mechanisms (i.e. it did not assess the quality of actual evaluations); WFP was rated adequate.

Quality is one of four evaluation principles in WFP’s 2008 evaluation policy; it is central to ensuring that the data used to draw conclusions are credible and thereby permit the use of evaluation findings for learning and decision-making purposes. As a building block for implementing the evaluation policy and to ensure the quality of evaluations conducted, WFP initiated its Evaluation Quality Assurance System (EQAS) in 2007. This system is based on norms and standards promoted by the following three evaluation networks: the United Nations Evaluation Group (UNEG); the Active Learning Network for Accountability and Performance (ALNAP); and the OECD-DAC Evaluation Network. EQAS maps out the required steps for quality assurance during the evaluation process and provides templates for the different evaluation products. These tools are meant to be applied systematically to evaluations at WFP. While there is clear evidence that the Office of Evaluation has implemented the EQAS procedures in all its evaluations,[60] there is currently no corporate mechanism in place to ensure that the EQAS is used in all decentralized evaluations. To help ensure the continued relevance and high quality of WFP evaluations, the Executive Director requested that the United Nations Evaluation Group (UNEG) and the OECD-DAC Evaluation Network conduct a second peer review of WFP’s evaluation function in 2013, with recommendations expected in 2014.[61]

MI 19.4 – Use of evaluation findings to inform decisions

MOPAN donors at headquarters were the only respondent group surveyed on whether WFP uses evaluation findings in its decisions on programming, policy and strategy; the majority (80%) rated WFP adequate or above.

MI 19.5 – Beneficiaries and stakeholders involved in evaluations

All respondent groups other than donors in-country were consulted on this MI. The majority of respondents (70%) rated WFP adequate or above for involving its direct partners and beneficiaries in evaluations of its projects or programmes.

Based on the documents reviewed, WFP was rated adequate on this MI. WFP’s evaluation policy (2008) requires that beneficiaries and other stakeholders be involved in the evaluation process. In the evaluations reviewed (which included those led by the Office of Evaluation and decentralised evaluations), there was evidence that beneficiaries and other stakeholders were included in the data collection phase through interviews, focus groups, and surveys. In some evaluations reviewed, stakeholders were also involved in the inception and design phases of the evaluation, but this practice was not systematic.

The “Report of the External Auditor on Working with Co-operating Partners” (2013) noted cases in which there was no dialogue with beneficiaries during monitoring of operations in the field (referred to as Beneficiary Contact Monitoring). The Annual Evaluation Report for 2012 (2013) noted room for improvement in the feedback mechanisms in place to communicate evaluation findings to beneficiaries.

KPI 20: Presenting performance information

Finding 20: Survey respondents considered WFP strong in reporting on outcomes and against the results in its organisation-wide strategy, the two areas they were asked to rate. The document review provided a range of ratings on the five areas of performance it examined. While it commended WFP for reporting on its use of performance information to adjust its organisation-wide policies and strategies, it considered WFP inadequate in reporting on outcomes and on use of performance information to adjust programming at the country level.

This KPI focuses on how well WFP’s reporting captures performance information related to its effectiveness. It should be read in tandem with KPI 9 regarding use of performance information in section 3.3.3 on operational management. WFP’s reporting on results is also examined further in Chapter 4.

Donors at headquarters and recipient government respondents were the only groups surveyed for this KPI.

Figure 3.30 | KPI 20: Presenting performance information, ratings of micro-indicators

MI 20.1 – Reports on achievement of outcomes

Surveyed respondents were asked to rate WFP on whether it reports to its governing body on its performance, including on outcomes achieved. Although the majority (71%) rated WFP adequate or above and only 9% as inadequate or below, 21% responded ‘don’t know’.

The document review rated WFP inadequate on this MI. Since 2003, WFP has produced annual performance reports (APRs) that provide information on WFP’s achievement of organisation-wide programming results and management practices; these serve as a key accountability and learning tool for WFP as well as an oversight mechanism for the Executive Board and donors. While the most recent report, the Annual Performance Report for 2012, includes and discusses programming data at the output level, this information is not clearly linked to outcome results other than for a few country-level examples. While these are shared results (benefitting from contributions of other actors such as governments, United Nations organisations, non-governmental organisations, and private sector partners), it is difficult to ascertain WFP’s organisation-wide contribution to humanitarian and development outcomes.

Annex II-a of the APR for 2012 presents data for each outcome indicator but does not report against targets set in the SRF. Furthermore, little to no information is presented on the change in behaviour or action observed in targeted populations for each outcome.[62] Comparisons of trends across years are also complicated by the fact that indicators have changed over the 2008-2013 cycle as learning has occurred and improvements have been made.

A key challenge for WFP’s reporting on results is the lack of available outcome data. The APR for 2012 indicates that the percentage of projects reporting on outcome indicators as planned increased to 77% in 2012. However, as discussed further in Section 4.2.2, not all of these projects reported values that WFP could use to assess progress on its contribution to organisation-wide outcome results (nearly a third of values had to be omitted from calculations). This problem is recognised by WFP; in its management response to the 2012 annual evaluation report, it acknowledged that:

“The appropriate [outcome] monitoring frequency, the quality and comparability of data, and the use of non- standardized monitoring processes and systems continue to pose significant challenges.”[63]

To improve its outcome monitoring and reporting, the organisation will develop a new outcome measurement strategy for the 2014-2017 strategic plan. It has also taken steps to strengthen the monitoring and evaluation (M&E) capacities of WFP staff in the field as part of its new organisational design, notably by creating M&E expert positions at regional bureaux in support of country offices.

MI 20.2 – Reports on performance using data obtained from measuring indicators

This MI, which was rated by document review only, received a rating of adequate.

WFP presents data for all its programming outcome indicators in the APR. Although there are some shortcomings, as noted in the previous MI (e.g. not reporting on targets), the assessment team commends WFP for its transparency. Since 2010 APRs have also presented comprehensive indicator data for WFP’s management results framework, including information and trends across multiple years. In addition, the APRs present indicator data on WFP’s corporate contribution to meeting the eight millennium development goals (MDGs).

MI 20.3 – Reports against its organisation-wide strategy, including expected results

Nearly three-quarters of respondents (73%) considered WFP to report adequately or better against its corporate strategy.

WFP was rated adequate by document review on this MI. In addition to the APRs, which report on WFP’s strategic results framework and management results framework, the organisation presented a Mid-Term Review of the 2008-2013 strategic plan to the Executive Board in 2012 that provided summary findings and recommendations based on the first four years of the plan’s implementation, and which informed the development of the 2014-2017 strategic plan. It is not clear from the documents reviewed whether WFP intends to complete a final review of the 2008-2013 plan to demonstrate cumulative achievements over the planning cycle; such a document would be valuable for both accountability and learning purposes.

MI 20.4 – Reports on adjustments to policies/strategies based on performance information

In line with its practices for using information to adjust policies (see MI 9.1), WFP was rated strong by document review for reporting on adjustments made to organisation-wide policies and strategies based on performance information. WFP has instituted a policy-development cycle in which evaluation is embedded; evaluations are timed to inform new policy or to assess an existing policy’s effectiveness within four to six years following its adoption. As of 2012, the Compendium of WFP Policies Relating to the Strategic Plan (a report presented to the Executive Board annually) provides an overview of changes made to policies and highlights those that are due for an evaluation. Information on adjustments is also reported in the revised policies themselves. In addition, the Report on the Implementation of the External Auditor Recommendations and management responses to evaluations presented to the Executive Board each year also provide evidence of changes WFP is making to its policies and strategies based on performance information.

MI 20.5 – Reports on programming adjustments based on performance information

This MI, which examined WFP’s reporting on adjustments in country-level programming based on annual performance information, was assessed by document review only. WFP was rated inadequate on the basis of the documents reviewed.

As noted in the text for MI 9.3, WFP has strong processes in place for adjusting programmes in response to performance information. However, the review of available documents suggests that WFP is not sufficiently capturing these changes in its country level reports.

Annual completion of the standard project report (SPR), WFP’s main project-level data collection tool, is mandatory for all WFP projects. As noted in the Standard Project Report 2012 Guidance Manual, these annual performance reviews are an important accountability mechanism for donors, and serve to demonstrate results achieved, as well as to highlight challenges and successes for each operation. Based on the sample of SPRs reviewed, it is not clear however how the documents are being used to systematically inform strategic and budgetary adjustments within concerned projects. In fact, a review of budget revision documents (which highlight not only budgetary but also strategic changes) for the same sample of operations generally referenced contextual factors (for instance, new government policies or migrating populations) or external assessments to justify the requested programming change. The project’s own performance was rarely mentioned as the reason for a change in programming; for example, some budget revision documents noted that they were informed by lessons learned in the project, but did not identify these lessons.

WFP indicates that its project approval process systematically takes into account evidence from evaluations that are relevant to the project in question, and that its process for allocating extrabudgetary and advance financing resources to emergency operations is based on an analysis which considers each project’s ability to meet planned objectives and its resulting impact. However, documents demonstrating decisions made by WFP’s Project Review Committee and Strategic Resource Allocation Committee are strictly confidential in nature and were thus unavailable to the assessment team for review.

KPI 21: Disseminating lessons learned

Finding 21: WFP is considered adequate in disseminating lessons learned. The organisation does not yet have an integrated system for knowledge management but is working on several initiatives to improve in this area.

Figure 3.31 | KPI 21: Disseminating lessons learned, ratings of micro-indicators

MI 21.1 – Reports on lessons learned based on performance information

Survey respondents considered WFP adequate for identifying and disseminating lessons learned from performance information: the majority of respondents (69%) rated WFP adequate or above, and 14% inadequate or below.

WFP was rated adequate by document review on this MI. Despite several knowledge-sharing initiatives undertaken by the organisation in recent years (described below) and progress made in its monitoring of information management, WFP does not yet have an integrated system for knowledge dissemination, and its use of lessons learned to inform decision making could be improved.

As identified in the 2008 evaluation policy, the Office of Evaluation bears responsibility for documenting and disseminating lessons learned and best practices. It has put in place a knowledge management initiative, Closing the Learning Loop, to enhance dissemination of evaluation findings and to establish a corporate repository of lessons learned. Under the Preparedness and Response Enhancement Programme (PREP), WFP has also been implementing a knowledge management strategy to ensure that lessons learned from past emergencies are disseminated and used. In addition, the WFP Centre of Excellence against Hunger, a global knowledge repository, was created in partnership with the government of Brazil in 2011 to enable capacity development of national governments in the areas of school feeding, nutrition, and food security. In 2012, WFP introduced a new indicator in its management results framework to track reporting on lessons learned; data indicate that 90% of reports presented to the Board in that year referred to lessons learned.

In 2012 WFP commissioned an exploratory diagnosis of its knowledge management needs and practices and performance in comparison to other organisations. The results of this diagnosis will inform the design of a knowledge management strategy and provide input on how it should be implemented. One of the initial findings from the consultation is that “knowledge management will require cultural transformation at WFP”.

MI 21.2 – Lessons shared at all levels of the organisation

All respondent groups other than peer organisations were consulted on the extent to which WFP provides opportunities for lessons from practical experience to be shared at all levels across the organisation. The majority of respondents (53%) rated WFP adequate or above, and 14% inadequate or below. A high proportion of respondents (33%) answered ‘don’t know’.

KPI 22: Availability of documents

Finding 22: The document review rated WFP very strong for making key documents available to the public.

Figure 3.32 | KPI 22: Availability of documents, ratings of micro-indicators

MI 22.1 – Key documents available to the public

The document review rated WFP very strong on this MI. WFP’s Directive on Information Disclosure (2010) as well as its Policy for Disclosure of Oversight Reports (2012) are publicly available on line. WFP also shares a multitude of key documents on its website, including Executive Board meeting minutes and decisions, performance information reports, strategies, financial information reports, and evaluations. While most of these documents are translated into Arabic, French, and Spanish, a number of these (including WFP’s strategic plan, annual performance reports, and WFP’s general financial regulations) are also translated into Chinese and Russian. In efforts to further improve its transparency, WFP now publishes internal audits and investigation reports on its website. WFP’s commitment to becoming a more transparent organisation is also demonstrated by its adherence to the International Aid Transparency Initiative (IATI) in 2012; WFP now complies with IATI reporting standards in its annual performance reports.

4. Main Findings: Humanitarian and development results component

4.1 INTRODUCTION

This chapter presents the results of the 2013 Common Approach assessment of WFP in measuring and reporting on humanitarian and development results. It includes three key performance areas:

•  Section 4.2: Evidence of progress towards organisation-wide results (KPI A)

•  Section 4.3: Evidence of contributions to country-level goals and priorities (KPI B)

• Section 4.4: Relevance of the organisation’s objectives and programme of work to country-level stakeholders (KPI C)

The assessment of this component uses the same “traffic light” colours used in the organisational effectiveness component but applies a simplified 4-point scale. The methodology is explained in Volume II, Appendix I.

Figure 4.1 provides a summary of the assessment of the three KPIs. The detailed findings on each KPI are presented in the sections that follow.

4.2 EVIDENCE OF WFP’S PROGRESS TOWARDS ORGANISATION-WIDE RESULTS

4.2.1 Overview

This section presents the results of an assessment of evidence of WFP’s progress towards organisation-wide results. KPI A suggests that an effective organisation should demonstrate progress towards organisation- wide, institutional outcomes. These are usually related to the organisation’s strategic objectives. In this assessment of WFP, the significant challenges facing the organisation (as well as other organisations engaged in humanitarian work) in measuring and documenting higher-level change in humanitarian programming have been recognised and taken into account. The assessment draws on available evidence on the organisation’s achievement of results.

WFP’s mandate and core functions

As noted in the profile of WFP (see Section 1.2), the heart of WFP’s work is to address the needs of the most vulnerable hungry poor economically, effectively and efficiently, and to advocate policies, strategies and operations that directly benefit the poor and hungry. The organisation operates across the continuum from emergency relief to development to protect human life and build self-reliance of people, communities and government.

WFP’s Strategic Plan for 2008-2013 marked a historical shift as it repositioned the organisation from a food aid to a food assistance agency. The change was made to address the evolving nature of the hunger challenge and of the environment in which WFP operates (e.g. global market shocks and climate change). The plan emphasises the need for increased analysis of the causes which lead to hunger to help define responses that are appropriately tailored to the local context, and calls for use of a wider range of tools to address these. For instance, rather than relying solely on the delivery of in-kind food, WFP is now piloting the use of cash and voucher transfers to address food needs.

Figure 4.1 | Humanitarian and development results component – overall ratings

WFP’s 2008-2013 strategic plan is centred on five strategic objectives. These are to:

• Save lives and protect livelihoods

• Prevent acute hunger and invest in disaster preparedness and mitigation measures

• Restore and rebuild livelihoods in post-conflict, post-disaster or transition situations

• Reduce chronic hunger and undernutrition

• Strengthen the capacities of countries to reduce hunger, including through hand-over strategies and local purchase

During a seminar organised by WFP’s Secretariat and attended by its member states in May 2011, it was determined that: emergency operations would primarily serve to address strategic objective 1; protracted relief and recovery operations would focus mainly on strategic objective 3; and development country programmes would focus on strategic objective 4. Strategic objectives 2 and 5 were considered relevant across all three programme categories.[64]

WFP’s results measurement and reporting at the organisation-wide level[65]

WFP’s corporate strategic results framework is described as bridging or aligning country-level monitoring and reporting with the five organisation-wide strategic objectives. As stated by WFP:

“It enables WFP to track outcomes and outputs at the project level, which can then be aggregated to show achievements at the corporate level. In this way, it provides the basis for accountability of actual country- level activities against planned activities aligned with the Strategic Plan (…).”

The organisation defines outputs as the conversion of inputs into physical results for specific WFP activities, and highlights that these provide measurable performance results guiding the results chain from outputs to outcomes and goals. Given that implementation of activities is conducted in close partnership with other humanitarian and development actors (governments, United Nations agencies, etc.), outcomes and impacts of programming are shared and not the sole domain of WFP; these high-level results cannot be attributed to WFP. However, the outcome indicators and targets in WFP’s organisation-wide strategic results framework refer to its contribution to outcome-level results.

WFP tracks progress on its corporate strategy by aggregating results from each of its field operations. Emergency, protracted relief and recovery, and development interventions are designed to align closely with the corporate strategic results framework: outputs, outcomes and indicators for each intervention must closely mirror those in the organisation-wide framework. Standard project reports (SPRs) are prepared on an annual basis for each of these operations. Data are reported on project outputs and outcomes using standardised indicators[66] and submitted through an online system (DACOTA). The SPRs are reviewed by regional bureaux and the Performance and Accountability Management Division (RMP) at WFP headquarters; country offices are consulted for revisions when inconsistencies or technical issues are found.

Not all five of WFP’s corporate strategic objectives, twenty outcomes, and nine outputs apply to all situations and all countries – rather, relevant expected results are determined based on specific needs and priorities in a country or region and on WFP’s comparative advantage at that particular location and time. Hence, only those operations which committed to reporting on a given expected result are expected to do so in their SPRs.

WFP’s annual performance reports (APR) present data from the SPRs on progress towards organisation- wide results. The APR includes SPR data on the number of projects that have shown improvement, but does not report on the targets set for the SRF outcome indicators. The APR also presents information on performance from independent project evaluations and examples of successes and challenges.

Data used for this assessment[67]

The assessment of KPI A is based on survey data from donors at headquarters and on a review of documentation shared by the organisation. The main documents consulted included: WFP’s 2008-2013 strategic plan and corresponding strategic results framework; annual performance reports (APRs) covering results achieved from 2008 to 2012;[68] the 2012 Mid-Term Review of the strategic plan; strategic, policy and impact evaluations; annual evaluation reports prepared by WFP’s Office of Evaluation; as well as audits conducted by WFP’s external auditor. The Mid-Term Review, evaluations and audits were reviewed to find complementary evidence and help validate reported achievements.

Attention was paid to the following elements: quality of results statements, including indicators, baselines and targets; the quality and consistency of evidence presented to substantiate the results achieved, including an assessment of contribution; and the evidence of progress towards organisation-wide outcomes reported by the organisation.

4.2.2 Evidence of extent of progress towards organisation-wide results

This section provides an overall rating for KPI A based on documentation reviewed and the survey data. It also includes assessments of the quality and consistency of reports on organisation-wide results and of evidence of WFP progress towards its expected corporate results and strategic objectives.

Overall assessment

Figure 4.2 presents the overall rating for this KPI based on the review of WFP’s reports on organisation- wide results (including evaluations) and on perceptions of surveyed stakeholders.

MOPAN donors at headquarters considered WFP’s delivery of results in line with its five strategic objectives as adequate overall, and expressed particular confidence in its performance related to saving lives and protecting livelihoods. WFP evaluations also attest to the organisation’s strong performance in achieving immediate, short-term results in emergencies, but highlight that there is much less available evidence to demonstrate achievement of medium and longer-term results. In fact, the lack of comparable data to monitor outcome-level results is consistently mentioned in reports and evaluations as impeding clear assessment of WFP’s performance. This is corroborated by the analysis of WFP’s corporate self-reported data in this report, which illustrates that the evidence base to assess WFP’s performance is often too limited to confidently demonstrate substantial contributions to outcome-level change.

Nevertheless, WFP has made significant progress in its results reporting over the 2008-2013 strategic plan cycle, with country-level reporting rates on outcome indicators on the rise (thus improving aggregate organisation-wide data). This is especially important given that the Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP) recounts in the 2012 “State of the Humanitarian System” report that very few examples of true outcome/impact evaluation of humanitarian assistance exist, that the issue of monitoring remains a key weakness in the humanitarian system, and that humanitarian organisations primarily report evidence of outputs as outcomes/impacts are more difficult to measure, especially in continually fluctuating humanitarian contexts.

Figure 4.2 | KPI A: Evidence of the extent of progress towards organisation-wide results, overall rating

Quality of WFP’s measurement and reporting on organisation-wide results

Finding 23: WFP’s corporate performance reports do not yet provide a holistic account of the organisation’s performance, largely owing to gaps in outcome monitoring data. WFP is aware of its monitoring shortcomings and is actively engaged in addressing these.

WFP has exhibited a strong commitment to institutionalising results-based management and this is reflected in the quality of its corporate strategic results framework (see KPI 2 in Chapter 3). However, challenges and gaps remain in its monitoring and reporting systems. The Mid-Term Review of the current strategic plan notes that WFP’s monitoring systems were designed to track food movements and beneficiary coverage rather than the effect of food assistance on long-term hunger, and that there has been a tendency to focus monitoring primarily on outputs rather than outcomes. In terms of reporting, the assessment team observed that the organisation’s annual reporting discloses outcome indicator data, but that the measurements provided are not aligned with the targets in the corporate strategic results framework.

WFP is aware of these monitoring and reporting challenges and has been working to resolve these issues, notably by developing a monitoring and evaluation strategy (first adopted for the 2011-2013 period and revised for 2012-2014), rolling out an organisation-wide monitoring platform that will track results in real- time, and updating guidance and strategies for outcome measurement.

Figure 4.3 presents a summary assessment of the quality of the organisation’s systems and practices for measuring and reporting on its organisation-wide results. The criteria represent elements of good results reporting (see Volume II, Appendix I for full descriptions). The organisation is assessed according to whether it has met, partially met, or not met the criteria.

Figure 4.3 | WFP’s measurement of and reporting on organisation-wide results

1. ‘Theory of change’ is understood in the sense defined by Rist and Morra Imas (2009) as,“a representation of how an intervention is expected to lead to desired results” and in the sense defined by Michael Quinn Patton who has stated that a theory of change is more than the sequential order of results statements presented in a logic model; it requires key assumptions related to the results chain and context (e.g. policy and environment), and important influences and risks to be made explicit - Qualitative Research and Evaluation Methods (2002).
2. This refers to the existence of reports on outputs as defined by the OECD (i.e. lower level results). Some MOs use different terminology for the various levels of results.
3. This refers to the existence of reports on outcomes as defined by the OECD (i.e. higher level results). Some MOs use different terminology for the various levels of results.
4. ‘Reporting according to a theory of change is understood to mean the extent to which organisations provide a narrative describing the actual implementation process and results achieved in relation to that foreseen in the initial ‘theory of change’.
5. According to Rist and Morra Imas, The Road to Results – “Reliability is the term used to describe the stability of the measurement – the degree to which it measures the same thing, in the same way, in repeated tests.”

Strategic planning and theory(ies) of change

WFP’s strategic shift from food aid to food assistance is based on a rationale that is explained briefly in public documents such as the 2008-2013 strategic plan. However, these documents do not provide complete theories of change explaining the underlying logic of WFP’s corporate-level programming (i.e. the linkage between inputs, activities, outputs, outcomes and impact) or the assumptions on which programmes and new approaches are based.

WFP’s Office of Evaluation has reconstructed theories of change for some impact evaluations (see sidebar).

A synthesis of four strategic evaluations prepared by the Office of Evaluation in 2012 suggests that WFP’s shift to food assistance is relevant to the on-going and dynamic changes in the organisation’s operating environment, and that WFP is undergoing positive and expansive change to adapt and innovate in accordance with its new strategic direction.[69]

However, the document also indicates that greater clarity is needed in WFP’s conceptual frameworks to allow the organisation to better prioritise its programming and develop a common understanding with partners and other actors about its role and positioning. The report notes that greater clarity could help address concerns about duplication of roles or mission creep – concerns which were also voiced by respondents in the MOPAN survey (see Section 3.2). By explicitly formulating theories of change, WFP could strengthen the clarity, focus and effectiveness of its programmes, bringing critical thinking to bear on the assumptions around its programming and thus increasing transparency about its operational modalities and mechanisms to deliver results.

Measuring results

Given systematic shortfalls in the funding of WFP operations (in the range of 41% to 46% over the 2009-2012 period), the corporate targets for outcome indicators presented in the WFP strategic results framework appear ambitious. Many require that 80% of the projects that included the indicator in their logframe demonstrate achievement of their project-level target. WFP has also noted that Country Directors often need to take mitigation measures to adjust programme implementation when an operation receives only partial funding. For instance, they must routinely prioritise or suspend activities within a project, reduce ration size, prioritise geographical areas being covered, adjust distribution cycle periods, and/or adjust beneficiary caseloads.[70] All of these actions make it more difficult for WFP to meet its planned targets at the project level, which in turn negatively impacts the aggregate results at the corporate level.

The “Report of the External Auditor on Management of Projects” (2011) noted that WFP could improve its performance and better respond to funding shortfalls by planning multiple scenarios for programme implementation based on resourcing trends/situation. The report also indicated that the current method for calculating beneficiaries reached can be misleading: When resources fall short, beneficiaries may receive smaller rations and over shorter periods than originally planned, yet in a standard project report a beneficiary receiving a single ration in a year will be counted the same as one who received daily rations for the entire year. In response to these issues, WFP indicated that it is developing guidelines for multiple- scenario project implementation and that it has developed a corporate monitoring and evaluation tool (COMET) that tracks ration days in addition to beneficiary numbers (COMET is expected to be fully rolled out by the end of 2014).

WFP’s external auditor recommended in 2011 that projects should set aside funds for baseline studies and evaluations and that it should be mandatory to use these funds to ensure reliable data are collected to track achievement of results. As of June 2013, WFP indicates that earmarking of funds for monitoring and evaluation within projects is being encouraged during its project review process.

As noted in the 2012 Mid-Term Review of the strategic plan, comparing results across the years of the 2008-2013 strategic plan is difficult at the moment, particularly at the outcome level, as the indicators in the corporate strategic results framework have been adjusted and refined each year on the basis of new experiences and lessons learned. This practice attests to WFP’s strong commitment to improving performance measurement but represents a challenge in the short term.

In addition, the low proportion of usable data for certain outcome indicators undermines the robustness of WFP conclusions on progress towards higher level results.[71] See section below on Assumptions in WFP’s Annual Performance Reporting; also see Volume II, Appendix X.

Reporting on and communicating results

WFP demonstrates a high level of transparency by sharing detailed data on outcome indicators in its annual performance reports (APRs), which are submitted to the Executive Board and made public on the organisation’s website. In an annex, the APRs also include an explanation of the quantitative methodology used to determine progress in achieving strategic objectives. In this regard, WFP’s practices set a very high standard, as few other organisations are as transparent in their reporting.

However, the APRs do not report on the achievement of outcomes using the indicator targets specified in the strategic results framework. While the corporate targets seek to determine whether a sufficient proportion of projects have achieved their targets (generally set at 80%), the reported data focuses only on whether improvement has been observed (i.e. the reports present information on the direction of change, but not on whether targets are being met). Moreover, WFP’s reporting at the outcome level does not tell a clear story of the effect its assistance has had on the lives of its beneficiaries.

In addition, the methodology upon which WFP’s performance reporting at the outcome level is based rests on some implicit assumptions that may not be met. For example, WFP is assuming that the performance observed in a subset of projects is representative of the performance achieved by all projects. From a methodological perspective, this assumption is questionable given that this is not a randomly selected sample. See the section on assumptions that follows below for additional details.

Taking all of this into account, WFP’s data on performance could be presented more clearly in the APRs. Notably, indicator data could be presented alongside its corresponding results statements to facilitate the interpretation of performance information.

Assumptions in WFP’s Annual Performance Reporting

The table below recreates a portion of the outcome indicator table presented in annex II-B of WFP’s Annual Performance Report for 2012. It presents two of the six corporate indicators WFP uses to track progress toward outcome 1.1 (reduced or stabilised acute malnutrition in target groups of children and/or populations).

* In some instances, a single project may report multiple disaggregated values for an indicator.

Data indicate that the first indicator on prevalence of low mid-upper arm circumference had a reporting rate of 57% in 2012 (i.e. only 8 out of the 14 projects that included the indicator in the logical framework actually reported on it). For each of the 8 projects which did report on the indicator, a single project value was shared. Moving across the row, we see that only 2 projects had the necessary comparable values to track progress (i.e. the value reported for 2012 could be compared to a baseline or previously collected monitoring value). Upon analysis by WFP, both projects were found to have demonstrated improvement on the indicator, leading WFP to report that a 100% improvement rate had been attained. However, as previously mentioned, there are little grounds to assume that results obtained in the 2 projects are representative of those which would have been observed in the 8, and less still to assume that they would have been representative of those observed from the full 14 projects meant to report on the indicator. In contrast, we highlight the supplementary feeding recovery rate indicator for which both reporting and the number of comparable values was high; in this situation, the improvement rate obtained is more likely to be representative of the existing trend, as the evidence base is high.

Still in relation to the above methodological issue, we observed the wording of certain outcome indicator results in the narrative of the Annual Performance Report for 2012 to be misleading. For instance, on page 43, which presents progress made toward strategic objective 2, WFP states that “[t]here were improvements in 70 percent of projects that reported on the household food consumption score”. However, closer inspection of the data in annex II-B demonstrates that only 33% of projects which reported values showed improvement; the statistic WFP is presenting is mislabelling the number of projects that provided comparable data (a smaller subset) as those which reported on the outcome indicator. From a methodological standpoint, this distinction is important.

See also Volume II, Appendix X for details on the methodology used to assess KPI A.

Evidence of the extent of progress towards WFP’s strategic objectives

Finding 24: Donors at headquarters perceived WFP to be making adequate progress towards its organisation-wide strategic objectives and its performance related to saving lives and protecting livelihoods was recognised as very strong. In contrast, the review of WFP’s reports and evaluations led to mixed conclusions due to the lack of consistent and robust outcome-level data.

Figure 4.4 presents the results of the survey of donors at headquarters on progress towards WFP’s five strategic objectives (44 donors responded). Respondent trends are further described in sections below, in parallel with the observations from the document review.

Figure 4.4 | Stakeholder survey of WFP progress towards strategic objectives - mean scores

Evaluations reviewed consistently mentioned the lack of documented, robust monitoring data as an important limitation for drawing conclusions on WFP’s performance in contributing to outcome-level change. Generally, they reported evidence of output and/or short-term outcome change, highlighting key achievements during emergencies, but pointed to challenges in assessing WFP’s performance in achieving medium to longer term results.

Similarly, the MOPAN analysis of WFP’s self-reported outcome-level data highlighted gaps in country-level monitoring and evaluation and, consequently, challenges in aggregating data at the corporate level. Given the limited evidence base for many outcome indicators, it is difficult to conclude with a high degree of confidence that strong progress is being made towards WFP’s strategic objectives.

Assessment of WFP’s strategic objectives

The following sections present the survey ratings and document review assessment for each strategic objective. Data on outputs from annual performance reports (APRs) for each of WFP’s five organisation- wide strategic objectives from the 2008-2013 strategic plan are shown in Volume II, Appendix IX.

The charts below (Figures 4.5 to 4.9) present data from annex II-B of the APR for 2012 that is recalculated and assessed in light of some of the observations in the sections above on measurement and reporting. In particular, the figures on outcomes include:

• A variable on the usability of data reported by country offices: As a criterion to help inform the analysis of evidence of results, the assessment team considered that outcome indicators for which less than 50% of the reported data could be used were less trustworthy as a source of information from which to draw conclusions on outcome-level progress (i.e. signs of improvement are less meaningful or representative if the majority of the data is thrown out).

• Variables that contrast the WFP analysis and the MOPAN assessment team’s analysis of the percentage of projects showing progress: The MOPAN analysis provides an estimate of the proportion of projects that reported on an outcome indicator which could demonstrate evidence of progress. It serves to illustrate existing limitations in the pool of data from which conclusions on organisation-wide results are drawn. A more severe assessment would consider not only the number of projects that reported values on an indicator, but the number of projects that were supposed to report based on commitments made in their logical framework.

Further information on the calculation of the variables is presented in Volume II, Appendix X.

MI A1 - Save lives and protect livelihoods

Over the 2008 to 2012 period, WFP invested 64% of its resources in strategic objective 1. In fact, this objective has consistently been the most highly financed of WFP’s current strategic plan. Likewise, the tonnage of food distributed has been highest for this objective, which is not surprising considering that WFP’s food distributions to save lives typically seek to meet all beneficiary food needs, unlike activities under other strategic objectives, which aim to ‘fill the food gap’. Since 2009, strategic objective 1 has also by far exceeded the number of beneficiaries assisted by comparison to the four other corporate programming objectives. (See outputs in Volume II, Appendix IX.)

In the survey, donors at headquarters rated WFP very strong for its progress towards this objective. In fact, the score for this MI was the highest in the MOPAN survey (across both the organisational effectiveness and results components).

Figure 4.5 presents evidence on expected outcomes for WFP strategic objective1. Calculations are based on self-reported outcome indicator data from WFP’s Annual Performance Report for 2012 and WFP’s own analysis of projects showing progress (see right-hand column).

Figure 4.5 | Evidence of extent of progress towards WFP’s expected outcomes related to strategic objective 1 for 2012*

Data considered less robust for analysing progress on the outcome indicator (percentage usable < 50%; see third column)

* The information in this table is based on source data from annex II-B of the APR for 2012 (i.e. issued in 2013). See Volume II, Appendix X for information on the calculation of the variables in this figure.
** The number of values reported that have comparable data to track progress (i.e. baselines or previously recorded values) over the total number of values reported for the indicator.
*** The number of values for which there was evidence of improvement (by comparison to baselines or previous monitoring data) over the total number of values reported for the indicator.
**** The proportion of projects that demonstrated improvement over those that reported values that can be assessed.

The available performance data for strategic objective 1 provides evidence that WFP made progress in 2012 in reducing or stabilising acute malnutrition (outcome 1.1), and in improving food consumption (outcome 1.2). Of note is that a very high proportion of projects reporting on the supplementary feeding indicators included usable data to track progress (i.e. thus increasing the level of confidence in the conclusions based on these indicators). At minimum, 62% of projects recorded progress in recovery rates of targeted beneficiaries (indicator 1.1.3 i), and 59% were able to reduce the proportion of beneficiary deaths (indicator 1.1.3 ii).

Although gaps in data make it difficult to ascertain the overall progress made by WFP in stabilising the enrolment of girls and boys at high risk of dropping out from primary schools in emergency settings (outcome 1.3), all three projects which had the necessary data for analysis demonstrated improvement.

While the outcome indicator data for 2012 suggest that progress is being made towards strategic objective 1, the absence of baseline values for a significant proportion of projects limit the conclusions which can be drawn from the self-reported data. Nevertheless, information presented in WFP evaluations since 2008 overwhelmingly indicates that emergency interventions have been effective in saving lives. For instance, achievement of results was noted in all three country performance evaluations completed in 2010, in the four completed in 2011, and in the four conducted in 2012. WFP’s geographic reach, logistics capacity, and ability to mobilise resources quickly were highlighted as strengths in these independent reviews. However, irregular or reduced funding, pipeline breaks, access difficulties, constrained operational space, and leakages were also identified as challenges requiring attention from the organisation. Moreover, all reports emphasised that outcome-level monitoring needed improvement and that the absence of such data limited rigorous assessment (evaluations noted a heavy reliance on anecdotal, qualitative evidence).

MI A2 - Prevent acute hunger and invest in disaster preparedness and mitigation measures

The proportion of resources allocated to strategic objective 2 annually has fallen from 35% in 2008 to 5% in 2012. The tonnage of food distributed and the number of beneficiaries assisted has followed the same downward trend. In fact, this objective is now second to last in terms of financing. (See outputs in Volume II, Appendix IX.)

Although the majority of donors at headquarters (59%) rated WFP strong or very strong for its progress in preventing acute hunger and investing in disaster preparedness and mitigation measures, the overall rating for this MI was adequate.

As shown in Figure 4.6 less than half of projects that reported on indicators for corporate outcomes 2.1 and 2.2 provided sufficient information to track progress during 2012. While at least 50% of values reported for outcome 2.3 on reducing the risk of hazard at the community level were usable, none of the projects that reported on the household asset score demonstrated any improvement (indicator 2.3.1). Nevertheless, some progress was observed in the creation of assets at the community level (indicator 2.3.2).

Overall, the evidence base to demonstrate progress towards achievement of strategic objective 2 is scant, perhaps owing to the small financial investment currently being made in this objective. According to the 2011 strategic evaluation “WFP’s Role in Social Protection and Safety Nets”, safety net programmes to support and strengthen resiliency to shocks, including adaptation to climate change, make the most significant contributions when they are linked to government priorities and are of sufficient duration.

Over the 2008-2012 period, proportional investments in strategic objective 3 have fluctuated between 4% and 15%. Though comparatively much smaller than strategic objective 1, this objective has since 2011 ranked second in terms of number of beneficiaries, direct expenses and tonnage of food distributed. (See outputs in Volume II, Appendix IX.)

In the survey, WFP was perceived as adequate by donors at headquarters for making progress towards its objective of restoring and rebuilding lives and livelihoods in post-conflict, post-disaster or transition situations.

As shown in Figure 4.7, a fair to very high proportion of projects reported usable data on the indicators under strategic objective 3 to track improvement in 2012. The strongest evidence of progress was for outcomes 3.2 and 3.3.

Figure 4.6 | Evidence of extent of progress towards WFP’s expected outcomes related to strategic objective 2 for 2012*

Data considered less robust for analysing progress on the outcome indicator (percentage usable < 50%; see third column)

* The information in this table is based on source data from annex II-B of the APR for 2012 (i.e. issued in 2013). Please refer to Appendix X for information on the calculation of the variables in this figure.
** The number of values reported that have comparable data to track progress (i.e. baselines or previously recorded values) over the total number of values reported for the indicator.
*** The number of values for which there was evidence of improvement (by comparison to baselines or previous monitoring data) over the total number of values reported for the indicator.
**** The proportion of projects that demonstrated improvement over those that reported values that can be assessed.

The four impact evaluations conducted jointly with UNHCR in 2011 and 2012 on the contribution of food assistance to durable solutions in protracted refugee situations (in Bangladesh, Chad, Ethiopia and Rwanda) indicated that the food assistance provided had reduced the prevalence of global acute malnutrition at targeted refugee sites in each country (in comparison to control sites). However, the evaluations noted that food assistance had limited effects on food security over the medium-term and that the expected evolution towards self-reliance over the longer term did not occur; improved livelihood opportunities and asset-building could not be demonstrated. However, the evaluations indicate that the protracted relief and recovery programme category fits uneasily with the traditional humanitarian or development donor funding patterns. In fact, the Annual Evaluation Report for 2012 states that:

“The international community’s response to refugees in protracted crises is failing to deliver. To resolve the issues blocking progress, there is need for concerted action among all actors, backed by political will and financing to enable refugees to make productive contributions to the countries where they live, and to support other long-term durable solutions where possible.”

Figure 4.7 | Evidence of extent of progress towards WFP’s expected outcomes related to strategic objective 3 for 2012*

Data considered less robust for analysing progress on the outcome indicator (percentage usable < 50%; see third column)

* The information in this table is based on source data from annex II-B of the APR for 2012 (i.e. issued in 2013). Please refer to AppendixX for information on the calculation of the variables in this figure.
** The number of values reported that have comparable data to track progress (i.e. baselines or previously recorded values) over the total number of values reported for the indicator.
*** The number of values for which there was evidence of improvement (by comparison to baselines or previous monitoring data) over the total number of values reported for the indicator.
**** The proportion of projects that demonstrated improvement over those that reported values that can be assessed.

MI A4 - Reduce chronic hunger and undernutrition

As noted in section 4.2.1, strategic objective 4 primarily concerns WFP’s development operations (referred to also as country programmes). Annual expenditures for this objective dropped from a peak of 23% in 2009 to a low of 7% in 2012. There was a similar decrease in the number of beneficiaries reached and the tonnage of food distributed. (See outputs in Volume II, Appendix IX.)

Overall, WFP was perceived as adequate in reducing chronic hunger and undernutrition by survey respondents. Nearly half of MOPAN donors at headquarters (48%) rated WFP strong or very strong, 36% rated it adequate, and 14% rated it inadequate or weak.

The 2011 strategic evaluation “WFP’s Role in Ending Long-Term Hunger” indicates that there is a lingering perception among stakeholders that WFP may not have a comparative advantage for addressing long- term hunger and that it is more suited to responding to emergencies. As stated in the report, this could jeopardise WFP’s development programming, as donors do not necessarily have confidence in WFP’s ability to make long-term commitments, notably due to its funding structure. When budgets become tighter in response to external pressures, donor support tends to shift from longer term development activities to shorter term relief operations. Development activities are thus more susceptible to funding shortfalls, which hinders achievement of results.

WFP is trying to shift donor support to multi-year agreements to help prevent the pipeline breaks that have had particularly negative effects on WFP’s mother and child health nutrition (MCHN) interventions and school feeding interventions, for example. Problems in achieving results also relate to the lack of monitoring (in particular of MCHN indicators), and to a lack of clarity regarding WFP‘s strategic aims in the field of nutrition as it shifted from food aid to food assistance.[72]

In general, a high proportion of projects that reported on the outcome indicators under strategic objective 4 in 2012 included comparable values to monitor progress, perhaps reflecting the greater ease with which data can be collected in development programming. The strongest evidence of progress was for increased access to education and human capital development in assisted schools (outcome 4.2). There was also a good evidence base indicating that WFP has improved adherence to antiretroviral therapy (outcome 4.4).

Figure 4.8 | Evidence of extent of progress towards WFP’s expected outcomes related to strategic objective 4 for 2012*

Data considered less robust for analysing progress on the outcome indicator (percentage usable < 50%; see third column)

* The information in this table is based on source data from annex II-B of the APR for 2012 (i.e. issued in 2013). Please refer to Appendix X for information on the calculation of the variables in this figure.
** The number of values reported that have comparable data to track progress (i.e. baselines or previously recorded values) over the total number of values reported for the indicator.
*** The number of values for which there was evidence of improvement (by comparison to baselines or previous monitoring data) over the total number of values reported for the indicator.
**** The proportion of projects that demonstrated improvement over those that reported values that can be assessed.

MI A5 – Strengthen the capacities of countries to reduce hunger, including through hand-over strategies and local purchase

Annual allocations to strategic objective 5 have been low throughout the duration of WFP’s current strategic plan, ranging from less than 1% to 3%. In fact, the number of beneficiaries reached annually has never even registered as a full percentage point within the portfolio of WFP’s work throughout the 2008-2012 period. (See outputs in Volume II, Appendix IX.)

The majority of MOPAN donors at headquarters (70%) rated WFP as adequate or above for progress towards this strategic objective, and 30% rated it as inadequate or below.

Nearly half of projects reporting usable values for outcome 5.1 on marketing opportunities created through local purchase registered progress (see Figure 4.9). This value is meaningful and the conclusions drawn from it can be considered with greater confidence as 100% of projects meant to report on the indicator did, and a very high proportion generated usable data. However, less than one-third of the data reported on outcome 5.2 was usable. This is not surprising however as the indicator was introduced in late 2011, giving country offices little time to both establish a baseline value and report on results.

Figure 4.9 | Evidence of extent of progress towards WFP’s expected outcomes related to strategic objective 5 for 2012*

Data considered less robust for analysing progress on the outcome indicator (percentage usable < 50%; see third column)

* The information in this table is based on source data from annex II-B of the APR for 2012 (i.e. issued in 2013). Please refer to Appendix X for information on the calculation of the variables in this figure.
** The number of values reported that have comparable data to track progress (i.e. baselines or previously recorded values) over the total number of values reported for the indicator.
*** The number of values for which there was evidence of improvement (by comparison to baselines or previous monitoring data) over the total number of values reported for the indicator.
**** The proportion of projects that demonstrated improvement over those that reported values that can be assessed.

Country portfolio evaluations over the last three years have indicated that a greater focus must be placed on capacity development, ownership and sustainability in WFP’s planning and operations. The 2011 strategic evaluation “How WFP Country Offices Adapt to Change” indicated that WFP’s shift in focus from a food aid to a food assistance agency was perhaps the biggest change in its history and that it signified transforming the organisation from primarily using a task-oriented and logistics approach to one focusing on national capacity building also. In order to fully transform and enact this change, the organisation will need to further expand its own technical capacity (for instance in nutrition).

4.3 EVIDENCE OF WFP’S CONTRIBUTION TO COUNTRY-LEVEL GOALS AND PRIORITIES

4.3.1 Overview

This section presents the results of the assessment of evidence of WFP’s contributions to country-level results. By separating the KPI at the organisation-wide level from KPIs at the country level, MOPAN recognises the demand-driven nature of many of the activities of a multilateral organisation and the key role that is played by its country programming or strategy documents, where expected results at the highest level (outcomes and impact) reflect a shared responsibility between the multilateral organisation and the partner country.

Section 4.3.2 examines evidence of the organisation’s contribution to country-level goals and priorities. While the 2013 MOPAN assessments were designed to assess contributions to relevant millennium development goals (MDGs), this aspect was not included in the WFP assessment as the organisation does not report on MDGs in its country-level reporting.

(Note: Section 4.4 examines relevance in terms of the extent to which partners and donors in-country believe the organisation supports country priorities and meets changing needs.)

WFP’s results and reporting at the country level[73]

WFP operationalises its humanitarian and development mandate at the country level by delivering food assistance through emergency operations (EMOPs), protracted relief and recovery operations (PRROs) and development-focused country programmes (CPs). EMOPs provide immediate assistance, usually lasting between 3 and 12 months and covering sudden disasters, slow-onset disasters, and complex emergencies. PRROs help sustain disaster-hit communities as they re-establish livelihoods and stabilise food security. Such operations are meant to be conceived six months prior to the end of an EMOP when it becomes apparent that two years of emergency assistance will be insufficient; PRROs can last at most three years. CPs seek to improve food security for communities by concentrating aid on pre-identified, food-insecure areas inside recipient countries (e.g. rural areas of low productivity, areas prone to natural disasters, areas vulnerable to periodic food shortages, or areas with high concentrations of malnutrition). CPs are generally aligned with the United Nations Development Framework (UNDAF) planning and reporting cycle at the country level.

As per the WFP classification, humanitarian interventions are considered to encompass both EMOPs and PRROs, while the CPs are the focus of development operations. Although there is only one country programme active in a country at any one time, multiple humanitarian operations can take place within the country.

WFP communicates its expected results at the outset of an operation through Board-approved project documents. These include a logical framework detailing outputs and outcomes that link to corporate strategic objectives. Each operation (EMOP, PRRO or CP) is required to produce a standard project report (SPR) on an annual basis, which includes self-reported data by WFP at both output and outcome levels, allowing the organisation to keep track of progress in achieving results. Though SPRs are not made public, they are shared with donors.

Data used for this assessment

In 2013, the country-level assessment was based on data from the five countries surveyed by MOPAN (Ethiopia, Guatemala, Indonesia, Mozambique, and Pakistan).

All results-related information provided in documents for operations in the five focus countries was reviewed. These included WFP’s Board-approved project documents, budget revisions, revised logframes, resource situation summaries, standard project reports, and any internal or external reviews or evaluations pertaining to the assessed operations. Attention was paid to the following elements: quality of the results statements; the relevance of indicators, baselines and targets; the strength of the link between results statements and results achieved; the quality of evidence presented to substantiate the results achieved, including an assessment of contribution; and the overall performance story.

The survey, which was addressed only to respondents in-country, asked questions based on the operations assessed in each of the five countries. Interviews with senior WFP country office staff also informed the analysis of context and ensured that the assessment team had a full set of documentation with which to conduct the document review and analyse results.

4.3.2 Evidence of contribution to country-level goals and priorities

This section provides an overall rating for KPI B based on documentation made available by the organisation and survey data. It also includes an assessment of the quality of measurement and reporting of results at the country level as well as an assessment of the extent of WFP’s progress towards results in the five sampled countries.

WFP generally operates in challenging and often volatile environments (e.g. conflict or war-torn regions and countries experiencing recurrent climatic shocks) where the delivery of food or non-food items is not simple and where needs may fluctuate. For instance, it is not uncommon for the number of beneficiaries within a targeted zone to increase midway through an operation, which can seemingly reduce evidence of earlier progress. Many other factors add to the complexity of WFP’s work in delivering products and services and measuring results: upsurge of violence limiting access to sites, disease outbreak such as cholera, beneficiaries’ inability to travel to distribution sites (e.g. inability to attend school), short duration of WFP operations, absence of baseline data in emergencies, limited capacity of implementing partners to monitor results, etc. A key challenge for WFP in capturing outcomes as well as outputs is that food is but one of many factors that affect food security and nutritional status.

Country offices often have limited resources and struggle to balance the humanitarian imperative to save the most lives possible with the need to dedicate resources to monitoring and evaluation to prove the effectiveness of its programming. Nevertheless, in order to demonstrate results achievement and attract continued funding, a strong monitoring and evaluation function at the country level is necessary. Though the sample of reviewed countries was very small, there is evidence that efforts are being made at the country level to better prioritise monitoring and evaluation.

Overall assessment and rating

Overall, across the five sampled countries, WFP’s performance in delivering its intended results was rated adequate by survey respondents. The document review found clear evidence of the achievement of expected outputs and some evidence of contribution to outcome achievement in these five countries.

Figure 4.10 shows the overall rating for KPI B based on the review of WFP’s self-reported data for operations and evaluations, and on perceptions of surveyed stakeholders.

Figure 4.10 | KPI B: Evidence of the extent of contribution to country level goals and priorities, overall rating

Quality of WFP’s measurement and reporting on country-level goals and priorities (for all countries sampled)

Finding 25: Despite strong evidence of the achievement of expected outputs, WFP’s reporting practices do not currently provide a sufficient evidence base to fully assess the organisation’s contributions to country-level outcomes. The document review noted that monitoring and evaluation is insufficiently resourced at the country level but that its importance is being increasingly recognised.

WFP’s recent reporting in the five assessed countries does not provide sufficient information to fully ascertain WFP’s performance. In line with the findings for KPI A, gaps in data for outcome indicators were reported in external/independent reviews. While improvements in monitoring are required, there is evidence of promising change underway at the country level. In addition to corporate-led measures (such as the roll-out of the COMET system), information from staff interviews suggest growing recognition and efforts to improve the monitoring and evaluation (M&E) function over the last few years. For example, the Ethiopia country office developed an M&E strategy and provided staff training to address challenges they had noted regarding M&E, especially in terms of capacity development. The Pakistan country office designed WFP Operation Works, known as WOW, a system that allows it to bridge the gap between corporate systems and work on a single platform to ensure consistency in data. The system also allows it to report on outputs and outcomes in real-time and to generate data for corporate and donor requirements, with possible aggregation and disaggregation of data from the final distribution point to the country office level for any specified time interval or partner. As noted by the Pakistan CO: “[s]tatistical analysis of the indicators helps to pinpoint a poor performer and enables corrective actions”.

Figure 4.11 presents a summary assessment of the quality of WFP’s systems and practices for measuring and reporting on its country-level results. The headings show the criteria established by MOPAN for this analysis.

Figure 4.11 | WFP’s measurement and reporting on country level results

1. ‘Theory of change’ is understood in the sense defined by Rist and Morra Imas (2009) as, “a representation of how an intervention is expected to lead to desired results” and in the sense defined by Michael Quinn Patton who has stated that a theory of change is more than the sequential order of results statements presented in a logic model; it requires key assumptions related to the results chain and context (e.g. policy and environment), and important influences and risks to be made explicit - Qualitative Research and Evaluation Methods (2002).
2. This refers to the existence of reports on outputs as defined by the OECD (i.e. lower level results). Some MOs use different terminology for the various levels of results.
3. This refers to the existence of reports on outcomes as defined by the OECD (i.e. higher level results). Some MOs use different terminology for the various levels of results.
4. Reporting according to a theory of change is understood to mean the extent to which organisations provide a narrative describing the actual implementation process and results achieved in relation to that foreseen in the initial ‘theory of change’.
5. According to the 2009 Rist and Morra Imas publication on The Road to Results, reliability refers to the stability of measurements, i.e. “the degree to which it measures the same thing, in the same way, in repeated tests”.

Strategic planning and theory(ies) of change

WFP’s strategic planning for each operation is formalised in project documents that are approved by WFP’s Executive Board. These documents include a situation analysis describing the general country context and providing specific data regarding food insecurity and nutrition challenges in the country. In addition, they specify the rationale for the proposed project as well as targeted beneficiaries (with disaggregation for women and children), and present a narrative on the project objectives as well as a results/logical framework detailing expected results.

In line with the observations noted under KPI A, the documents reviewed did not provide complete theories of change, but did include some elements that were presented more cohesively than at the global level. For instance, all logical frameworks reviewed included a column for risks and assumptions in addition to a results chain of outputs and outcomes. For the most part, the assumptions were sufficiently detailed and well connected to the corresponding results statements. However, given that inputs are not presented in the logframes, but rather in separate tables in the narrative sections of the report, assumptions related to the availability of inputs were often presented alongside outputs. The assumptions/risks generally covered issues related to the enabling environment and not those related to the project logic (i.e. assumptions which if not met would invalidate the link between outputs and outcomes).

Measuring, reporting on, and communicating results

The section on measuring results for KPI A highlighted challenges and areas for improvement relevant to country level operations. Of particular importance is the issue of funding shortfalls, and monitoring and evaluation.

In determining its programming, WFP identifies beneficiaries in need of food assistance, the budget required to provide this assistance, and the results expected from its provision. However, the requested resource levels are rarely obtained. Hence, WFP country offices set results targets that are likely overambitious and difficult to attain. Defining more realistic budget scenarios would improve WFP’s accountability for results. As it currently stands, performance problems may be masked by too great a focus on funding constraints.

Standard project reports (SPR) reviewed generally provided robust monitoring data for expected outputs. For the most part, data presented for outcome indicators also appeared robust, though some indicators at the outcome level are inherently more difficult to measure accurately (e.g. national capacity index and community asset score). Owing to challenges in data collection, there were instances where only a single value was measured over the course of an operation, or where changes in methodology rendered data non-comparable. There were also many outcome indicators included in most operations’ results frameworks (even revised versions) for which no data was presented. Several evaluations have noted that the absence of sufficient outcome level data impedes a thorough assessment of the effectiveness of WFP’s programming, especially for longer term results such as capacity development. For some countries sampled there was little evaluative evidence available.

The document review found that little outcome indicator data was reported for capacity development programming. Country offices indicated during interviews that the corporate indicators for this topic require further development: in particular, process indicators to monitor capacity development, so that achievements can be demonstrated well within SPRs.

In addition, results could be better communicated in SPRs. Although these reports are shared with donors for transparency and accountability purposes, one cannot easily determine from the indicator data presented what results are being achieved as the results statements in the logical framework are presented in a separate document.

Evidence of the extent of progress towards WFP results at the country level

Finding 26: There is evidence across the sample of five countries that WFP is achieving its planned results at the output level and partial progress towards expected outcomes.

Surveyed stakeholders at the country level consistently rated WFP’s delivery of humanitarian or development results as adequate or strong, and regarded the improvement of food consumption in emergencies as WFP’s highest performing area. The document review found clear evidence of achievement of outputs and some evidence of contribution to outcomes. Nevertheless, these conclusions are to be interpreted with caution, as the sample of WFP countries assessed is small (only 5 of the 80 countries in which WFP provides food assistance).

All operations assessed across the five countries were under-resourced by comparison to budgeted needs (the funding gap ranged from 14.7% to 65% across operations). Generally, under-resourced operations were least able to demonstrate achievement of expected results, but some relatively well-resourced operations underperformed.

Methodology used to assess the extent of progress at the country level

In the following sections on each of the five countries sampled, the assessment examined both WFP’s humanitarian and development results. As noted above and in accordance with WFP’s own classification, emergency operations (EMOPs) and protracted relief and recovery operations (PRROs) are considered to contribute to WFP’s humanitarian results, while country programmes (CPs) are viewed as leading to development results. The operations selected for inclusion in this assessment had to meet two criteria: be active over the 2010-2011 period to ensure they reflect WFP’s recent practices, and that medium- or long- term operations such as PRROs or CPs have been underway for at least two years in order to have sufficient results data from which to glean progress.

The MOPAN assessment examined humanitarian operations in all sampled countries. In countries where there was a single humanitarian operation, its expected outcomes were used as the basis for the MOPAN micro-indicators. In countries where there were multiple operations, similar outcomes were regrouped, and the wording of the MOPAN micro-indicator was adjusted accordingly. The outcome results statements for country-level operations closely mirror those in the WFP corporate framework (a process imposed to facilitate aggregation of results at the corporate level). Hence, the wording used for grouped outcomes in the MOPAN assessment was also aligned with WFP’s corporate framework. The micro-indicators were assessed through document review to determine evidence of achievement of targets for expected outputs and outcomes. The indicators also served as the basis for the survey questions. When the indicator (or WFP outcome statement) included multiple ideas, it was transformed into multiple survey questions.

In order to limit the survey questions to a reasonable number, the MOPAN micro-indicators prepared to assess WFP’s development results in each country focused on high level programming areas from the relevant country programme. The decision to put a greater focus on WFP’s humanitarian results in the survey was based on the rationale that a much larger proportion of WFP’s work is humanitarian in nature than development-focused.[74] Nevertheless, evidence of both outputs and outcomes of development programming were assessed in the document review, following the same practice as for the humanitarian interventions.

The selection of operations to be assessed and the micro-indicators to be used were validated by country offices. On the basis of dialogue with MOPAN and the organisation assessed, some micro-indicators were modified, and some operations were added even though they did not fully meet the criteria.

The sections below provide an overall assessment of results achieved in each country sampled for this assessment. See Volume II, Appendix XI for project-specific examples of the types of results achieved. The following analyses should be regarded as indicative rather than conclusive as there were considerable variations between the data reported in WFP’s standard project reports (SPR) and the results initially planned in logframes (particularly at the outcome level), as well as significant differences in the availability of evaluations across countries.

ETHIOPIA

Ethiopia’s economy is fast-growing, with the World Bank reporting an average annual growth rate of its gross domestic product of 9% over the 2001-2011 period. Yet at the same time, 39% of its population lives below the international poverty line and the country was ranked 173 out of 187 countries on the 2012 UNDP Human Development Index. Despite improvements in chronic malnutrition in the last 11 years, the 2011 Ethiopia Demographic and Health Survey reports that 44% of children are stunted, 10% suffer from acute malnutrition, and 29% are underweight. Climate change (especially changes in seasonal precipitation), land degradation, population density, economic shocks, droughts, floods, disease and conflict remain important drivers of food insecurity in the country.

The assessment of WFP’s humanitarian results in Ethiopia is based on two protracted relief and recovery operations: PRRO 106650 covered the period from 2008 to 2011 and focused on acute malnutrition, while PRRO 101273 was implemented from 2009 to 2011 and provided food assistance to refugees. On the development side, WFP’s country programme for the 2007-2011 period (CP104300) involved community- based, food-supported activities to build resilience in fragile livelihood settings and improving the quality of and access to primary education.[75] See Volume II, Appendix XI for detailed assessment.

In the survey, WFP was considered adequate overall in contributing to its expected humanitarian outcomes. The majority of respondents (64%) perceived WFP strong or very strong in improving the food consumption of households in emergencies. WFP was rated adequate on its effectiveness in achieving results in all areas of its development programme. The proportion of ‘don’t know’ responses was high (ranging from 22 to 37%) for all three development areas and for three humanitarian outcomes (B4, B5, and B6).

Based on the documentary evidence reviewed, WFP was found to perform well overall in delivering its intended outputs in Ethiopia, though annually planned targets were generally not attained in the sampled humanitarian interventions. Funding shortfalls and the 2011 drought in the Horn of Africa hindered WFP’s achievement of planned results at both the output and outcome levels. The drought affected food security in Ethiopia and also led to an influx of refugees from neighbouring countries. Nevertheless, a 2011 impact evaluation of WFP’s contribution to durable solutions among refugee populations indicated that “ensuring adequate food energy consumption has in large part been achieved, and food energy consumption has improved in recent years”.[76] The evaluation emphasised, however, that results achieved have been those with short-term effects, and that the organisation has not been as successful in improving livelihood opportunities and asset-building.

Interestingly, surveyed respondents rated WFP highly on the two questions asked for micro-indicator B8 (contributions to incorporating solutions for hunger into broader national policy frameworks; and contributions to increasing government capacity – particularly at local levels). While the documents reviewed support the reported achievement of outputs, there is no data on outcomes. Hence, it is not possible to corroborate the survey finding.

Another divergence between the survey results and documentary evidence was noted in WFP’s development programming. Two of the three MIs related to WFP’s development work in Ethiopia (B9 and B11) received the lowest survey ratings in the assessment for this country. However, some of the strongest documented evidence of progress is available precisely for WFP Ethiopia’s country programme. In particular, the MERET impact evaluation (2012) noted that households participating in the MERET programme, which seeks to improve sustainable land management in chronically food-insecure woredas, more commonly demonstrated increased incomes in the past three years than those in communities serving as controls (58% versus 35% respectively).Moreover, participating households tended to own higher value-assets, which as the evaluation notes “suggests that MERET sites have a richer reserve of assets to draw upon to support their resilience strategies”. Additionally, the annual rate of growth in enrolment is on average higher in WFP-assisted schools than in the rest of the country, while drop-out rates are lower. Nevertheless, WFP’s country programme faces some of the same challenges as its humanitarian operations in Ethiopia. Although the 2009 mid-term evaluation of WFP’s 2007-2011 development programme found it to be highly relevant as it focused on issues of major significance and was well-targeted, it noted that the food disbursed represented approximately half of the planned values and that hundreds of watersheds in the most food insecure regions of the country had to be removed from the programme.[77]

GUATEMALA

Although Guatemala is a middle-income country, high levels of income inequality lead to high levels of poverty (53%) and extreme poverty (13%). Chronic undernutrition affects half of the population; this is due primarily to inadequate dietary intake and disease, but is compounded by the lack of access to health services, safe water and sanitation. According to the 2011 Geneva Declaration on Armed Violence and Development, Guatemala is also one of the 14 most violent countries in the world. It is also highly vulnerable to climate change, being prone to natural disasters: In 2011 and 2012, Guatemala was hit by Tropical Depression 12-E, a prolonged drought, and an earthquake of magnitude 7.2, all of which caused substantial damage and losses (USD 285 million) and worsened food security in poor households. Indigenous peoples (particularly women, girls and boys) are most vulnerable to food insecurity in the country.

MOPAN’s assessment of results achieved by WFP in Guatemala focused primarily on EMOP 200111 on the humanitarian side and on CP 200031 for WFP’s development programming. EMOP 200111 lasted from 2010 to 2011 and focused on severely and moderately food insecure families affected by emergencies. WFP’s country programme began in 2010 and is scheduled to end in 2014. It focuses on reducing chronic undernutrition, improving the livelihood of subsistence farmers, supporting small farmers’ access to markets, and strengthening government capacity to conduct effective food-based interventions. At the request of the WFP country office in Guatemala, PRRO 200043, a regional relief operation, was also included in the document review, although it was not included in the survey. See Volume II, Appendix XI for detailed assessment.

WFP’s humanitarian results were assessed through a single MI in the survey; almost half of respondents (46%) rated WFP strong or very strong for improving food consumption of vulnerable groups affected by emergencies. On the development side, WFP’s programming was consistently rated adequate.

Documentary evidence indicates that despite efforts to mobilise resources in Guatemala, financial contributions were not reached as planned. WFP’s final Resource Situation Summary (January 2012) for emergency operation 200111 confirmed that the amount received (almost USD 10 million) covered only 35% of requirements. Funding constraints also affected the country programme and led WFP to reduce the number of targeted beneficiaries. It is therefore not surprising that expected outputs were considered to be only partially met in the document review, and likely affected the contribution to outcomes also.

Within its development programme, WFP made notable progress and achieved annual targets for MI B4 related to the Purchase for Progress (P4P) initiative – which was by far the most well-resourced programming area (in 2012, 84% of contributions received were allocated to this programme as they were earmarked). As noted in the strategic evaluation of the P4P initiative,[78] it has helped farmers in Guatemala increase their yield by 97% from 2009 to 2011. Participating small farmers improved the quantity and quality of their maize and bean production, reduced post-harvest losses, and sold surpluses. Furthermore, 16 farmers’ organisations have become WFP suppliers of maize and beans in the last two years, which is remarkable progress considering that no farmer organisation had previously participated in a WFP tendering process. The percentage of local purchases showed a growing trend, demonstrating the improved capacity of farmer organisations in increasing production and quality with reduced post-harvest losses. As reported by farmer organisations, sales of maize and beans at fair prices have increased not only with WFP but with other buyers, generating income for improved living conditions.

The regional humanitarian operation was assessed by document review only, which found that Guatemala SPRs demonstrated achievement of expected outcome level results. During the interview with the Guatemala country office, it was noted that this operation is better resourced.

The Guatemala country office also indicated that documenting outcomes in the medium and long term is complex, and that the external environment in which WFP operates is more prone to change and sometimes requires WFP to adapt its activities and/or tools. At times, this affects the consistency in data. The office reports that in many countries, including Guatemala, concrete steps are being taken to develop a set of programme monitoring and evaluation tools for projects with a medium/long term impact.

INDONESIA

Since the 1997 Asian financial crisis and the country’s political transformation in 1998, Indonesia has made significant progress and has become an emerging low middle income country. It is a member of the G-20 and, as of 2012, is a first time donor to WFP. Nevertheless, the country continues to face disparities among regions in terms of human development and resources, which are amplified by low technical and administrative capacity at decentralised levels. As noted in the 2009 Food Security and Vulnerability Atlas, 87 million Indonesians are vulnerable to food insecurity, and 36.5% of children under 5 experience stunting, a figure which increases to 58% in some eastern provinces. Additionally, as an archipelago of more than 17000 islands situated on the Pacific ring of fire where three tectonic plates meet, Indonesia is vulnerable to natural disasters and to the adverse effects of climate change.

The assessment of WFP’s humanitarian results in Indonesia is based on PRRO 10069.2, which addressed micronutrient deficiencies and sought to improve the nutritional and health status of vulnerable groups in Eastern Indonesia over the 2008-2011 period. Development results were assessed based on the recently initiated 2012-2015 CP 200245, which reflects the recent shift in WFP’s role from direct operational engagement to capacity building and empowerment of government and communities to address hunger and nutrition challenges.[79] See Volume II, Appendix XI for detailed assessment.

Surveyed stakeholders considered WFP’s contributions to humanitarian outcomes in Indonesia adequate overall. Noteworthy was WFP’s performance in improving food consumption for households in emergencies, which was rated strong or very strong by nearly half (48%) of respondents. For the outcome on increasing communities’ access to assets (MI B3), however, respondent views were mixed: 48% rated WFP adequate or above, 30% rated it inadequate, and 21% responded ‘don’t know’. Similarly, on the development side, two of the three micro-indicators led to divergent viewpoints. MI B9 on strengthening Indonesia’s national capacity in disaster preparedness and response was rated more positively by MOPAN donors at headquarters, direct partners, peer organisations and recipient governments than donors in- country and this difference was statistically significant. In addition, for MI B10 on strengthening national capacity to reduce undernutrition below critical levels, responses were split: 51% rated WFP adequate or above, 24% rated it inadequate, and 24% answered ‘don’t know’.

The review of documents found that the humanitarian protracted relief and recovery operation (PRRO) in Indonesia experienced difficulty in mobilising the necessary funds for implementation: confirmed contributions reached USD 46 million, which represents only 41% of the operation’s estimated needs; the approved budget was USD 112 million. Due to funding shortfalls, many planned outputs were only partially executed, especially in 2010 and 2011. The funding shortfalls may explain why multiple humanitarian outcomes were not achieved. In fact, the Corrigendum to the project document for the PRRO indicated that WFP was expected to phase out all programming in Indonesia by the end of 2011. However, this decision was revised during implementation of the project as a result of consultations with government and other stakeholders. It was decided that WFP would pursue its work in the country, though primarily focusing on development rather than humanitarian assistance.

The on-going country programme, which covers the 2012-2015 period, is also encountering some difficulties in mobilising funds. This, and the fact that the programme is still in its first year of implem- entation, provides some explanation as to why outcomes are not yet fully achieved.

MOZAMBIQUE

Mozambique has enjoyed strong economic growth in recent years but remains one of the poorest in the world, ranking 185th out of 187 countries on the 2012 UNDP Human Development Index. According to the World Bank’s World Development Indicators (2012), 60% of its population lives below the international poverty line. In addition, 34% of the population is chronically food insecure and 44% of children are stunted. More than 60% of the population lives in coastal zones that are highly vulnerable to rapid onset climatic disasters such cyclones, storms and flash floods.

The assessment of WFP’s humanitarian results in Mozambique focused on PRRO 10600.0, which supported food insecure and nutritionally vulnerable people from 2008 to 2012. Over approximately the same timeframe (2007-2012), CP 10446.0 aimed to develop human capital through school feeding; this operation is the focus of the assessment of WFP’s development results. See Volume II, Appendix XI for detailed assessment.

WFP’s performance in contributing to humanitarian outcomes was deemed adequate overall by survey respondents. WFP was recognised in particular for its contributions to improving the food consumption of households affected by emergencies; it was rated strong or very strong by more than half of respondents. Although WFP’s performance in supporting national agricultural sectors by purchasing food locally was also considered strong, respondent views were mixed on this MI: 51% rated WFP adequate or above, 23% rated it inadequate or weak, and 26% responded ‘don’t know’. WFP’s two development programme areas, rated adequate overall, also generated a high proportion of ‘don’t know’ answers (33% for MI B7 and 26% for MI B8).

The document review found evidence that progress has been made at the output and outcome levels in Mozambique for the protracted relief and recovery operation and country programme assessed. This observation is in line with the survey data. However, only for one MI was WFP’s self-reported data sufficient to conclude achievement of the outcome target. As in other countries sampled, funding constraints in Mozambique impeded the full delivery of outputs, which has had repercussions on the extent of change that can be observed at the outcome level. Funding shortfalls were 30% for the assessed humanitarian operation and 35% for the development programme.

The 2012 “Productive Safety-Net Evaluation Report” highlighted positive results regarding use of cash transfers (MI B4) in the protracted relief and recovery operation. Although there were initial problems concerning timeliness of the transfers and fine-tuning required for the asset creation component, the evaluation found that the cash transfer approach was highly appreciated by all involved stakeholders (i.e. participants, beneficiaries, local authorities and partner organisations). The vast majority of participants indicated preferring to receive cash over food and reported that the money had helped them to improve their livelihood. The money was used not only to purchase food, but also to pay for non-food items such as school fees/materials, clothes and soap, medical items and doctor fees.

The school feeding component of WFP’s country programme was also considered to be leading to positive results for beneficiaries. The 2009 “Summary Evaluation Report for Mozambique Country Programme 10446.0” noted that school feeding, which accounted for 70% of WFP’s country programme from 2007 to 2009, was an effective way of promoting retention or reducing drop-out and that it is “likely that school feeding has a long-term positive educational impact on individual beneficiaries and whole communities, through increased enrolment and retention rates among targeted schoolchildren”. Interviews with parents, teachers and community members suggested that the programme was helping get children from marginalised groups, as well as orphans and other vulnerable children, into school. However, the summary evaluation estimated the cost of on-site meals as approximately USD 48 per child per year, the cost of food provided to boarding schools as USD 190, and the cost of take-home rations as USD130. Given these elevated costs, the evaluation concluded that the intervention was expensive and therefore next to impossible for the government to replicate sustainably. In spite of revisions to the country programme to improve government ownership of operations, the evaluation found that school feeding activities were still being implemented outside strategic and programmatic frameworks; it was therefore doubtful that the intended ownership was truly being built.

PAKISTAN

Pakistan, a lower middle income country, is characterised by a fragile security situation. Escalating sectarian violence as well as law and order operations continue to have important social and financial ramifications for the country, and this is compounded by unrelenting inflation in commodity markets, chronic energy shortages, population pressures, and natural and man-made disasters. According to the 2011 National Nutrition Survey, 58% of the population is estimated to be food insecure, and approximately 44% of children under 5 are stunted.

WFP’s humanitarian results in Pakistan were assessed through four operations: EMOP 107680 covered the 2008-2010 period and focused on providing food assistance to vulnerable populations affected by the high food prices; EMOP 108280, which took effect in 2009 and lasted till 2010, addressed the food assistance needs of internally displaced and conflict-affected people in Pakistan’s North West Frontier Province and its Federally Administered Tribal Area (FATA); EMOP 200177 lasted from 2010 to 2012 and provided emergency food assistance to families affected by the monsoon floods; and PRRO 200145 provided assistance to households to increase food security, early recovery, peace and social stability. On the development side, CP 10269.0 aimed to improve access to food while addressing gender disparities in education, health and livelihoods over the 2004-2011 period. (See Volume II, Appendix XI for detailed assessment.) Despite the range of interventions assessed, the WFP Pakistan country office notes that the MOPAN assessment does not fully capture the wide range of activities completed and many innovative approaches used by the country office during the period under review.

In the survey, respondents considered WFP adequate overall in contributing to both its expected humanitarian outcomes and its development programme areas. Its performance in improving food consumption was recognised as a strength, with 66% of respondents rating WFP strong or very strong, 26% rating it adequate, and only 7% rating it inadequate or below. All three development programme areas and humanitarian outcome B4 registered a high proportion of‘don’t know’responses (ranging from 20% to 27%).

Self-reported data indicates that in general WFP has partially achieved expected outputs. On the humanitarian side, more recent interventions seem to have been generating better results.

The project documents for the assessed operations (both humanitarian and development) indicate that disparities are problematic in the country, with women and children disproportionately affected by poor human development, particularly in terms of literacy and health. Some operations struggled to involve women in activities, yet the logical frameworks for two of the five operations assessed did not list the participation of women as an assumption. This is surprising, in particular for the development country programme, the three components of which focused on providing assistance to improve girls’ primary education, promoting safe motherhood, and creating assets for rural women.

Nevertheless, WFP Pakistan reports that it has made considerable progress in promoting gender equality given the context and existing limitations, and there is certainly evidence of great efforts in this direction (see sidebar).

4.4 RELEVANCE OF WFP’S OBJECTIVES AND PROGRAMME OF WORK TO STAKEHOLDERS

For this KPI, MOPAN assessed relevance as a measure of the extent to which stakeholders perceive that the multilateral organisation supports country priorities and meets the changing needs of its partners and the target population. The assessment is based exclusively on perception data obtained from surveyed respondent groups (i.e. MOPAN donors in-country, direct partners, recipient governments, and peer organisations) in the five assessed countries.

Overall assessment

Respondents in four of the five countries rated WFP strong in responding to countries’ development priorities, while respondents in Indonesia considered WFP’s performance in this area adequate. At the time of the MOPAN survey, WFP’s development programme in Indonesia was still in its first year of implementation and stakeholder perceptions should be understood in this context. WFP Ethiopia also indicates that it has introduced cash and vouchers in various programme activities, such as for refugees and relief, in line with the development priorities expressed in the Government of Ethiopia’s “Technical Proposal for a Greater Role of Cash in Food Assistance Disbursement in Ethiopia” (June 2012). This should further strengthen perceptions that WFP is responding to the country’s development priorities.

Across the five countries, respondents rated WFP adequate overall for providing innovative solutions to development challenges. The WFP Guatemala country office reports that it is keeping abreast of new solutions and tools for combating hunger and that it seeks to integrate these wherever relevant to the Guatemalan context. Recent and upcoming examples include: the purchase for progress (P4P) initiative, the resilience agenda, cash and voucher modalities for food assistance, etc.

Perceptions regarding WFP’s adaptability to changing conditions at the country level were mixed: Although country-level respondents in Ethiopia and Pakistan rated WFP strong, their counterparts in the three other countries rated WFP adequate.

Figure 4.12 shows the overall assessment rating and the mean scores on the three survey questions on which the assessment is based.

Figure 4.12 | KPI C: Relevance of objectives and programme of work to stakeholders, overall rating and survey mean scores by country

* Detailed scores are shown in Volume II, Appendix V.

5. Conclusions

These conclusions step away from the specific ratings of the MOPAN assessment and look at the major messages that can contribute to dialogue between individual MOPAN members and WFP and its partners. It draws on the survey findings and principal observations of the assessment of WFP’s practices and systems (key performance indicators 1-22) and the assessment of humanitarian and development results component (key performance indicators A-C).

Conclusions on organisational effectiveness

Surveyed stakeholders reported different views on what constitutes WFP’s core mandate, on the agency’s recent shift to food assistance, and on the type of programming that WFP should focus on in the future.

Nearly a quarter of survey respondents cited the organisation’s mandate and its strategic shift to food assistance as areas of concern. Although WFP’s dual humanitarian and development mandate dates back to its establishment in 1961, the organisation is primarily regarded as a humanitarian agency. Donors, in particular, appear concerned that WFP’s recent shift to food assistance may extend the organisation further into development programming and result in a duplication of roles and responsibilities with other United Nations agencies. While some donors at headquarters suggested that WFP should concentrate nearly all efforts on its humanitarian mandate by providing assistance during emergencies, some recipient government and peer organisation respondents suggested that WFP should strengthen its development work and enhance activities in livelihood and resilience-building. Evaluations note that there is a need for WFP to clarify and better communicate its mandate in response to stakeholder concerns.

WFP has built a strong reputation for responding quickly to emergencies, and documented evidence confirms the strength of its practices and systems for launching and sustaining humanitarian interventions.

Stakeholders expressed confidence in WFP’s capacity to deliver assistance in emergencies; their comments highlighted the organisation’s logistical capabilities, expansive coverage and deep field presence as organisational strengths. The MOPAN assessment found that WFP is well-positioned to deliver assistance in emergencies and has a strong comparative advantage in humanitarian settings due to many noted strengths: WFP’s strong investment and focus on emergency preparedness and response across the organisation, the reliability of needs assessments and their use to inform programming, robust security measures to protect staff, effective procurement practices, strong risk management strategies, timely response to events and disasters, harmonised procedures with programming partners, and active contribution to inter-agency plans and appeals.

WFP’s performance in implementing the seven cross-cutting themes assessed by MOPAN was generally considered adequate. Areas requiring attention include gender mainstreaming throughout WFP’s work and the need for formal directives to require the consistent application of human rights-based standards in its development programming.

Although WFP formulated a policy and action plan to improve gender equality mainstreaming following an evaluation in 2008, in practice there have been insufficient dedicated financial and human resources to ensure their implementation. As a result, there are gaps in staff capacity to translate mainstreaming efforts into practice, with few projects qualifying as gender sensitive according to the Inter-Agency Standing Committee gender marker.

In its humanitarian work, which constitutes the bulk of WFP’s portfolio, WFP has developed a policy, implementation approach, guidelines and training, and defined roles and responsibilities on protection to ensure the application of humanitarian principles and standards. On the development side, however, WFP’s commitment to integrating human rights-based standards could be strengthened. The organisation indicates that its protection policy presents standards that are coherent with those found in the UN Common Statement of Understanding on Human Rights-Based Approaches to Development Co-operation and Programming (2003), and that its protection approach and guidelines are thus also applicable in development settings. While some offices have initiated measures to integrate protection standards into their development programming and have trained staff accordingly, the assessment team notes that there are currently no formal directives requiring country offices to mainstream the protection approach in their development work.

Results-based budgeting, which remains challenging for all multilateral organisations, was identified as an area requiring improvement.

Budgeting by results is intended to link expected or achieved results with allocations and disbursements to improve transparency, efficiency (value for money), and accountability with respect to diverse programmes.

WFP currently presents allocations and expenditures by strategic objective, but not yet by all outputs and outcomes. The organisation is committed to performance-informed budgeting and to aligning its performance planning and budgeting process fully in the coming years. In 2014, WFP will require each division or unit to articulate a results statement and to cost all contributing activities for the 2014 management plan.

WFP is committed to results-based management and has made considerable progress in improving the quality of its results frameworks and indicators. However, the organisation is not yet able to collect sufficient data to fully report on these frameworks due to gaps in staff and partner capacity and additional challenges on the ground.

The development of WFP’s strategic results framework and associated indicators has been an iterative learning process that has required country offices to make adjustments and adaptations to keep pace with changes and new demands. While the changes in indicators over time have been positive, this has also made it difficult to compare data across years. In addition, the organisation has been unable to report on the targets for outcome indicators presented in the corporate strategic results framework as the evidence from project reporting has been limited. The organisation is working to address these issues through its 2012-2014 strategy on monitoring, evaluation and reporting. It may take some time for results of this strategy to be achieved organisation-wide, especially if further changes are made to the results framework and indicators as part of the next strategic cycle (2014-2017).

As WFP moves to a new strategic cycle, the transition to the new strategic plan (2014-2017) will likely bring about substantial changes in WFP’s systems and practices.

The transition period, which will necessarily accompany the change from one cycle to the next, represents an opportunity for the organisation to revise and adjust procedures that go beyond the current MOPAN assessment. For example, the recently conducted rapid organisational assessment, the on-going change management process, and lessons learned from implementing the 2008-2013 strategic plan have made WFP quite aware of the weaknesses that it must address.

Conclusions on evidence of humanitarian and development results

WFP’s corporate performance reports do not yet provide a holistic account of the organisation’s results achievement, as the evidence base is often too limited to confidently demonstrate contributions to outcome-level change.

Both survey respondents and the document review considered the organisation’s ability to deliver immediate, short-term results during humanitarian emergencies (i.e. distributing food to those in need and saving lives) to be key strengths. However, limitations in the availability of data currently constrain the degree of confidence that can be given to conclusions on achievement of outcome and strategic objective-level change throughout WFP’s programming, Challenges in measuring and reporting on higher- level results are common among organisations in the humanitarian system. The 2012 report by the Accountability and Performance in Humanitarian Action (ALNAP) on the state of the humanitarian system indicated that there are very few examples of outcome/impact evaluations of humanitarian assistance, and that humanitarian organisations are primarily reporting evidence of outputs. Nevertheless, WFP’s evidence of higher-level results should improve in the coming years as the organisation continues to build capacity to monitor and report on outcomes in the field.

At the country level, there is evidence that WFP is achieving its expected outputs and making some contributions at the outcome level. However, a number of factors affect WFP’s measurement of results at the country level.

The document review found clear evidence of the achievement of expected outputs and some evidence of contribution to outcome achievement at the country level. Across the five sampled countries, WFP’s overall performance in delivering its intended results was rated adequate by surveyed stakeholders.

WFP generally operates in challenging and often volatile environments (e.g. conflict or war-torn regions and countries experiencing recurrent climatic shocks) where the delivery of food or non-food items is not simple and where needs may fluctuate significantly and rapidly. For instance, it is not uncommon for the number of beneficiaries within a targeted zone to increase midway through an operation, which can seemingly reduce evidence of earlier progress. Many other factors add to the complexity of WFP’s work both in delivering products and services and in measuring results: upsurge of violence limiting access to sites, disease outbreaks such as cholera, beneficiaries’ inability to travel to distribution sites (e.g. inability to attend school), short duration of WFP operations, absence of baseline data in emergencies, limited capacity of implementing partners to monitor results, etc. A key challenge in capturing outcomes as well as outputs is that food, which is the main commodity provided by WFP, is only one of many factors that affect food security and nutritional status.

Country offices often have limited resources and struggle to balance the humanitarian imperative to save the most lives possible with the need to dedicate resources to monitoring and evaluation to prove the effectiveness of its programming. Nevertheless, in order to demonstrate results achievement and attract continued funding, a strong monitoring and evaluation function at the country level is necessary. Though the sample of reviewed countries was very small, there is some evidence that efforts are being made at the country level to improve monitoring and evaluation systems.

List of Figures

Figure 1.1 Number of survey respondents for WFP by country and respondent group
Figure 1.2 WFP - Distribution of responses (n=298) on all questions related to micro-indicators on organisational effectiveness
Figure 3.1 Overall ratings on key performance indicators of WFP’s organisational effectiveness (survey mean scores and document review ratings)
Figure 3.2 Overall ratings of WFP organisational effectiveness by respondent group
Figure 3.3 Performance area I: Strategic management, survey and document review ratings
Figure 3.4 Performance area I: Strategic management, mean scores by respondent group
Figure 3.5 KPI 1: Providing direction for results, ratings of micro-indicators
Figure 3.6 KPI 2: Corporate strategy and mandate, ratings of micro-indicators
Figure 3.7 KPI 3: Corporate focus on results, ratings of micro-indicators
Figure 3.8 KPI 4: Focus on cross-cutting priorities, ratings of micro-indicators
Figure 3.9 KPI 5: Country focus on results, ratings of micro-indicators
Figure 3.10 Performance area II: Operational management, survey and document review ratings
Figure 3.11 Performance area II: Operational management, mean scores by respondent group
Figure 3.12 KPI 6: Resource allocation decisions, ratings of micro-indicators
Figure 3.13 KPI 7: Results-based budgeting, ratings of micro-indicators
Figure 3.14 KPI 8: Financial accountability, ratings of micro-indicators
Figure 3.15 KPI 9: Using performance information, ratings of micro-indicators
Figure 3.16 KPI 10: Managing human resources, ratings of micro-indicators
Figure 3.17 KPI 11: Performance-oriented programming, ratings of micro-indicators
Figure 3.18 KPI 12: Delegating authority, ratings of micro-indicators
Figure 3.19 KPI 13: Humanitarian principles, ratings of micro-indicators
Figure 3.20 Performance area III: Relationship management, survey and document review ratings
Figure 3.21 Performance area III: Relationship management, mean scores by respondent group
Figure 3.22 KPI 14: Procedures consider local conditions and capacities, ratings of micro-indicators
Figure 3.23 KPI 15: Supporting inter-agency plans and appeals, ratings of micro-indicators
Figure 3.24 KPI 16: Contributing to policy dialogue, ratings of micro-indicators
Figure 3.25 KPI 17: Participation in the cluster system, ratings of micro-indicators
Figure 3.26 KPI 18: Harmonising procedures, ratings of micro-indicators
Figure 3.27 Performance area IV: Knowledge management, survey and document review ratings
Figure 3.28 Performance area IV: Knowledge management, mean scores by respondent group
Figure 3.29 KPI 19: evaluating results, ratings of micro-indicators
Figure 3.30 KPI 20: Presenting performance information, ratings of micro-indicators
Figure 3.31 KPI 21: Disseminating lessons learned, ratings of micro-indicators
Figure 3.32 KPI 22: Availability of documents, ratings of micro-indicators
Figure 4.1 Humanitarian and development results component – overall ratings
Figure 4.2 KPI A: Evidence of the extent of progress towards organisation-wide results, overall rating
Figure 4.3 WFP’s measurement of and reporting on organisation-wide results
Figure 4.4 Stakeholder survey of WFP progress towards strategic objectives - mean scores
Figure 4.5 Evidence of extent of progress towards WFP’s expected outcomes related to strategic objective 1 for 2012
Figure 4.6 Evidence of extent of progress towards WFP’s expected outcomes related to strategic objective 2 for 2012
Figure 4.7 Evidence of extent of progress towards WFP’s expected outcomes related to strategic objective 3 for 2012
Figure 4.8 Evidence of extent of progress towards WFP’s expected outcomes related to strategic objective 4 for 2012
Figure 4.9 Evidence of extent of progress towards WFP’s expected outcomes related to strategic objective 5 for 2012
Figure 4.10 KPI B: Evidence of the extent of contribution to country level goals and priorities, overall rating
Figure 4.11 WFP’s measurement and reporting on country level results
Figure 4.12 KPI C: Relevance of objectives and programme of work to stakeholders, overall rating and survey mean scores by country

---

[1] WFP’s strategic plan for the upcoming 2014-2017 cycle maintains the organisation’s focus on food assistance, but responds also to an evolving external operating environment and to lessons learned from the 2008-2013 cycle.

[2] WFP programming expenses in 2012: Ethiopia (USD 358.6 million), Guatemala (USD 9.7million), Indonesia (USD 6 million), Mozambique (USD 23.1 million), and Pakistan (USD 256.2 million). According to WFP’s Annual Performance Report for 2012, programming was primarily development-focused in Indonesia, but focused on relief interventions (i.e. emergency operations and protracted relief and recovery operations) in the other four countries. For detailed information on the country selection, please consult Appendix I in Volume II of this report.

[3] Resolution XVII of the United Nations General Assembly in 1962 noted that: “The General Assembly (...) [e]xpresses its satisfaction that, following joint action by the Economic and Social Council and the Council of the Food and Agriculture Organization of the United Nations, pursuant to resolutions 1496 (XV) and 1714 (XVI), a United Nations/FAO World Food Programme was established will play a vital role in the efforts of member countries to meet the need for food supplies in the event of emergencies and to assist those countries in their economic and social development.”

[4] WFP (2010), Revolution: From Food Aid to Food Assistance – Innovations in Overcoming Hunger.

[5] WFP (2011), How WFP Country Offices Adapt to Change: A Strategic Evaluation.

[6] The plan for the next cycle builds on WFP’s current strategic direction, responds to an evolving external operating environment, and incorporates lessons learned. As a result, the organisation’s strategic objectives for 2014-2017 have been reviewed and amended.

[7] WFP (2013), Annual Performance Report for 2012.

[8] WFP (2010), General Regulations and Rules.

[9] WFP (2013), Update on WFP’s Role in the Humanitarian Assistance System.

[10] Ibid.

[11] WFP is actively engaged in the following clusters at both the global and country level: education; water, sanitation and hygiene (WASH); health; nutrition; protection; early recovery; emergency shelter; and agriculture.

[12] See Volume II, Appendix I, section 3.4 for an explanation of the weighting formula and scheme.

[13] While most KPIs and micro-indicators were considered in the document review, not all were rated. See section 2.3.

[14] Respondent comments that included more than one theme were coded in various categories.

[15] WFP (2012), Fit for Purpose – WFP’s New Organizational Design.

[16] This refers to WFP’s strategic plan for the 2008-2013 period.

[17] For additional information, please refer to UNGA resolutions 1714(XVI) and 2095 (XX).

[18] Throughout this report, the assessment of WFP’s corporate results frameworks is based on the management results framework revised in March 2013 and the strategic results framework revised in September 2012.

[19] The output statement is: “Food and non-food items, cash transfers and vouchers distributed in sufficient quantity and quality to target groups of women, men, girls and boys under secure conditions.”

[20] For instance, there would be little value in trying to determine in advance the number of children WFP will aim to help lift out of acute malnutrition globally before there is knowledge of the crises that will occur. WFP’s humanitarian work is very much based on responding to assessed needs.

[21] Cross-cutting issues included in this KPI purposefully do not reflect the organisation’s core mandate, but rather other key priorities that the MOPAN members consider important to integrate both vertically and horizontally across the organisation.

[22] The wording of the four priorities varies slightly to reflect each organisation’s mandate and operations.

[23] WFP (2012), Decision and Recommendations of the Annual Session of the Executive Board, 2012.

[24] The Annual Performance Report for 2012 reports that only 24% of new projects in 2012 achieved an IASC gender marker code of 2a or 2b (projects that have the potential to contribute significantly to gender equality or whose principal purpose is to advance gender equality); this is well below the 2012 target of 50%. In addition, according to reports from February 2013, WFP was only meeting targets for 4 of the 15 UN SWAP indicators which are measured across the United Nations to assess progress within the system on gender-related work, including the mainstreaming of the gender perspective across all operations.

[25] WFP (2012), WFP Management Plan (2013-2015).

[26] WFP (2012), Annual Performance Report for 2012.

[27] Given the nature of WFP’s mandate, the focus of this MI is on examining how the organisation promotes capacity building for good governance by strengthening countries’ ability to reduce hunger and promote food security.

[28] Based on lessons learned from consultations with stakeholders, independent evaluations, and the Mid-Term Review of the 2008-2013 strategic plan, WFP has recognised the importance of further providing capacity development support to governments and communities so that they may in turn improve access to food for their poor and vulnerable populations. Thus, the 2014-2017 strategic plan integrates capacity development across all of the organisation’s strategic objectives.

[29] WFP (2012), Humanitarian Protection Policy.

[30] WFP (2012), Getting to Zero: WFP’s Role as a UNAIDS Cosponsor.

[31] WFP (2010), WFP HIV and AIDS Policy.

[32] The number of countries which received WFP assistance for tuberculosis (TB) and HIV/AIDS prevention activities dropped from 43 to 33 between 2009 and 2012, and the number of beneficiaries fell by 40%.

[33] WFP (2013), Update on WFP’s Response to HIV and AIDS.

[34] This refers to the strategies developed for emergency operations, protracted relief and recovery operations and development country programmes.

[35] As noted in WFP’s Indicator Compendium (n.d.): “The Indicator Compendium is a compilation of corporate and project-specific indicators for WFP country offices, regional bureaux and headquarters. It is designed as an appendage to the Strategic Results Framework, and forms the basis for its food assistance interventions.”

[36] Emergency operations (EMOPs) and protracted relief and recovery operations (PRROs)

[37] Country programmes (CPs)

[38] WFP (2009), Summary Report of the Strategic Evaluation of WFP’s Contingency Planning (2002-2008).

[39] Voluntary contributions are payments left to the discretion of individual member states. Assessed contributions are payments made as part of the obligations that nations undertake when signing treaties. The UN Budget Process, http://www.betterworldcampaign.org/issues/funding/the-un-budget-process.html consulted May 19.

[40] Only distributed commodities (food, cash and vouchers) are costed.

[41] According to the OECD/DAC definition, results-based budgeting is: “A budgeting method that links appropriations to outcome level performance targets. Expected results justify resource requirements and actual performance in achieving results is measured by predefined performance targets.”

[42] 163 medium-risk and high-risk internal audit recommendations were outstanding for more than a year in 2012, and 34% of external audit recommendations for the 2007-2013 period were outstanding as of June 2013.

[43] In 2012, food procurement alone accounted for more than a quarter (28%) of WFP’s direct expenses (WFP Annual Performance Report for 2012, issued in 2013). While contributions can be made in cash to WFP, the organisation also receives contributions of food (e.g. flour, beans, oil, salt and sugar) and non-food items necessary to grow, store and cook food (e.g. kitchen utensils, agricultural tools and warehouses) from donors. Since WFP has no independent source of funds, all donations (either in cash or in-kind) must be accompanied by the cash needed to move, manage and monitor WFP food assistance.

[44] WFP (2013), Annual Performance Report for 2012.

[45] The commodity movement processing and analysis system, or COMPAS.

[46] JIU (2010), Review of Enterprise Risk Management in the United Nations System.

[47] WFP (2013), Annual Report of the WFP Inspector General.

[48] WFP (2013), Annual Performance Report for 2012.

[49] The Management Assessment Centre is a joint initiative of the UN Rome-based agencies.

[50] WFP (2012), Report of the External Auditor on the Management of Human Resources.

[51] UNSMS policies which WFP follows include the Security Risk Management (SRM) processes, Minimum Operating Security Standards (MOSS), Guidelines for Determining Acceptable Risk, the Programme Criticality Framework, and Security Level System.

[52] WFP (2012), WFP Enterprise Risk Management: the Corporate Risk Register, Directive No.: RM2012/004 (internal document).

[53] OECD (2000), Results based management in the development co-operation agencies: a review of experience.

[54] Annex I of the 2014-2017 strategic plan indicates that: “WFP will provide assistance in a manner that is operationally independent of the political, economic, military or other objectives that any actor may hold with regard to areas where such assistance is being provided.”

[55] Crawford et al. (2010), “WFP Protection and Food Assistance Programmes: Promoting Safety, dignity and more Effective Hunger Outcomes in Humanitarian Crisis”, in Revolution: From Food Aid to Food Assistance, WFP, Rome. Also: WFP (2010), Humanitarian Assistance in Conflict and Complex Emergencies: International Conference.

[56] WFP (2010), Humanitarian Assistance in Conflict and Complex Emergencies: International Conference.

[57] The Policy on Transition is scheduled to be presented to WFP’s Executive Board in November 2013.

[58] These are known also as the IASC Principals.

[59] The Office of Evaluation compiled a list in 2013 of reports described by country offices as evaluations, reviews, self-evaluations and end-line surveys for the 2008-2012 period which had been reported to regional bureaux. Based on this list, the yearly target for single- operation decentralised evaluations has not been attained.

[60] Evaluation reports and preceding products are only approved by the Office of Evaluation Director when the EQAS criteria are met.

[61] The first UNEG/OECD-DAC peer review of WFP’s evaluation function was conducted in 2007 and is referred to under MI 19.1.

[62] For the majority of outcome indicators, information is only available on the number of projects that have improved or stabilised during the year. While the assessment noted in MI 3.3 that it was sensible for WFP to not ascribe an absolute numerical value for indicator targets organisation-wide (given its main line of work in responding to emergencies for which demand is hard to foresee), the organisation could provide such values/estimates in its reporting once results have been achieved.

[63] WFP (2013), Management Response to the Recommendations of the Annual Evaluation Report 2012.

[64] World Food Programme (n.d.). Summary of the Project Planning and Prioritization - Internal document shared by WFP.

[65] Please refer to sections of chapter 3 on KPIs 2, 3, 9 and 20 for a more in-depth analysis of WFP’s results-based systems and practices.

[66] All indicators are defined in an indicator compendium to ensure consistency in measurement and analysis.

[67] A list of documents consulted is provided in Vol. II, Appendix VII.

[68] Given changes in indicators used to track progress and improvements in reporting rates over time, the analysis places greater focus on the Annual Performance Report for 2012 (a document issued at Executive Board meetings in 2013).

[69] WFP Office of Evaluation (2012). Four Strategic Evaluations on the Transition from Food Aid to Food Assistance: A Synthesis.

[70] World Food Programme (n.d.). Summary of the Project Planning and Prioritization - Internal document shared by WFP.

[71] The assessment team estimates that nearly a third of reported values in 2012 could not be used by the organisation to track progress on projects.

[72] World Food Programme (2012). From Food Aid to Food Assistance-Working in Partnership: A Strategic Evaluation.

[73] Please refer to the sections in chapter 3 on KPIs 3, 5, 19 and 20 for the analysis of WFP’s results-based systems and practices.

[74] According to Annex IX-A of WFP’s Annual Performance Report for 2012, only 7 to 8% of the organisation’s direct expenses have been within the development category since 2009.

[75] As noted by WFP Ethiopia, “all outcomes reported on the Country Programme are UNDAF-related, as was agreed by the UN country team. Hence, some of these outcomes need to be understood as per corporate outcomes.”

[76] WFP Office of Evaluation (2011). Impact Evaluation: The Contribution of Food Assistance to Durable Solutions in Protracted Refugee Situations - Its impact and role in Ethiopia.

[77] WFP Office of Evaluation (2009). Mid-term Evaluation of the Ethiopia Country programme 10430.0 (2007-2011), Final Report.

[78] WFP Office of Evaluation (2011). WFP 2008-2013 Purchase for Progress (P4P) Initiative: A Strategic Evaluation (Mid-term).

[79] The country programme was included in the assessment even though it has only been active for a year. This exception was made because the CP represents a major strategic shift for WFP in the country, and in response to a formal request by the Indonesia country office.