2015/The Philippines/Partnership for Growth

Below is the Executive Summary. Click here for the full report (PDF).

Executive Summary

The Partnership for Growth (PFG) aims to achieve accelerated, sustained, and broad-based economic growth in partner countries, including El Salvador and the Philippines, through bilateral agreements between the United States Government (USG) and the partnering countries’ national governments. Using principles set forth in President Barack Obama’s September 2010 Presidential Policy Directive on Global Development, PFG requires rigorous, joint analyses of countries’ individual constraints to growth to develop joint action plans to address the most pressing of these constraints and to establish high-level mutual accountability for the goals and activities selected to alleviate them.

The countries of El Salvador, the Philippines, Tanzania, and Ghana were selected as the first group of countries in which the United States (U.S.) and partner governments would attempt to structure new PFG initiatives, with selection based, in part, on each country’s record of accomplishment in implementing ongoing Millennium Challenge Compacts (MCC).

In November 2011, USG and the Government of the Philippines (GPH) signed a Joint Country Action Plan (JCAP) for a 5-year period for implementing two primary constraints, each with three sub-constraints, to Philippine economic growth. Each constraint was accompanied by a set of goals, policies, and actions to relieve them. The agreed-upon actions are designed to lead to goal and policy achievement, which, in turn, will mitigate the effects of currently binding constraints and, therefore, accelerate and sustain the Philippines’ rate of broad-based, inclusive economic growth.

Purpose of Evaluation

Optimal Solutions Group, LLC (Optimal) was contracted to conduct a mid-term evaluation of the PFG initiative in the Philippines. According to the evaluation statement of work (SOW), this evaluation serves two purposes.

First, the evaluation analyzes whether the PFG process demonstrates improvements over pre-PFG assistance approaches. In particular, the evaluation team examined the extent to which PFG’s whole-of-government approach (WGA) and constraints analysis (CA) led to a change in the way USG delivered development assistance.

The second purpose of the evaluation consists of two parts:

(i) determining whether PFG efforts “have been developed in such a way as to allow for the eventual determination of their impact in addressing the identified constraints and desired outcomes.”

(ii) determining whether “measurable elements of PFG are moving in the right direction, are considered necessary and sufficient to achieve PFG goals, and are contributing to national interests through the integration and coordination of work done by both governments.”[1]

The evaluation was guided by the following six evaluation questions as outlined in the SOW.[2]

Cross-Cutting Questions

1) What are the advantages and/or disadvantages of the PFG whole-of-government approach to development assistance?

2) To what extent has PFG affected the workload on national government and U.S. government staff, as compared to the workload created by traditional forms of development-assistance delivery?

3) What contributions has “non-assistance” made to the PFG process, and how can it be utilized moving forward?

Country-Specific Questions

1) For each of the constraints, are the goal-level commitments set forth in the JCAP capable of achieving the constraint-level objectives and outcomes?

2) Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results?

3) At the midterm, are the performances of the selected PFG interventions on target and creating the necessary outputs to achieve the desired outcomes?

Methodology

The evaluation team employed a mixed-methods approach for data collection and analysis of the Philippines PFG activities. The study included an extensive desk review of all available project documentation and monitoring data; an evaluability assessment and selection of goals for an in-depth case study; interviews with PFG architects[3] and leadership in Washington, D.C.; a site visit to the Philippines to conduct individual semi-structured, face-to-face interviews; telephone interviews with stakeholders who were not available in person; and an online survey administered to USG and GPH PFG staff.

The evaluation team conducted a qualitative research evaluation, which reviewed, collated, and analyzed the perceptions gathered. Using a mixed-methods approach, quantitative approaches were also used to analyze and verify information provided by PFG in situations when documentation and data were available for such analysis. The evaluation team employed triangulation to verify findings from various vantage points. Upon completion of the analysis, the team developed this report, which details the findings and conclusions, and provides recommendations and course corrections that could be employed by the Philippines PFG partners at midterm. These findings, conclusions, and recommendations are provided in response to the six evaluation questions specified in the SOW.

Findings, Conclusions, and Recommendations

Overall Advantages and Disadvantages of PFG

In general, the evaluation team found that PFG has aided in the positive movement of policy reforms in the Philippines. The initial PFG planning through the CA activity laid a good foundation for implementing the overall initiative. Additionally, PFG increased leverage for implementing USG development activities in the Philippines.

Cross-Cutting Question 1: What are the advantages and/or disadvantages of the PFG Whole-of-Government Approach to development assistance?

Findings – Advantages of the PFG WGA within USG and GPH PFG Staff

• The CA and JCAP development framed WGA dynamics.

• WGA has led to a change in the way USG programs’ development assistance is conducted in the Philippines.

• WGA has focused human and operational resources to increasing consistency and coherence in the programming and policy objectives of PFG.

• WGA has improved coordination between USG and GPH.

• WGA has led to a more consistent USG message on key GPH reforms.

Challenges/Disadvantages of the PFG/WGA

• The number of agencies to which the WGA is relevant in terms of their daily activities and decision-making is highly limited.

• There has been some dissatisfaction because of a perceived delay of PFG project implementation.

• There is confusion about decision-making authority within the PFG system, limiting WGA implementation.

• WGA requires greater staff time to be effective.

Conclusions

• PFG WGA represents a positive change in development assistance compared with previous approaches.

• WGA is most likely to be effective if purposefully incorporated into the design of the initiative and backed by high-level leadership.

• WGA promoted collaboration within and across governments.

• WGA coordination is time-consuming.

Recommendations

• Provide stronger mechanisms for inter-goal collaboration through a management team.

• Develop performance measures and collect better data to track the coordination of activities in promoting effectiveness.

• Develop a set agenda for coordination meetings.

• Introduce accountability measures.

• Identify and inform staff and key stakeholders of responsibilities for the overall management of PFG coordination.

• Establish (and update) an overarching work plan to track progress on coordination.

• Identify and track activities or actions that demonstrate coordination.

Cross-Cutting Question 2: To what extent has Partnership for Growth affected the workload of national government and U.S. government staff, as compared to the workload created by traditional forms of development-assistance delivery?

Findings

• PFG has resulted in an increase in the workload of USG and GPH staff.

• PFG has prompted a significant increase in workload focused on coordination and communication, both intra- and inter-governmental.

• There was no discernible difference in workload by gender.

• Increased workload is associated with perceived effectiveness of PFG and WGA.

Conclusions

• PFG has unambiguously increased the workloads of both USG and GPH staff.

Recommendations

• Identify management staffing that could assist in reducing the workload of PFG staff across the initiative.

• Identify and promote the use of limited resources more efficiently to reduce the overall workload.

Cross-Cutting Question 3: What contributions has “non-assistance” made to the PFG process, and how can it be utilized moving forward?

Findings

• The concept of non-assistance and examples of its application is unclear to many PFG stakeholders (particularly lower-level technical staff). Knowledge about non-assistance is varied.

• Whereas non-assistance likely facilitated broader reforms in the Philippines, few specific positive results can be directly attributed to these efforts.

• Aligning USG foreign policy goals with development goals through the WGA has the potential to increase the impact of “non-assistance” and to advance those goals, but it might also reduce the effectiveness of development aid.

• Not enough is known about the business opportunities created by PFG and the value of changing public perception as a result of PFG non-assistance activities.

Conclusions

• The lack of familiarity of USG and GPH program managers and implementers with the concept of non-assistance may mean that opportunities to employ non-assistance will be missed.

• Embassy leadership, particularly within the United States Agency for International Development (USAID), provided the clearest example of the successful use of non-assistance in the Philippines.

• The value of promoting PFG goals, attained and pending, is not always evident.

Recommendations

• PFG participants need to become more knowledgeable about the diversity of forms non-assistance takes and its value should be made known to a broad array of beneficiaries.

• Develop a management and reporting system for promoting non-assistance activities.

• Improve reporting and public awareness on non-assistance activities.

• Increase staffing whose assignments are geared toward identifying and working on non-assistance activities.

Country-Specific Question 1: For each of the constraints, are the goal-level commitments set forth in the JCAP capable of achieving the constraint-level objectives and outcomes?

Findings

• The JCAP for the Philippines PFG is outdated and has not been updated in response to implementation changes, even though it is supposed to be a living document.

• An overall formal theory of change linking activities, goals, and constraints was not fully developed.

• Additional goals and activities were included in the Philippines JCAP that were not emphasized in the CA.

• A number of JCAP goals were not being fully implemented.

• During the development of the JCAP, there were proactive efforts to absorb the views of a broad cross-section of society; nevertheless, there were differences of opinion on whether the constraints analysis and the JCAP development process were inclusive.

Conclusions

• Without a fully articulated overall theory of change, it is not possible to provide a theoretically grounded assessment of whether or not achieving goals included in the JCAP can have the desired effect on constraint-level objectives and outcomes.

• In lieu of a theory of change, it is difficult to determine whether the JCAP goals were chosen partially based upon bureaucratic considerations, organizational convenience, or the judgments of influential decision makers.

• The ambitious goals in the JCAP are unlikely to be accomplished given the resources currently available for PFG activities.

• Even though broad and ambitious, the JCAP did not satisfy some influential elements of society (e.g., civil society, independent experts, business community).

Recommendations

• Clarify the status of the JCAP and update components of the guiding document as necessary.

• Better align the JCAP with available resources.

• Be more proactive in involving civil society and the private sector at the implementation stage.

Country Specific Question 2: Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results?

Findings

• The implementation of the JCAP—viewed as a comprehensive integrated program—is not being adequately managed.

• The monitoring and evaluation (M&E) framework required by the JCAP, which would ensure standardized M&E across the PFG initiative, has not been finalized.

• The M&E plans of individual USAID projects are comprehensive, complete, and ready to be used to manage activities, but have only recently been finalized, so they are not in use for projects that are already underway.

• Quantitative information is being collected at the project and activity level, but it is not (yet) being assembled, analyzed, and used to manage JCAP implementation.

• Individual activities have M&E plans and indicators, but these are not mapped to the overall M&E of PFG implementation.

Conclusions

• JCAP implementation—viewed as a comprehensive integrated program—is not being managed.

• Since the overall M&E framework required by the JCAP has not been finalized, there is no explicit framework within which overall JCAP implementation might be monitored and evaluated.

• At the individual project level, during the time of the evaluation team’s fact-finding in-country, it would have been difficult to authoritatively use quantitative and objectively verifiable information for management because the M&E plans of most individual USAID projects had not been finalized at that point. However, recent approvals should aid the program in moving forward in terms of M&E.

• Although quantitative information is being collected at the project level, it is not (yet) being used to manage JCAP implementation.

Recommendations

• A decision should be made about whether PFG JCAP implementation should be managed as a single program.

• PFG staff and implementers should be provided with guidance on how the detailed indicators in project M&E plans are to be used as quantitative management tools.

Country-Specific Question 3: At the midterm, are the performances of the selected PFG interventions on target and creating the necessary outputs to achieve the desired outcomes?

Findings

• Stakeholders’ perceptions varied in terms of whether PFG is on target. Many stated that government-to-government negotiations, collaboration on project design, and procurement timelines delayed the initiation of projects and activities.

• Implementation at the project level is generally on target, given when projects were initiated.

• Most USG staff feel that PFG is on target. Most GPH staff cite delays in procurement.

• The annual PFG scorecards indicate macro-level progress in the Philippines that is less indicative of PFG progress.

• PFG is not far enough along at this point to effectively measure progress.

Conclusions

• The PFG process takes time, particularly if initiatives are starting from scratch, as was the case in the Philippines.

• Monitoring systems for the PFG Philippines initiative need to be augmented rapidly in order to accurately measure progress toward PFG objectives.

Recommendations

• The PFG Philippines management team should define “midterm” and “on target” under PFG as these apply to each particular country and context, taking into consideration the analytical, consensus-building, design, and joint-implementation process.

• PFG Philippines leadership should facilitate the rapid completion of a comprehensive M&E framework that links outputs to outcomes for all JCAP goals, and also appropriately links these goals to the constraints.

• Reporting and monitoring from guidelines provided in one overarching standardized M&E framework needs to begin and occur regularly at the technical sub-committee level for each constraint, with regular reporting and reviews of all constraints by the steering committee.

• Conduct an annual strategic review of PFG progress across USG and GPH with a pre-established date, both as a means to review progress and to incentivize implementation.

Overall Summary and Conclusion

Overall, the evaluation team found the PFG Philippines initiative to have made progress in developing a true partnership, where both governments have a seat at the table in decision making. PFG has led to a complete shift in how development assistance is performed in the Philippines and has led to positive policy reforms, with perceptions of improved economic growth. However, the absence of an explicit M&E framework within which overall JCAP implementation is monitored and evaluated has made it a challenge to determine the performance of PFG in the Philippines at midterm. Updating the overall monitoring system will aid in appropriate tracking of the initiative, and, eventually, will provide the ability to determine the impact of PFG,

 

FOOTNOTES

1. Evaluation Statement of Work, 2013. pgs. 7–8

2. Evaluation Statement of Work, 2013. pgs. 9–10.

3. Architects refer to individuals who were part of designing and mapping out the overall PFG initiative, which led to the selection of the four countries that then initiated and implemented their own initiatives. See annex 4 (Stakeholder Types) of this report.