Below is the Executive Summary. Click here for the full report (PDF).

EXECUTIVE SUMMARY

Evaluation Purpose and Evaluation Questions

The subject of the evaluation was post-conflict reconstruction and stabilization programming in Georgia, supported by Section 1207 funding in FY 2008 and 2009. The evaluation purpose was to assess whether the full set of 1207 program activities implemented through USAID and DoS/INL, PRM, and PM achieved their stated objectives. The evaluation was also intended to extract lessons learned from its findings and conclusions to help inform decision-making regarding future conflict stabilization initiatives. Evaluation questions included:

1. Did the three programs achieve their objectives? If not, why not?

2. What are the takeaways and lessons learned, such as things that worked well or didn’t, that would benefit any on-going programs or future stabilization initiatives?

3. Did the funded agencies fulfill the requirements of the 1207 program as a unified reconstruction and stabilization program with specific implementation guidance and reporting requirements? Did the 1207 funds keep their identity as a stand-alone program or were they subsumed into other initiatives?

Program Background

The five-day war between Russia and Georgia in August 2008 killed hundreds, displaced 192,000 ethnic Georgians, and caused substantial damage to the country’s infrastructure and economy. In response, the USG formally pledged $1 billion in overall post-conflict assistance to the GoG. Of this amount, $100 million was to be provided through 1207 funds in support of humanitarian relief and economic revitalization and stabilization efforts. The overarching goal of the 1207 program support was to: meet urgent needs; help the country stabilize in the immediate aftermath of the war; maintain public confidence in democratic governance; and, restore economic gains enjoyed prior to the war.

Evaluation Approach, Methods, and Limitations

The Evaluation Team employed a flexible, industry-standard, mixed-methods approach that utilized an array of tools and resources to provide information about the nearly two dozen 1207-funded projects’ design, implementation, results, and long-term effects. The mixed methods included document reviews, key informant interviews, group discussions, and in-person visits to activity sites throughout Shida Kartli – the region most damaged in the conflict and where most of the IDPs were resettled. Information and data obtained through these qualitative methods were augmented by an electronic mini-survey focused on gathering data to assist in responding to the third SOW evaluation question

Extensive pre-fieldwork in Washington, primarily in the form of key informant interviews and document reviews, lasted three months. It was followed by the fieldwork phase of the evaluation, which took place over a three week period in January and early February 2015. Data collected came from various sources, including: interviews with GoG, Implementing Partner (IP), and Embassy/USAID personnel and IDP beneficiaries; review of documents provided by DoS, USAID, and IPs; and, site visits in Shida Kartli.

Projected as a potential limitation in the evaluation Workplan, the unusually large number of 1207 program projects (23) and associated USG funding agencies (4) and IPs (16) involved did create major difficulties for the Team. Keeping up with the volume of documents became a continuous issue, as did the ability to cover all the projects in sufficient breadth and depth in the fieldwork to generate sound findings. The end result of these factors was the Team effectively ran out of the contracted amount of LOE as the fieldwork was ending but, with DoS/CSO’s support, arrangements were made for the remaining work to be completed.

Findings

Question 1: Results Achieved

Did the three programs – food, shelter and livelihood requirements for IDPs, economic revitalization and stabilization, and police support – achieve their objectives? If not, why not?

Food, shelter and livelihood requirements for IDPs

The three objectives established in connection with the allocation of 1207 funding under this Program heading were accomplished. Emergency food assistance was provided to IDPs during the winter 2008-2009 by the World Food Program, with the support of $5 million in 1207 funds. The assistance focused on three activities: general relief food distribution; direct cash transfers; and food-for-work and cash-for-work.

In the case of the second objective, during 2009 UNHCR used $9 million in 1207 funding to implement activities in Shida Kartli and elsewhere in Georgia aligned with its “shelter-plus” strategy; i.e., IDPs targeted for shelter assistance also received aid aimed at promoting their self-sufficiency and well-being. The activities resulted in the provision of adequate shelter for and the local integration of IDPs, in addition to developing enhanced protection for particularly vulnerable IDPs, including the elderly, victims of gender-based violence, and vulnerable women.

Regarding the third objective, rebuilding livelihoods of IDPs in Shida Kartli, activities implemented by the Citizens Network for Foreign Affairs (CNFA) and FAO provided agricultural inputs and technical assistance and animal feed and veterinary services to farmers, respectively. Through its $19.5 million ($14 million of which were 1207 funds) Georgia Agricultural Risk Reduction Program (GARRP), CNFA provided assistance for winter wheat and corn planting and orchard inputs that resulted in crop production from 36,121 hectares of agricultural land. FAO’s $5.5 million 1207-funded livestock assistance activity – animal feed, de-worming, water supply, and training – protected the livelihood of vulnerable recipients in Shida Kartli by allowing them to refrain from slaughtering or selling their cattle.

Economic Revitalization and Stabilization

Similarly successful outcomes were produced regarding the five objectives under this Program heading. One objective was displaced persons resettlement assistance, which embraced two activities: a $1.5 million energy subsidy program implemented by USAID and a $500,000 IDP property registration project carried out by the Association for Protection of Landowner Rights (APLR). The energy subsidy program covered payments on behalf of newly resettled IDPs who were getting gas and electricity at no cost under a deferred billing arrangement between utility companies and the GoG. The APLR project provided IDPs property registration assistance and help in obtaining title to the land transferred to them by the GoG.

Another objective, continued recovery assistance for Shida Kartli, was and/or is being accomplished through two major projects. The first of these was the $5.1 million CNFA-implemented Access to Mechanization Project (AMP), which sought to increase smallholder farmers’ productivity and incomes by improving their access to and utilization of machinery services. The other is the $20 million (including $7.37 million in 1207 funds) New Economic Opportunities (NEO) Initiative still being implemented by Chemonics. NEO is an undertaking that attempts to move from initial efforts to restore livelihoods of conflict-affected populations to longer-term, comprehensive economic development by: increasing rural incomes; improving agricultural productivity and food security; addressing critical, small-scale infrastructure constraints in communities; and helping IDPs to sustain their households.

A third objective, employment and vocational training in construction trades, involved two related activities, the Vocational Education Project (VEP) and the Job Counseling and Placement Project (JCP). VEP was a $5.5 million workforce development effort supported by USAID and 1207 Program funds ($1.8 million), implemented by the American Institutes for Research (AIR). It successfully increased the supply of trained graduates in the tourism and construction trades through the establishment of five vocational education centers. Implemented by the International Organization of Migration (IOM), the $3 million, 1207-funded JCP provided job counseling, referral services, training, and issued supplemental grants to new small businesses.

Under the objective, rebuilding schools and municipal infrastructure, several major projects have produced successful results, including the School Rehabilitation (SRP) and BAVSHVI Projects, implemented by CHF and the Municipal Infrastructure Project (GMIP), implemented by the Municipal Development Fund (MDF). Through the SRP, an extension of its Georgia Employment and Infrastructure Initiative (GEII), and the ensuing BAVSHVI project, CHF worked to renovate, rebuild and/or improve school buildings and housing for orphans, the disabled, and other vulnerable individuals. Supported by $2.4 million in 1207 funds, the SRP renovated, repaired, and upgraded 16 badly deteriorated and/or conflict-damaged schools in and around the Shida Kartli municipalities, Gori and Kareli. The SRP’s overarching goal, which it achieved, was to create a healthier learning environment for the students attending these schools. A $12.9 USAID-supported activity that included $9 million in 1207 Program funds, BAVSHVI enhanced learning opportunities and better living conditions for orphans and vulnerable children by improving the physical conditions of 50 schools and 25 small group homes, while at the same time providing employment opportunities for hundreds of VEP graduates and JCP clients. Lastly, under the $17.7 million GMIP, the $11.1 million portion in 1207 funds has been dedicated primarily to infrastructure activities in Shida Kartli, such as the rehabilitation of roads and improvements in water and irrigation works.

The last objective, security for IDPs, involved the HALO Trust and NATO Maintenance and Supply Agency (NAMSA) in clearance of explosive remnants of war (ERWs) and mines. As part of a joint DoS/UNICEF effort, HALO received $2.1 million in 1207 program funding to clear ERWs in Shida Kartli ($.9 million) and Abkhazia ($1.2 million). HALO completed its clearance operations in Shida Kartli in 2009. In March 2011, the NAMSA activity, supported in part by $500,000 in 1207 funds, trained 66 members of the Georgian Army Engineer Battalion in humanitarian mine clearance, battle area clearance, and large ordnance disposal.

Police Support

Adding to previous and ongoing USG support for GoG efforts to reform its law enforcement agencies, INL’s $20 million, 1207-funded Police Support Program responded to the immediate need to assist the MoIA in rebuilding the operational capacity and physical infrastructure of the nation’s police forces after the Georgia-Russia conflict. Specifically, the Program focused on strengthening the police forces to help:

1. secure conflict-affected areas for those that had temporarily fled and IDPs who could not return to their homes;
2. provide a safe environment for commercial activity; and,
3. demonstrate an active patrol presence and investigative capacity to deter organized crime or militia activity in these same areas.

To these ends, INL efforts were organized around an array of “commodity-based,” technical support, and training activities, including: patrol vehicles, a national crime database, emergency communications, IT and radio packages, tactical gear, emergency data recovery, and training and development. Overall, the intended objectives of these activities were accomplished and, in some cases – e.g., the expansion of the TETRA emergency communications system – produced very important results , i.e., connecting the eastern and western parts of Georgia for the first time. In addition, however, INL’s overall record of accomplishment with the Police Support Program has been marred by implementation problems that have brought about significant delays in achieving results within the anticipated timeframes.

Question 2: Takeaways and Lesson Learned

What are the takeaways and lessons learned, such as things that worked well or didn’t, that would benefit any on-going programs or future stabilization initiatives?

Lessons Learned

1. Future post-conflict/stabilization initiatives in general – but, large-scale ones in particular – need to anticipate and plan for items such as increased personnel, specialized expertise, and technical equipment that will likely have to be deployed in relatively short timeframes for the assistance to be effective.

2. Capacity building for officials and staff at the national, regional, and local levels became an essential part of shoring up the GoG’s limited ability at the outset to manage its response to the difficult and complicated situation in conflict-affected areas, such as Shida Kartli.

3. Cost sharing is an effective way to engage local government authorities and keep them involved in following through on project activities.

4. Flexibility in being able to adapt funds to changing circumstances and conditions can be an important asset for IP officials and staff.

5. The participatory approach used in the Schools Rehabilitation Project could well serve as a model for future projects where community involvement is viewed as a key feature of the implementation plan.

6. Being part of an overall assistance effort, which involved enormous sums of money focused on a small country, created problems for some IPs whose activities were supported by 1207 funds.

7. Location of IDP settlements are thought to have had unintended consequences.

8. The absence of 1207 program performance management (PMP) and monitoring plans and other related reporting guidance has been consequential.

Things That Worked Well

Food-for-work/cash-for-work (WFP) – 61 projects, affecting 65,000 beneficiaries (including 15,000 IDPs) that rehabilitated irrigation channels, improved arable land and feeder roads, constructed potable water pipelines, and dug water wells.

Anti-parasite treatment (FAO) – 38,000 animals treated (78 percent of the total in Shida Kartli), which helped ensure good nutritional intake during the feeding period and reduced the chance of re-infection upon return to pasturelands and mixing with other cattle.

Electronic vouchers (CNFA/GAARP) – bank card-like voucher that enabled eligible IDP farmers to purchase necessary agricultural tools and supplies for their orchards; worked so well that it became a model for other GoG programs and activities.

Farm machinery services (CNFA/AMP) – 21 farm machinery service centers (MSCs) addressed countrywide shortages of agricultural machinery by purchasing 82 tractors and 235 other farm implements and providing planting and harvesting services to 16,539 small-scale farmers.

Disposal of explosive remnants of war (EMWs) (HALO Trust) – HALO teams cleared and returned a total of 3,400 hectares of land to safe, productive use, benefitting 38,000 individuals.

School improvement Plans (CHF/SRP) – the extraordinary end-product of a highly inclusive process conducted during the rehabilitation of 16 public schools in Shida Kartli, which involved school staff, students, parents, trustees, and community members working together.

Increased employment opportunities (CHF/BAVSHVI) – a concerted effort among IPs that helped hundreds of graduates/clients of USG-supported vocational education and job placement programs obtain short-term employment in school/small-group home rehabilitation activities.

Things That Didn’t Work Well

Unwanted settlement buildings (UNHCR) – bathhouses were built in several IDP settlements that did not provide any benefit to residents, after being constructed over their objections.

Land documentation (APLR) – IDPs were interested in registering their abandoned properties, but this diminished as hope faded for a return. IDPs the Evaluation Team spoke with about this were unenthusiastic; still, there is now hard data on what someone owned, if this ever comes up.

Disabled access (CHF/BAVSHVI) – because there were no standards or local design templates and very little tradition of providing for disabled access, during the early part of the project contractors were forced to implement ad-hoc solutions, resulting in questionable deliverables.

Design-build contracts (MDF) – local contractors “lack[ed] the capacity to execute these innovative contracts in any meaningful sense,” causing major delays and disputes in a number of important irrigation projects.

Question 3: Program Requirements

Did the funded agencies fulfill the requirements of the 1207 program as a unified reconstruction and stabilization program with specific implementation guidance and reporting requirements? Did the 1207 funds keep their identity as a stand-alone program or were they subsumed into other initiatives?

Regarding the first question which, in brief, asks whether or not the funded agencies fulfilled 1207 program requirements, the bottom line answer is: yes, they did. In the case of the second question, the summary finding is, yes, on both accounts – i.e., in some instances 1207 funds kept their identity while in other cases they were subsumed into other initiatives.

Conclusions

As administered by USAID and DoS/INL, PRM, and PM, 1207 funds clearly served their intended purpose and, equally important, did so in a manner consistent with the guiding principles regarding their use established by the 1207 Selection Committee. Without exception, the nearly two dozen projects that received 1207 funds accomplished their stated objectives or, in those few cases where their activities were still ongoing, had already achieved significant results.

Notwithstanding these noteworthy successes, it is important to point out that the $100 million in 1207 funding committed and spent was part of the USG’s overall $1 billion aid package and was in effect at the same time that an additional $4.5 billion was committed by other bilateral donors and multilateral organizations. This enormous amount of funding was an expressed concern among some key informants. One of them, for example, repeatedly used words like “overlap” and “over-saturation” in describing the situation he observed in 2009, while another said there was “too much money, too quickly” and insufficient staff to implement the activities the funds supported.

Finally, there are the perhaps confusing findings in response to evaluation question number three. Most particularly, the evidence suggests that in some cases 1207 funds kept their identity in a “stand-alone” sense, but in others were subsumed into other initiatives – the ratio was about evenly divided. Moreover, aside from Embassy/USAID officials and staff and a smattering of senior IP and GoG personnel, the Team found that virtually no one at the activity level knew or, one might argue, had any reason to know, that their activity was all or partly funded by 1207 program support. At most, they might have been aware that the funding was from the USG and/or USAID. In the end, it remains to be seen if the stand-alone/subsumed identity issue will be deemed a matter for further consideration by DoS/CSO and/or the 1207 Review Committee.

Recommendations

1.The experience with the 1207 Program in Georgia suggests that there is room for improvement in planning for and initially responding to future post-conflict situations in general, and large-scale ones in particular. In the case of Georgia difficulties at the Embassy arose at the outset and, in some instances continued on well into program implementation, in the areas of personnel, specialized expertise, and technical equipment. Accordingly, DoS/CSO and/or the 1207 Review Committee should consider the development of appropriate procedures – possibly in the form of an amendment to the 1207 Guidance or other such documentation – to better enable those responsible to respond to post-conflict/stabilization situations and assure that necessary staffing and resources are in place as quickly and sustainably as possible.

2. In most, if not all, cases government agencies of countries selected to receive 1207 program funds are unlikely to possess sufficient capacity to manage emergency humanitarian assistance and other urgent needs effectively. Accordingly, DoS/CSO and/or the 1207 Review Committee should consider the idea of having some kind of capacity building “toolkit” available for immediate use upon a country’s selection as a recipient of 1207 program funding.

3. Performance management (PMP) and monitoring plans should be a “no-exceptions” requirement for all 1207 country programs, since without them there is no systematically sound way to track progress (or lack thereof) in achieving project objectives. Nor is there a basis for comparing the situation at the start of a project with what it may have achieved by the end. While there were understandable reasons why the decision was made to forego the specified reporting requirements in Georgia’s case, the latter situation of having no basis for comparing the program’s start and end points is what occurred.