This publication was produced at the request of the United States Department of State. It was prepared independently by Development & Training Services, Inc. (dTS)

Executive Summary
 

A. Introduction

This report presents the findings from the evaluation of the Resident Legal Advisor and Intermittent Legal Advisor (RLA/ILA) Program funded by the Bureau of Counterterrorism (CT), and is being provided to CT. CT has employed a new results-based management approach to programming over the past year, and conducting evaluations is one piece of that equation, allowing for course corrections and better strategic management. The RLA/ILA program is the subject of its second evaluation.

B. Purpose of the Evaluation

The Statement of Work (SOW) for the evaluation lists the following six objectives:

1. Identify the threshold reason for the placement of the RLA/ILA

2. Describe the work plan developed to advance this assistance priority

3. Conduct an analysis of the work of the RLA/ILA related to this articulated priority; what percentage of the RLA/ILA portfolio was devoted to issues other than the identified priority? What is the justification for work on these other issues?

4. Prepare an objective analysis of the impact of the RLA/ILA on the objectives articulated in the work plan

5. Articulate the relationship of the RLA country or regional priorities identified in the work plan to any articulated country, regional, or U.S. National strategic priorities

6. Recommendations to improve the effectiveness of the RLA/ILA program

C. Evaluation Methodology

The methods used for the evaluation were: document review, key informant interviews, and online surveys. Documents included work plans, quarterly reports, After Action Reports (AARs), interagency agreement and acquisition (IAA) documents, program database entries, and administrative files such as a contact list of RLAs/ILAs and their dates of service. A sample of documents was drawn from Bangladesh, Kenya, Malaysia, Mali/Niger, Pakistan, Turkey, and the United Arab Emirates (UAE) based on longevity of programs within the evaluation period as well as to achieve a mix of geography and funding sources. Thirty-one key informants were interviewed from three primary audiences: 11 RLAs and ILAs, 16 CT Bureau staff, and two Department of Justice (DOJ) staff (two Embassy personnel were also interviewed). Two surveys, one directed at RLAs and ILAs and a second directed at the collaborating agencies and departments surrounding the RLA/ILA program, enlarged the sample of respondents. One survey of 20 current and former RLAs/ILAs had a response rate of 55%. The second survey of approximately 50 individuals representing the collaborating agencies, posts and departments who fund and work with RLAs/ILAs had a response rate of 22%. 

D. Evaluation Findings
 

1. Findings on Pre-Placement Recruitment and Program Design

Application

• RLAs are selected from among state and federal prosecutors. Qualifications include a J.D. degree, a law license, bar membership, a minimum of four to five years of legal experience, functionality in multi-cultural environments, and interpersonal and communication skills. In assessing professional qualifications, DOJ also looks for complex litigation experience, such as financial crimes experience, which translates well to complex terror prosecutions.
• 91% of survey respondents at State and DOJ said they were “very satisfied” with the knowledge, skills and abilities of RLAs. Among the minority, some CT informants view the federal prosecutor pool as a restriction, particularly where language requirements are an issue.
• The majority of CT key informants said CT encourages autonomy in the recruitment process because the track record is a good one.

Placement

• The threshold reason for placing an RLA or ILA is to support national security interests relative to the terrorism threat presented, or to support other aspects of U.S. security sector assistance policy.
• The Counterterrorism Finance Unit (CTF) and Regional Strategic Initiative (RSI) account for 63% and 31% of the 13 current and other, planned RLA/ILA programs, respectively. Counterterrorism Engagement (CTE) funds one ILA program in North Africa, based in Malta and focused on establishing an International Institute for Justice and the Rule of Law. Each funding source has distinct processes for initiating RLA programs. Respectively, these are generally based on Embassies’ articulation of needs (RSI), the Terrorist Finance Working Group’s (TFWG) articulation of priorities (CTF), or promoting the implementation of the Global Counterterrorism Forum (GCTF) Rabat Memorandum Good Practices (CTE).
• Multiple methods are used to assess the host country’s political will prior to program placement in a particular location. RSI relies on embassies to confirm host country commitment at the proposal stage, whereas CT engages in a consultative process with the TFWG and interagency, the country desk, Post and occasionally DOJ. Embassy personnel participating in the collaborating agency survey confirm that outreach to host country counterparts figures into program design.
• The sources for program design are embassies and the TFWG. Once on the ground, RLAs and ILAs are exceptionally well-positioned to vet program design, assess which issues have momentum, and identify which interlocutors are behind those issues.

2. Findings on Establishing Individual Program Goals

• The work plan is generally developed by DOJ’s Overseas Prosecutorial Development Assistance and Training Program (OPDAT) in consultation with CT.
• Work plans have historically centered on compliance with the inter-governmental Financial Action Task Force (FATF) and its 40 international standards for combating money laundering and the financing of terrorism and proliferation, and the development of relevant legislation. Since 2012 the GCTF Rabat Memorandum good practices have been incorporated into several work plans including Algeria, the African Union, Bangladesh, Niger, Turkey and North Africa.
• RLAs and CT both highlight the value of positioning RLA/ILA programs as promoting international standards rather than solely bilateral concerns. While the FATF recommendations are flexible and useful for assisting countries in various stages of compliance, RLAs also note that host country counterparts are most receptive to anti-money laundering and counterterrorism finance (AML/CTF) reform in the context of global economic competition and safeguarding the ability to attract investment as opposed to moral or didactic arguments.
• Work plans identify bench marks, goals, and objectives, most often referred to as goals and performance measures. Goals are structured for the longer term, particularly appropriate for U.S. diplomacy. There are no time tables specified for achieving the performance measures.
• RLA work plans reflect CT Bureau priorities. CT cites the need for more policy leadership in the Bureau (see Chapter II Section B.3 on p. 19).
• Twenty-five percent of key informants from CT are concerned about the prevalence of AML/CTF issues in RLA programming, and propose more balanced programming with broader rule of law interventions. RSI programs are perceived as more flexible.
• To be strategic, some advocate that programs stem from a rule of law framework that considers legislation, the protection of individual rights, and institutional support for equal application of the law in CT matters. A rule of law strategy has been drafted but cannot be approved or impact the RLA/ILA program before a new Coordinator is appointed.
• CTF is most active in offering strategic guidance to RLAs. In CTF the focus on a particular program can vary because the unit is not able to divide its staff regionally. Other CT key informants believe they should do more to provide policy oversight on non-CTF issues, but they have not engaged in the task. Per one informant, the Bureau lacks staff with an appropriate background to offer policy guidance in the nexus between CT and a functioning criminal justice sector. This could be improved by hiring advisors with specialized backgrounds in police, prosecution, and prisons.

3. Findings on Monitoring and Evaluating Program Progress

• Weekly and quarterly reports and AARs on trainings are the major mechanisms used to track progress in the RLA/ILA program. Weekly reports are informal; quarterly reports are required by IAAs.
• Quarterly reports are written as narratives and do not facilitate quick uptake of progress being made against all of the work plan goals and, particularly, performance measures. Some program results—particularly those related to raising counterpart skill levels—get buried amidst lengthy narratives.
• RLAs and ILAs generally have three masters: the Embassy, OPDAT, and CT. On an operational level, RLAs spend more of their time reporting out at Post as part of country teams and on working groups than to either OPDAT or CT. RLAs submit weekly and quarterly reports to OPDAT and receive stability, resources, and quality control oversight in return. RLAs characterize their frequency of communication with CT as occasional, and CT calls its relationship with RLAs informal.
• 50% of respondents to the collaborating agency survey said they give feedback on program progress “fairly often,” with another 20% answering, “very often.” CT sees its oversight role as ensuring that RLA/ILA programs reflect interagency priorities and that dynamics in the field feed appropriately into policy analysis. Two particular outcomes of this oversight are informing the work of the TFWG and expanding the work of the GCTF.
• Program successes as a whole (see Exhibit 1 on pp. 24-25) are commendable in the areas of legislative reform, institutional development for FIUs and other institutional capacity-building, enhancing international cooperation, and supporting effective prosecutions in countries such as Malaysia and Indonesia. Effective prosecutions remain a challenge for other RLA programs.
• Tallying each program’s progress toward each of its work plan goals and performance measures (collectively, “total benchmarks”) reveals that every program except Turkey has progressed toward 70% or more of its total benchmarks. Turkey has only progressed toward 38% of its 24 benchmarks, hampered by: 1) performance measures predicated on prosecutorial statistics that the RLA cannot collect; 2) a number of total benchmarks that may not be manageable for one person to collect and report on.
• The RLA/ILA program’s reporting on trainees’ use (or expected use) of their new knowledge is weak. The best RLA reporting addresses use of training by noting specific new techniques, like opening statements, that prosecutors are using in courtrooms.
• A fixed process for justifying continued funding is not in place. Usually precedent prevails; when it doesn’t, the discussion is consultative and dynamic.
• Typically circumstances outside CT’s control are the major determinants for significant changes to programs, including the coup in Mali that prompted a relocation of its RLA to Niger, and the drawdown at US Embassy Baghdad resulting in the return of the RLA to Washington six months prior to the anticipated end of her tenure. While some CT informants indicated a need for a more formal process for managing evacuation and drawdown scenarios, most felt these situations are case-specific and cannot be formalized.

4. Findings on Efficiency in Implementation/Management
 

Program Funding

• Most program funds support in-country and third-country training, legislative drafting, and related expert travel. Smaller amounts have funded a criminal justice assessment for Mali, and equipment for the Terrorism and Transnational Crime Task Force in the Indonesian Attorney General’s Office.
• The proportion of spending on the RLA to program support funds is 40% to 24%. However, RLAs secure additional program money for programs from organizations like the United Nations Office on Drugs and Crime (UNODC), United States European Command (EUCOM) and European governments; leverage expertise from international organizations like the World Bank and interagency counterparts for training; and secure cost shares from counterparts to provide training venues or offset the cost of per diem and travel for training opportunities. The RLA in Kenya is working with the EU to direct 6 million Euro to fund capacity building AML/CTF initiatives.

Length of Assignment

• The average length of service for an RLA at post is 834 days, based on the RLAs who have completed their tenure. RLAs point out that they stay longer at post on average than their Foreign Service colleagues. This assists them in capitalizing on close working relationships with host country counterparts that provide clear value to broader USG efforts.

Length of Time on Tasks

• Two-thirds of RLA survey respondents spend less than 20% of their time, or 4 days per month, on additional projects. One-third of respondents spend more than 4 days per month on additional projects. CT recognizes that, to some extent, allowing RLAs to do non-CT-related work is the cost of doing business.
• 100% of RLA survey respondents do additional work, usually for the Ambassador, Deputy Chief of Mission (DCM), or as a member of the country team. Embassy duties include chairing or participating in working groups, contributing to human rights reporting, assisting with election monitoring, and supporting operational issues. Other, additional projects include providing assistance to GCTF and the Bureau for International Narcotics and Law Enforcement Affairs (INL), and supporting DOJ partners [such as the Federal Bureau of Investigation (FBI) and Treasury] and other RLAs in the region.
• More than 70% of RLA survey respondents said they have few competing priorities in implementing their work. RLAs’ management of priorities generally requires working longer hours, but on balance they are willing and able to contribute to priorities at Post while staying on mission with respect to their AML/CTF duties.

5. Findings on Effectiveness Across Programs

• RLAs facilitate exceptional inter-agency coordination on CT. Ample evidence was provided by joint activities with ATA, Treasury, and the FBI; briefings for DOJ’s International Criminal Investigative Training Assistance Program (ICITAP); sharing information with the FBI, the Drug Enforcement Administration (DEA), Immigration and Customs Enforcement (ICE) attaches and the DOJ National Security Division on the law enforcement playing field; and participation in INL criminal justice assessments (Tunisia, Niger). Additional coordination on issues important to the overall USG presence in a location can be found in Chapter II Section E.3 (pp. 34-35).
• RLAs’ common strengths include professional expertise, deep knowledge of and access to host country legal systems and stakeholders, interpersonal skills, the ability to run and manage programs, and creativity to work around obstacles.
• Common challenges include learning different legal systems, learning to work within an Embassy (both administratively and logistically), and engaging counterparts under the parameters of the program. This includes dealing with the slow pace of reform and operating under the constraints of procedures like Leahy vetting.

E: Conclusions
 

• The RLA and ILA program is a force multiplier for U.S. foreign policy, program funding, international collaboration, U.S. law enforcement effectiveness, and reinforcing the whole of government approach:

a) Promoting US foreign policy interests in the countries where programs are active. RLAs’ ability to dialogue with counterparts candidly, consistently, and as co-equal professionals has helped convince counterparts to address policy issues in potentially intractable situations, including the Turkey FATF crisis.

b) Multiplying the US investment. Many RLAs secure additional funding and expertise for programs from USG counterparts, European governments, and host country counterparts themselves. The RLA in Niger secured an additional $600,000 from RSI to improve courthouse security, and the RLA in Kenya is working with the EU to direct 6 million Euro in funding to appropriate CT targets in the country.

c) Shaping international assistance. RLAs and ILAs use their relationships to direct available resources from across the USG to appropriate beneficiaries in country. In addition, they ensure coordination with assistance programs sponsored by other providers to avoid conflicts and duplication. Examples include RLA collaboration to carry out GTCF programming in the Sahel and coordination with the UNODC in offering terrorism-related programming in Kuwait.

d) Assisting law enforcement effectiveness. RLAs assist U.S. and host country law enforcement in pursuing terrorist targets, and reduce the global threat to U.S. security. Examples include the arrest and prosecution of Al Qaeda operatives including Yazid Sufaat in Malaysia and the Kenyan conviction of two Iranian terrorists for possessing explosives and related charges.

e) Reinforcing the whole of government approach. The RLA program has brought a special focus to targeting the legal framework as a means to propel reform, raised understanding about what the justice sector can contribute to policy development and programming in the Department, and advanced not only CT programming but appreciation for holistic USG interventions overseas, in line with US security sector assistance policy.

• Trusted relationships are the biggest factor in RLA and ILA program success. Resident assistance is the best vehicle for building relationships. RLAs stress that personal interaction is vitally important in the cultures where they operate.
• CT staff do not believe the Bureau is staffed robustly enough to perform the program management functions they believes the Bureau should be performing. This pertains largely to offering technical direction that supports the policy dimension, and to monitoring progress.

F. Recommendations
 

1. Pre-Placement Recruitment and Program Design

• The RLA orientation should ensure that RLAs receive as much information about working conditions at Post as possible, including whether support staff are in place or have to be hired, whether office space is shared, and what the process for acquiring computers, other office equipment, and working phone lines involves. While OPDAT notes that these topics are part of existing OPDAT pre-deployment training, and that they are primarily under the control of the Embassy, OPDAT may want to debrief with outgoing RLAs to determine how they can improve upon this segment of the orientation.

2. Establishing Individual Program Goals

• CT and OPDAT should consider reducing the number of goals in the work plans to one or two goals per component and they should also ensure there is at least one corresponding performance measure for each goal, and that performance measures in aggregate are manageable for the RLA or ILA to collect and report on. One way to do this is to break out measures representing large milestones into incremental steps (see Chapter II, Section B.2 on p. 19).
• The CT Bureau should give the draft rule of law strategy serious consideration, and otherwise resolve an apparent split in thinking about its strategic focus so staff are clear about what kind of strategic guidance to offer RLAs and ILAs.
• To improve communication over strategic decisions, OPDAT should routinely advise CT if it is poised to receive funding from other sources for country programs where CT funds the RLA.

3. Monitoring and Evaluating Program Progress

• OPDAT should consider adding an element to quarterly reports allowing for “quick uptake” of progress against the work plan, such as an activity progress table at the end of each quarterly report that provides a snapshot of progress toward work plan goals and objectives. An example based on UAE’s Q2 report for 2012 appears as Annex D.
• Within the existing format, quarterly reports could be shortened by focusing less on inputs and more on outputs and outcomes.
• OPDAT should consider incorporating weekly reporting into the work plan.
• OPDAT should ensure that the findings from training evaluation questionnaires are included in quarterly reporting. While according to OPDAT the use of post program evaluations is routine practice, neither quarterly reports nor AARs consistently capture how delivered training will help participants do their jobs better or change practices and procedures within trainees’ parent institutions.

4. Efficiency in Implementation/Management

• CT and OPDAT should both explicitly value the diplomatic role of RLAs. OPDAT already acknowledges that RLAs are “small d” diplomats, and currently discusses this in RLA orientation. More broadly, CT and OPDAT should consider referencing Embassy duties in work plans. The Pakistan work plan for FY12 and Kenya work plan for FY13 substantially accomplish this with references to Embassy activity in the background section. Such references could be made part of all work plans.

5. Effectiveness Across Programs

• CT should consider re-orienting RLA/ILA programs in the Gulf (based in UAE) to better align with TFWG collaborators in the field, maximize collaboration with the Gulf Cooperation Council, and address critical capacity gaps.
• CT informants offered suggestions for enhancing staffing in the Bureau: 1) hiring advisors with specialized backgrounds in police, prosecution, and prisons; and 2) hiring two additional program implementation and monitoring staff in CTF. The evaluator recommends that CT undertake a comprehensive staffing assessment as issues surrounding its strategic focus are resolved, which examines how to best offer technical direction, manage training providers, and monitor and evaluate programs.
• CT should consider establishing guidelines for operational effectiveness in RLA programs for the Bureau. Guidelines would help define success, set out steps to be taken when success is not being met, and identify the decision-makers for initiating each step.
• OPDAT and CT should consider using more RLAs, attachés, and other, field-based interagency representatives as training presenters in line with the relevant expertise among such individuals, and rely less on U.S.-based presenters who are not familiar with the cultures and the specifics of the host country legal systems. While this recommendation is not intended to deprive RLA programs of necessary and specialized expertise, OPDAT is encouraged to assess and utilize available on-ground expertise, on a case-by-case basis, as much as possible to reinforce programs’ applicability to local circumstances. To the extent U.S.-based presenters participate, they should be briefed in advance about the content of local laws and the structure of local institutions.