2015/Timor-Leste/U.S. Mission 1207 Program

Below is the Executive Summary. Click here for the full report (PDF).

EXECUTIVE SUMMARY

In 2006, Congress enacted Section 1207 of the National Defense Authorization Act, which authorized the Secretary of Defense to transfer up to $100 million to the Secretary of State to fund “whole-of-government” strategies and civilian agency–led activities that address reconstruction and stabilization risks which, if neglected, could negatively impact U.S. policy and security interests. The authority was renewed every year for additional funding, through fiscal year 2010. U.S. Department of State’s Bureau of Conflict and Stabilization Operations (DoS/CSO) is the chair of the interagency 1207 Committee, which reviews and approves the conflict-focused programs, which are selected annually through a competitive process. CSO also serves as the 1207 Secretariat, monitoring project implementation and adherence to program goals.

In 2010, the U.S. Embassy/Dili in Timor-Leste was awarded $11.32 million from State Department 1207 funds to “decrease risks of violence and instability in anticipation of the ongoing drawdown of the United Nations Mission in Timor-Leste (UNMIT) police monitoring presence by building capabilities of vetted law enforcement institutions and providing opportunities for at-risk Timorese youth.”[1] The program, which came to be known as “Supporting Police, Sustaining Peace” (SPSP), was designed to assist in mitigating the potential for violence and instability[2] as the UNMIT policing monitoring presence declined. The SPSP program was made up of seven component activities designed to address critical weaknesses within the National Police of Timor-Leste (PNTL), to support civil society in monitoring developments in the security sector, and to foster positive police-community relations. The program funded a program coordinator who also served as a conflict analyst for the Embassy.

EVALUATION PURPOSE AND EVALUATION QUESTIONS

The purpose of this performance evaluation was to assess all 1207-funded projects managed by the DoS Bureau for International Narcotics and Law Enforcement (INL) and the U.S. Agency for International Development (USAID)/Timor-Leste under the SPSP program in order to provide accountability and inform future programming of similar interventions. The program funded a program coordinator who also served as a conflict analyst for the Embassy.

The evaluation team collected evidence to identify the relevance of the original 1207 design and assess whether program outcomes were aligned with 1207 program objectives. The following evaluation questions were assessed: (1) Was the design of this 1207 program and its elements appropriate for the evolving context in Timor-Leste? 2) Were the activities as implemented consistent with the 1207 principles and funding authority? 3) What were some of the intended and unintended outcomes of the completed projects, both positive and negative? and 4) Is it likely that the results measured will be sustainable given the context of Timor-Leste?

EVALUATION DESIGN, METHODS, AND LIMITATIONS

The evaluation consisted of three phases: 1) a comprehensive desk review of relevant documents and DC-based interviews; 2) approximately two weeks of field data collection in Timor-Leste; and 3) analysis, report writing, and final presentations.

The primary methods of data collection were: 1) a document review of USAID- and Embassy-provided documents as well as secondary data sources such as relevant assessments, 1207 program documents, security sector surveys, and other donor studies; 2) 55 key informant interviews (KIIs) to explore key issues in depth with individual stakeholders using a combination of open-ended questions and closed-response (yes/no) or respondent rating (Likert or scale rating) questions to quickly quantify responses and identify response patterns; 3) four small group discussions (SGDs) with 45 PNTL officers and staff; 4) one focus group discussion (FGD) to assess changes and improvements in policing or security practices relating to gender considerations; and 5) a small sample quantitative knowledge, attitudes, and behavior (KAB) exercise with targeted beneficiaries of the SPSP program, specifically PNTL officers who had taken part in training programs and specific 1207 activities and interventions.

The SPSP program has operated in a complex post-conflict context in one of the newest countries in Southeast Asia. Timor-Leste is in the process of building new security, justice, and governance institutions and PNTL is still evolving and growing as a national security institution. Anticipating the Timor-Leste context, the team identified several important challenges that the evaluation approach needed to address. The resources and timeline available for the evaluation meant that the sample size for data collection was small and centered in the capital. Similarly, staff and respondents were not randomly selected and may have had significantly different perspectives from those who did not or could not be included in the evaluation. National perspectives between PNTL officers and civil society and donors were varied in their view of government capacity and security sector actors. Examples of mitigation strategies included utilization of multiple data points for each question, confirmation of data through triangulation and documentation review, customized data collection protocols, and staggered data collection timeframes to provide additional time to locate beneficiaries and other stakeholders who were no longer affiliated with the SPSP Program.

FINDINGS AND CONCLUSIONS

The evaluation findings, conclusions, and recommendations have been broken down by intervention component rather than by evaluation question. The table below briefly details the SPSP program and provides information about the component area objectives, a brief description drawn from the original 1207 proposal documentation, the United States Government (USG) agency responsible for their monitoring and management, and the implementing partner tasked with the implementation of the 1207-funded activity. This is followed by a summary of some of the key findings and conclusions developed by the evaluation team.

 

Component

USG Lead

Implementing Partner

Conflict Mitigation through Community Policing

USAID

The Asia Foundation
(TAF)

Objective: Strengthen Timorese police capacity in community policing.

Component description: Assistance to the PNTL to help implement its new policy of adopting community policing practices across the force. Putting this policy into practice requires educating both the police and the community regarding community police practices, establishing Community Police Councils throughout the country, and facilitating the initial work of these councils.

Accountability Strengthening in the PNTL

USAID

Technical Assistance
(Legal Advisor)

Objective: Strengthen civilian oversight of Timorese police in conduct and disciplinary procedures.

Component description: Training assistance to the PNTL to the Prosecutor General’s office to improve investigative practices and build stronger ties between the two institutions in the investigative process. Poor cooperation and substandard investigative practices impeded justice and were undermining the rule of law and security in Timor-Leste.

Investigations Training Program

DoS/ INL

Trafficking Investigations Program

IOM

--------------------

Training for Police Investigators

Creative Corrections

Objective: Strengthen Timorese police capacity in investigations.

Component description: Training assistance to the PNTL to the Prosecutor General’s office to improve investigative practices and build stronger ties between the two institutions in the investigative process. Poor cooperation and substandard investigative practices impeded justice and were undermining the rule of law and security in Timor-Leste.

Strengthening Maritime Borders

--------------------

Strengthening Land Borders

DoS/ INL

NCIS, Maritime Border

--------------------

UNODC, Land Border

Objective: Strengthen Timorese border and maritime policing capacity.

Component description: Specialized training and limited equipment purchases for PNTL Border Guard and Maritime Police units.

Logistical Support to PNTL

DoS/INL

Creative Corrections

Objective: Strengthen the logistical capacity of the Timorese police.

Component description: Technical assistance and advice to the PNTL on the procurement, storage, and maintenance of logistical and communications equipment and supplies.

Youth Engagement to Promote Stability

USAID

Search for Common Ground
(SFCG)

Objective: Reduce the likelihood of Timorese youth involvement in violence.

Component description: A comprehensive youth engagement program targeted at underemployed youth in Dili and the districts, many of whom are also active in martial arts groups, which were banned in Timor-Leste in 2010 due to their involvement in violence.

Civil Society Monitoring of the Security Sector

USAID

Belun
**sub-partner, Fundasaun Mahein

Objective: Strengthen the capacity of Timorese civil society to monitor the performance of the Timorese police and threats to stability in Timor-Leste.

Component description: Assistance to Timorese NGOs to support activities that seek to monitor development in the security sector, the performance of the PNTL, police-community relations, and general and emerging threats to stability. Project activities will be designed to provide independent and objective feedback and advice to the GoTL, PNTL, and the international donor community.

 

CONFLICT MITIGATION THROUGH COMMUNITY-ORIENTED POLICING (CMCOP)

CMCOP was well-researched and -planned, and adjusted to the changes in the political context as well as donor coordination needs. The implementer, The Asia Foundation (TAF), is the only agency involved in supporting Timor-Leste police reforms that works directly with communities, which places TAF in a unique position that has distinct advantages in addressing village-level security issues. The TAF approach to community policing emphasizes community and police interventions. This method is important in the Timor-Leste context, where the role of the police in providing human security is still forming and there is still relatively weak access to justice. The TAF program focuses primarily on community policing—the delivery of safety and security—and targets key elements of PNTL necessary to improve community policing practices in Timor-Leste. This is critical for USAID and reflects an emphasis on community and human security needs most relevant to development program objectives.

In examining costing issues as a measure of efficiency, CMCOP's approach in principally employing Timorese staff is both economical and enables deeper engagement with Timorese networks, including the PNTL and the Government of Timor-Leste (GoTL). USAID’s management and CMCOP’s monitoring and reporting regime are extremely effective and deserve to be recognized as a model within the 1207 framework of components. However, more data and information have been generated from analysis of community policing efforts than can be adequately captured and retained by senior- level decision makers both within the USG and the GoTL. This information must be analyzed and prioritized so that it is more accessible for policymakers working to improve human security in Timor-Leste.

ACCOUNTABILITY STRENGTHENING IN THE PNTL

A Senior Legal Advisor provided technical support and mentoring for the Office of Inspection and Audit (OIA) of the Ministry of Interior (MOI), formally the Secretary of State for Security (SoSS), the General Inspectorate Office of PNTL (PNTL/GIOO), and the Department of Justice of the PNTL. The Advisor worked extensively to assist staff within the MOI in refining and improving its systems for audits and inspections as a means to improve accountability across PNTL. As originally designed, one aspect of this component was to assist the MOI to revise and adopt new Disciplinary Regulations for the PNTL, but this proved to be challenging due to the political climate and changes in leadership within the MOI. Changes between the V and VI Constitutional Government in early 2015 as well as new legal procedures within the PNTL based on the transition to a new ministerial structure made changes in component targets necessary. This was not a design flaw from the original 1207 proposal but rather demonstrated the need for moderation in component design and in identifying ambitious targets, such as revising disciplinary regulations in a new and developing security institution such as PNTL.

PNTL faces a number of challenges in implementing the current regulations and exercising more accountable oversight functions for its commanders and officers. These challenges range from limited awareness and support for the current disciplinary system to limited experience among staff in handling disciplinary issues or conducting performance monitoring, as well as adopting a standard approach to disciplinary training for senior PNTL staff and officials. In addition, some questions remain concerning how the current disciplinary system is aligned with the organic laws for the PNTL.

INVESTIGATIONS TRAINING

The original 1207 proposal design of the Investigations Training component specified that the Police Training Advisor was to support, assist, and advise the Timor-Leste Police Development Program (TLPDP) in developing country-specific investigation training packages for two advanced levels. Evidence from fieldwork and document review suggests that the program was generally successful; however, the original design of the project did not consider that the three-tiered investigations training model was generally too advanced for PNTL trainees. Moreover, the Portuguese police-training model already in use at the training academy differed greatly from the U.S. and Australian models that were used for the investigations training in the TLPDP. However, the Advisor was able to quickly recalibrate the program to meet the needs of PNTL officers and created an accredited training curriculum in conjunction with the Australian police.

There was some difficulty in working with a nascent institution such as the PNTL. Evidence suggests that the PNTL’s investigative capacity is not up to international standards, and there is limited use of technology, or even basic evidence-storage facilities, to conduct high quality, modern investigations. The PNTL has been afforded numerous training opportunities over the past ten years from a variety of sources, including from Australia, the United States, Portugal, Indonesia, Malaysia, and the Philippines. This training has varied greatly and has not necessarily been coordinated or adapted to the local Timorese context, making it difficult for a national training approach to be adopted by PNTL.

STRENGTHENING LAND AND SEA BORDERS, MARINE PATROL UNIT (MPU)

The program design phase for this component did not include INL or potential partners. It was challenging to find suitable partners willing to take on this project. The Naval Criminal Investigative Service (NCIS) did not have a long-term presence in Timor-Leste and was not well versed in the operating environment or what would be appropriate in terms of program design and implementation. The design of the Marine Patrol Unit (MPU) training was intended to be highly practical and therefore was contingent on the possession of functioning equipment. The design did not sufficiently take into account the limited infrastructure and the effect that this would have on training protocols. Remote management also proved to be challenging, especially for a program largely focused on training and capacity development. It would have been useful for an international advisor to have regular, ongoing interactions with MPU staff to ensure utilization of training and to address any issues as they arise. Formal training was often not enough for capacity development, as ongoing mentoring and support, in conjunction with training activities, were necessary to contribute to sustainability.

STRENGTHENING LAND AND SEA BORDERS, BORDER PATROL UNIT (BPU)

According to Border Patrol Unit (BPU) trainees, PNTL senior staff, and USG sources interviewed, the United Nations Office of Drugs and Crime (UNODC) program applied regional and global training models that were not adapted specifically for the realities of border protection needs in Timor-Leste. The two-day training needs assessment conducted by UNODC was not comprehensive enough to adequately inform design, and the proposal written by the UNODC Bangkok office may not have been as consultative as necessary. The trainers were experts in their subject matter, and the trainings were of a high quality but were not sufficiently adapted to the current operational modality of the BPU. UNODC needed to better ascertain the needs of the BPU and adapt training accordingly. In addition, it was unclear whether UNODC as an implementing partner fully understood the national and regional stakeholders working on border protection issues in Timor-Leste.

The program design and management efforts for this component also needed to ensure that implementing partners had fully assessed the border protection stakeholders in Timor-Leste before embarking on training. Border management is a collaborative effort that should ideally involve multiple agencies, all of which should be adequately trained. However, the other agencies along the border (Customs, Immigration, and Quarantine) were not adequately incorporated into this UNODC training and have not received significant training to sufficiently handle complex border issues. Despite participating in numerous training activities and acquiring much-needed equipment, it is unlikely that the BPU can continue to grow and develop without appropriate external assistance. As one of the newer branches of PNTL, BPU has a particularly limited budget. Without sufficient budgetary resources, BPU can provide neither ongoing, high-quality training to staff nor the capacity to implement acquired skills.

POLICE LOGISTICS

While it may have been logical to include an element within the 1207 component for logistics support, both trainee recipients and technical advisors working with the USG on security sector reform (SSR) programs stated that the INL/Third Party Contractors program was not adapted specifically for the realities of working in Timor-Leste. Respondents stated that there was not a strong understanding of the needs of the Logistics Department before entering into this program. For example, after becoming acquainted with the Logistics Department, the Advisor determined that it would be more useful to focus on systems development, particularly the development and implementation of a comprehensive asset management system. This approach was not specifically conveyed in the original Statement of Work; however, it was an appropriate approach in that mentorship and capacity development of key staff within the Logistics Department was challenging in light of systemic issues that needed to be addressed (e.g., development of an asset management system). Currently, the PNTL uses Excel files to track assets, and PNTL key users cannot accurately ascertain, for example, the total number of cars in the fleet and their condition. This lack of data affects all departments across the PNTL as it is often difficult for individual units to track assets, and corruption concerns are evident due to weak management of assets within the institution. There is still work to be done within the Logistics Department that could not be accomplished during the limited programmatic period of performance. Logistics is an important cross-cutting department that interacts with and affects all other departments of the PNTL. The U.S. Mission and the Logistics Advisor have made slow but steady progress in this area. While there was a lack of support to liaise with higher-level PNTL officials when the Logistics Advisor first began, this has improved recently. As a result, there is increased buy-in from PNTL leadership, which is helping to generate increased cooperation of key actors in the Logistics Department. While logistics is an area likely to meet significant resistance and challenges, the work is essential to successful continuation of outcomes in other areas.

YOUTH ENGAGEMENT TO PROMOTE STABILITY (YEPS)

The YEPS program benefited from the fact that its activities were a continuation of a well-established youth program already implemented by Search for Common Ground (SFCG). SFCG has strong national relationships and is a key player in youth advocacy and politics in Timor-Leste. SFCG acted on lessons learned from their previous program (YR4PB) and national youth policy planning priorities, and appears to be one of the strongest of the 1207 components in terms of gender integration. SFCG emphasized gender-sensitive programming and design, which were addressed at many levels including program management, monitoring and evaluation (M&E) systems, and in staffing and recruitment of SFCG staff, YEPS participants, and partners.

YEPS may not have been a suitable choice for 1207 funding. Despite its stabilization intent and the specific contextual justifications, there were still gaps in targeting. Accessing at-risk youth is a challenge in Timor-Leste, but this program would have benefited from more analysis of youth engagement in such activities as martial arts gangs and better coordination with PNTL, particularly the National Community Policing Unit. It would be beneficial to understand how to best target at-risk youth in the Timorese context and use this to inform strategy and project design for future youth programs.

CIVIL SOCIETY MONITORING OF SECURITY SECTOR DEVELOPMENT (CSM-SSD)

This component was developed based on the need for an independent source of monitoring for PNTL activities and incidents at the local level, specifically after UNMIT departed in 2012. According to most sources, it was a well-designed component of the 1207 SPSP program. This component was implemented by Belun, a national non-governmental organization (NGO), and largely focused on expanding access to information and reporting on human security and community security issues for government institutions and civil society actors working on SSR. It was developed from an existing USAID-funded program and was implemented entirely by national institutions with noted success and efficiency. This component is in line with USAID Forward Policy and is also a legacy program, as USAID has supported Belun since 2003.

Of the 1207 components evaluated, CSM-SSD works with the most ministries and national stakeholders. Despite this reach, Belun has struggled for years to establish memorandums of understanding (MOUs) with PNTL and other ministries. Most sources felt that civil society security monitoring and independent incident reporting were important for national planning and policy, particularly the early response, early warning function. Ministries seem to increasingly recognize the value of the trend analysis and the independent reporting of SSR issues that Belun and partners provide.

SELECTED RECOMMENDATIONS

Conflict Mitigation through Community-Oriented Policing

  • There is a need to refine the approach to community policing in Timor-Leste, specifically to consider the costing issues associated with specific interventions and ensure that donor efforts are in close alignment with the institutional needs of PNTL writ large, as well as current organic law, procedures, and policies.
  • Donors should promote a stronger understanding of community policing within the Timorese parliament and government, where communication about the program is critical to national planning and budgeting.

Accountability Strengthening in the PNTL

  • Work with other donors and PNTL to finalize the existing draft of the Disciplinary Procedure Manual to be used by both PNTL and OIA and assist senior officials in establishing technical procedures for the proposed PNTL Disciplinary Law and Regulations.
  • Support PNTL in developing a reporting and infraction system that is in alignment with Civil Service 286/296 penal code. PNTL officers that commit crimes as public officials are a part of the system of criminal infractions and enforcement is needed to minimize the potential for corruption within PNTL.

Investigations Training

  • Ensure that training content and methodologies are in line with the local context and culture. Take into consideration that the current technical capacity of PNTL for investigations and evidence protection is still developing. Remain flexible during implementation to ensure that all donors working on training cooperate to support cohesion for PNTL staff working in the PNTL Training Center.
  • Ensure that training Advisors are embedded within well-established institutions such as within PNTL or the PNTL Training Center. This will ensure more effective donor coordination on shared training efforts, minimize confusion within PNTL, and ensure that training is effective, well-targeted, and cost-effective for the USG and the GoTL.

Strengthening Land and Sea Borders (MPU)

  • As Timor-Leste moves toward ASEAN membership, it is critical that the MPU have a strategy and plan for addressing and prioritizing marine protection issues. There needs to be a comprehensive approach to prioritizing marine protection needs within PNTL and a specific strategy and plan for addressing funding needs at the national level.
  • The MPU requires a training approach that is catered to the context and resources currently available within PNTL. A maritime police unit requires functioning boats and the internal capacity to repair and maintain boats. Any training provided should take into account the possession or lack thereof of functioning equipment—as well as an ability to properly procure parts and conduct repairs. MPU also needs to more realistically assess its marine protection needs to assure more realistic asset management and resource requests.
  • Remote management as an Advisor for training activities is not the ideal model. It would be more effective for an international staff member to be based in Dili to provide ongoing support and mentorship.

Strengthening Land and Sea Borders (BPU)

  • If global or regional standards for border protection training are provided in low-capacity environments such as the PNTL in Timor-Leste, a comprehensive training needs assessment (TNA) needs to be conducted that takes into account the specific roles and functions of the BPU. This TNA should be targeted at the BPU and include information about coordinating with other entities also responsible for border protection issues in Timor-Leste.
  • Coordinate training among all four agencies responsible for securing the official border crossings. BPU is only one part of the equation. All agencies along the border as well as prosecutors involved in border crimes require the same basic knowledge as BPU in order to perform their jobs sufficiently.

Police Logistics

  • A mentoring approach to capacity development is useful in this context. It is critical that all future efforts include both ongoing mentoring and specific technical training to ensure that logistics staff are able to manage and monitor the systems developed for the PNTL Logistics Department.
  • Developing a comprehensive asset management system, ideally building upon what is in existence at the government, should take priority, as this is an essential tool for PNTL as an institution and is critical for operations.

Youth Engagement to Promote Stability

  • Pilot a feasibility study on how to increase the capacity of national ministries and local organizations to reach at-risk youth in Timor-Leste. Work with national partners to develop youth strategies to address this evolving and pressing need. This analysis will need to include the added pressures of rural-to-urban youth migration that is likely to continue into the future in Timor-Leste.

Civil Society Monitoring of Security Sector Development

  • Establish a CSM Secretariat role in the Prime Minister’s office as a way to increase the value and usefulness of this type of security sector reporting. It would be best if this were made up of a consortium of national and local organizations where the lead secretariat role could rotate between groups. This function would allow for synthesized analysis and targeted recommendations to be shared and integrated into national decision-making and SSR reform needs, particularly through the annual planning process.
  • This is a legacy program for USAID, and Belun has the potential to share what it has learned to help others develop more effective early warning, conflict-prevention response systems and civil society oversight of SSD. Assess the potential for elements of the Early Warning Early Response (EWER) system to be integrated into the protocols of the Association of Southeast Asian Nations (ASEAN) or other regional bodies to enhance and learn from conflict-prevention and response systems.

--------------------

Notes:

1. Taken from Congressional Notification Report, Timor-Leste.
2. Original 1207 Program Proposal, Timor-Leste.