2013/Bangladesh/The Anti-Terrorism Assistance Program
This publication was produced at the request of the United States Department of State. It was prepared independently by Development & Training Services, Inc. (dTS)
This report presents the findings of the performance evaluation of the Anti-terrorism Assistance (ATA) program in Bangladesh. The Bureau of Counterterrorism (CT) is conducting evaluations as part of its results-based management approach to programming and the Bangladesh evaluation is the second it has commissioned.
B. Purpose of the Evaluation
This evaluation’s purpose is to study and appraise the effectiveness of training which ATA has conducted for Bangladesh from January 1, 2008, to August 1, 2013. The major research questions addressed were:
- Were new skills, knowledge and capacities developed as a result of participation in the training?
- Have the new skills, knowledge and capacities been applied, and if so, in what way(s)?
- What individual and organizational changes resulted from these applications?
Institutionalization and Sustainability
- How have ATA interventions (training and occasionally equipment) enhanced the Government of Bangladesh’s (GOB) capacity to assess counterterrorism needs as well as to plan and carry out appropriate actions?
- How has the GOB expressed political will and allocated resources to institutionalize ATA training?
- To what extent has ATA helped the GOB make its increased counterterrorism capacities sustainable?
- To what extent is the ATA program progressing toward its FY 2013 SOs for Bangladesh, and are those objectives measureable and realistic?
- How much progress did the program make toward the FY 2012 SOs and in years earlier?
Integration with Other Counterterrorism Efforts
- How well has ATA-Bangladesh integrated with other counterterrorism programs in the country and with U.S. Embassy/Dhaka’s counterterrorism strategy?
C. Evaluation Methodology
Two evaluators, Dr. Frederick C. Huxley (Team Leader) and Mr. Jeffery Lusk (Law Enforcement Specialist), conducted the evaluation with important support from dTS’ Ms. Julia Czaplinski (Research Associate) and Ms. Janet Kerley (Project Director). The theoretical framework used to structure the program effectiveness part of the evaluation is the Kirkpatrick Model, which focuses on behavioral change and assesses four outcomes of training:
1. Overall satisfaction of the participants with the ATA training course;
2. Extent to which each participant learned the content of the various training objectives (curriculum), with special attention to the critical capabilities that were the foci of the courses;
3. Application/use of the new skill(s) and knowledge learned; and
4. Extent to which the application of training led to behavioral and/or institutional change(s).
The evaluation was conducted from August to October 2013. The evaluators carried out a desk review of program documentation in August, interviewed DOS officers in the Washington, DC metro area on August 14 and 15, and travelled to Bangladesh for fieldwork from August 31 to September 13.
The data sources were:
- An anonymous survey, with a 13% response rate (153 responses from 1,032 individuals trained), while not a random sample, it was broadly representative of the courses provided;
- Five individual or group interviews with DOS officials in Washington, DC;
- A telephone interview with the coordinator of the Regional Strategic Initiative;
- Eight individual or group interviews with US Embassy officials in Dhaka;
- 25 guided group discussions or key-informant interviews with Bangladeshi law enforcement officials in the Dhaka metro area;
- One group interview with the foreign director of the multilateral National Academy for Security Training;
- Two group interviews with Bangladeshi civil-society organizations (CSOs);
- One group interview with law enforcement officials from the UN, the EU, and foreign embassies in Bangladesh; and
- Six visits to training schools, laboratories, or operational units in the Dhaka metro area.
D. Project Design and Activities
ATA is the U.S. government’s (USG) leading counterterrorism capacity-building program for law enforcement agencies of selected partner nations. ATA’s primary goal is to enhance the host country’s ability to detect, deter, and apprehend terrorists, and to respond to terrorist incidents. ATA encourages respect for human rights and builds productive bilateral counterterrorism relationships with the United States. The ATA program meets its mandate by providing training, related equipment, and follow-on mentoring.
Over the past five and two-thirds years (the period covered by this evaluation) the ATA program presented 52 training courses (37 different courses and 15 repeats) in Bangladesh to 1,053 individuals from 11 different law enforcement organizations (Figure 1). While all of the organizations are under the authority of the Ministry of Home Affairs, their areas of responsibility and action are somewhat overlapping, with new organizations created frequently and transfers of personnel common from one to another (or even outside, especially to the military). The ATA program in Bangladesh itself changed during the period under study—for example, two FY 2012 SOs were narrowed in focus, and two others dropped, to establish the FY 2013 SOs.
Figure 1: Number of Different Courses by Focus
Critical Incident Response I (CRT I)
Critical Incident Response II (CRT II)
Border Security I (BoS I )
Border Security II/ Critical Incident Response II (BoS II/CIR II)
Investigative Competence (INV)
Protecting National Leadership (PNL)
Institutional Development (INST)
Contributing to all of the above SOs
E. Evaluation Findings
The evaluation findings indicate that progress toward ATA-Bangladesh program objectives is mostly on track, and that the changes in SOs seem a step in the right direction. The main exception to this pattern concerns integration with other CT efforts at the post: currently little such integration has occurred. While the report details the findings for all of the evaluation questions, the key findings and recommendations are found below.
Findings on Program Effectiveness
Nearly all trainees reported learning new skills and using the training, as evidenced by the following points:
- Nine-in-ten of the questionnaire respondents said that they were satisfied (81% “very” and 11% “somewhat”) with the training they received (Figure 2).
- Nine-in-ten (91%) reported they had learned new practical skills for their current jobs.
- Pre/post scores of knowledge increase after training were available for 42 percent (22/52) of the courses, and they showed an average increase of 33 points.
- Eight-in-ten respondents said they were using “nearly all” (39%) or “a great deal” (43%) of the training in their current jobs.
Trainees also reported changes on the individual level that resulted from the training.
- Seven-in-ten (71%) had seen their responsibilities increase after the instruction.
- Half of the trainees had been reassigned to new jobs following the courses.
Both trainees and supervisors said changes in their organizations had resulted from the training.
- CID affirms that, as a result of ATA training, most of the evidence it now presents to prosecutors for legal action comes from investigations by trained personnel; before the training it came mostly from oral testimony and confessions.
Findings on Institutionalization and Sustainability of the Training
- The Special Security Force (SSF), responsible for protecting national leadership and foreign dignitaries, no longer requires ATA training since it has institutionalized ATA techniques and methods. SSF now even trains other units (such as the Special Branch and the Special Protection Battalion) on how to perform this task better.
- Law enforcement agencies in Bangladesh are now de facto stratified in terms of their levels of effectiveness, institutionalization, and sustainability, and ATA training has helped produce that outcome. The Train-the-Trainer (TTT) methodology is widespread and often successful, though its use at the Police Staff College is restricted importantly.
Findings on Progress on Strategic Objectives in FY 2012 and FY 2013
- FY 2013 SOs are measureable both subjectively and objectively, and trainees perceive more progress on Critical Incident Response than on Border Security.
- For FY 2012 SOs, good progress was made on building law enforcement capacities, less on protecting the national leadership, and results were mixed about collaboration on borders and regional collaboration.
- Narrowing the SOs in FY 2013 will likely lead to improved progress and may enable a more considered process for setting SO time frames in the future.
Findings on Integration with Other Counterterrorism Efforts
- Currently there is little integration of ATA programs with other CT Bureau or interagency counterterrorism efforts in Bangladesh, but the potential exists—especially with NAST, CVE, and DOD efforts in the future.
- ATA now has the Kirkpatrick Model for understanding how its training works to help build capacities for counterterrorism at individual and organizational levels. It should continue to address the complexities of working in Bangladesh by using that model, sensitively but resiliently, and by building on it to improve capacities of the law enforcement system as a whole. The recommendations following will illustrate how this might be done.
- As with the ATA program in Morocco, the program in Bangladesh needs to change its record-keeping systems to focus on capturing the individuals actually trained, not those enrolled. This change will enable the program to document who learned what, in which course, when, how, and how much.
- The program needs to ensure that instructors actually do conduct pre/post tests of trainee learning for each course taught. This standard and objective educational measure complements more subjective measures (e.g., participants’ self-reported responses to questionnaires) and contribute to an internal quality-control process.
Institutionalization and Sustainability
- ATA should improve efforts to help Bangladeshi law enforcement units institutionalize results of the training they receive by first distinguishing between duplicative and integrative processes of institutionalization.
- Then, keeping in mind both the units and the law enforcement system of which they are parts, ATA can help commanders decide which blend of the two processes works best for a particular unit as well as for the whole system. The effort to build and institutionalize cyber-capacities can be an opportunity to illustrate this course of action.
- ATA may improve the FY 2013 Strategic Objectives for Bangladesh over time by using them again in FY 2014 and then monitoring results of the training to learn how much and how fast trainees upgrade their capacities for Critical Incident Response (CIR) and Border Security (BoS). Then it will have two full years of data—not just “first fruits”—to help set factually-grounded boundaries of the time frames for those SOs thereafter.
- Initial results in FY 2013 for both CIR and BoS indicated that Bangladeshi capacities for the former were developing well, while those for the latter were lagging. This suggests that ATA should reexamine the content of the BoS courses, or the manner in which they were taught, or other factors to understand the reason(s) behind this difference and how it can be addressed.
Integration with other Counterterrorism Efforts
- ATA should develop its integration with NAST and CVE as those projects come on line, and it should explore collaboration with DOD regarding the 2016 test.
- The program could follow up the issue of cost-effectiveness (raised at the debriefing) to show that its model for connecting training to capacity-building on individual, organizational, and systemic levels is more cost-effective than may appear.