2013/Malaysia/Gender-Based Violence Prevention Programs with Refugees

July 26, 2013
This publication was produced at the request of the United States Department of State. It was prepared independently by Erica Holzaepfel and Diane Paul through Social Impact, Inc.

IDIQ Contract Number: S-AQM-MA-12-D-0086
Technical and Advisory Services for Program Evaluation Requirements

Task Order Number: S-AQMMA-13-F0964

DISCLAIMER
The authors' views expressed in this publication do not necessarily reflect the views of the United States Department of State or the United States Government.

CONTENTS

Acronyms

Executive Summary

Evaluation Purpose
Evaluation Questions
Program Background
Evaluation Design, Methods, and Limitations
Evidence and Findings
Recommendations

Evaluation Purpose and Questions

Evaluation Purpose
Evaluation Questions

Program Background

Malaysia Country Background
Program Response

Evaluation Design, Methods, and Limitations

Evaluation Design and Data Collection Methods
Limitations

Evidence and Findings

Evaluation Question 1
Evaluation Question 2
Evaluation Question 3
Evaluation Question 4
Evaluation Question 5
Evaluation Question 6
Evaluation Question 8
Evaluation Question 9

Conclusions and Recommendations

Evaluation Question 1
Evaluation Question 2
Evaluation Question 3
Evaluation Question 4
Evaluation Question 5
Evaluation Question 6
Evaluation Question 8
Evaluation Question 9

Annexes

Annex I: Evaluation Statement of Work
Annex II: Data Collection Instruments
Annex III: Evaluation Contacts and Key Informants
Annex IV: GBV Prevention Indicator Compendium
Annex V: Disclosure of Conflict of Interest

ACRONYMS

ACTS - Kumpulan A Call To Service
AWAM - All Women’s Action Society
CBT - Cognitive Behavior Therapy
CEDAW - Convention on the Elimination of All Forms of Discrimination Against Women
CELC - Community Empowerment and Livelihood Center
CHW - Community Health Worker
CRC - Convention on the Rights of the Child
DoS - U.S. Department of State
FGD - Focus Group Discussion FY Fiscal Year
GBV - Gender-Based Violence
GBVIMS - Gender Based Violence Information Management System
HEI - Health Equity Initiatives
IASC - Inter-Agency Standing Committee
ICMC - International Catholic Migration Commission
ILO - International Labor Organization
IOM - International Organization for Migration
IPT - Interpersonal Therapy
IRC - International Rescue Committee
KII - Key Informant Interview
LGBTI - Lesbian, gay, bisexual, transgender, intersex
M&E - Monitoring and Evaluation
MOU - Memorandum of Understanding
NGO - Non-governmental organization
PKGS - Pusat Kebijikan Good Shepard
PRM - Bureau of Population, Refugees, and Migration
PTSD - Post-Traumatic Stress Disorder
RBM - Results-based Management
RWPC - Refugee Women’s Protection Corps
SCAN - Suspected Child Abuse and Neglect
SMART - Specific, Measurable, Achievable, Realistic, and Time-bound
UN - United Nations
UNHCR - United Nations High Commissioner for Refugees
USD - United States Dollar
WAO - Women’s Aid Organization

EXECUTIVE SUMMARY

Evaluation Purpose

This performance evaluation examines the effectiveness of gender-based violence (GBV) prevention programming funded in Malaysia by the U.S. Department of State Bureau of Population, Refugees, and Migration (DoS/PRM) during fiscal years 2010-2012 (October 1, 2009 – September 30, 2012). Fieldwork conducted as part of this evaluation contributes to a one-year evaluation of GBV prevention programming supported directly by PRM or indirectly by its partner organization, the United Nations High Commissioner for Refugees (UNHCR). The purposes of the evaluation are as follows:

Assess the effectiveness of GBV prevention programming for individuals and communities at risk;
Identify appropriate indicators for measuring the effectiveness of GBV prevention interventions in refugee settings; and
Characterize best practices and lessons learned in engaging men and boys in GBV prevention and response interventions in refugee settings.

The evaluation will provide DoS/PRM, multilateral organizations such as UNHCR, and non-governmental organization (NGO) implementers with guidance about conducting priority GBV prevention initiatives; monitoring and evaluating field-based GBV prevention programs; and engaging host country, international, and local NGOs in best practices for GBV prevention.

Evaluation Questions

The evaluation seeks to answer the following questions:

1. Did partners achieve the program activities defined in their project proposals? What were the barriers and facilitators to implementing program activities?

2. Were the objectives of the program based on evidence such as needs assessments or other forms of data? Were they realistic, measureable objectives? If not, how can the objectives be improved?

3. Did the GBV programming conform to internationally accepted GBV guidelines produced by the humanitarian community?

4. Are the indicators produced by the humanitarian community for GBV programming appropriate for measuring the outcomes of PRM-funded GBV prevention programs? Are the indicators in the project proposals specific, measurable, achievable, realistic, or time-bound (SMART)? How can proposal indicators be improved? Do indicators from the GBV guidance documents effectively capture the impact of GBV prevention programs? Are some more useful than others and for what reasons?

5. Were there any unexpected negative or positive consequences of PRM-funded GBV programs? Did organizations address negative consequences and how?

6. What factors explain intended and unintended negative or positive consequences?

7. What outcomes did GBV awareness campaigns achieve? Are the indicators for these programs SMART? How can indicators be improved for GBV awareness campaigns?

8. To what extent have men and boys been included in GBV prevention programs? If they were not included, why was this? If they were, what was the impact and how was it measured?

9. What were the short- and long-term outcomes of PRM-funded GBV prevention programs?

Program Background

Malaysia depends heavily on the cheap and readily available migrant labor force from the Asian region. The demand for labor has resulted in unwanted, irregular movements of economic migrants. Malaysia’s relative political and economic stability has also attracted refugees and asylum-seekers who are fleeing conflict and persecution. Low-skilled migrant workers have a marginal and semi-marginal existence in Malaysia. However, policies toward irregular migrants and refugees, tempered by issues of ethnicity and racism, are less tolerant.[1] Malaysia makes no distinction between refugees and undocumented migrants. There are an estimated 4 million migrants in the country, including 2 million who are undocumented and considered illegal. In January 2013, there were 90,185 registered refugees and 11,650 asylum seekers; the total population of concern to UNHCR was slightly less than 222,000. There are large numbers of undocumented, urban refugees—predominantly from Burma, but also from other countries throughout Asia. Refugees are subject to arrest, detention, physical punishment, and deportation. In the past several years, a decrease in arrests of documented refugees and asylum-seekers has been observed; however, large-scale arrests and detention of unregistered asylum-seekers persist.[2]

DoS/PRM’s mission is to provide protection, ease suffering, and resolve the plight of persecuted and uprooted people around the world. The work of NGO implementing partners is instrumental to ensuring that PRM achieves its humanitarian objectives and fulfills its mandate. PRM directly funds NGO programs designed to fill critical gaps, such as GBV prevention in humanitarian assistance and protection. PRM programming goals in Malaysia include the primary prevention of GBV. Primary prevention aims to prevent violence before it happens.

In Malaysia, the evaluation team examined programs implemented by three NGOs with PRM funding. Two of the programs, implemented by International Catholic Migration Commission (ICMC) and Women’s Aid Organization (WAO), had explicit objectives for the prevention of GBV. The third NGO, Health Equity Initiatives (HEI), provides medical and mental health services that indirectly treat GBV survivors as well as refers survivors to the appropriate GBV-specific treatment and prevention services.

Health Equity Initiatives (HEI): HEI is a local organization based in Kuala Lumpur with the objective of improving mental health and medical services for Burmese and other refugees and asylum-seekers, with an emphasis on victims of torture, forced labor, human trafficking, and exploitation. To achieve this objective, HEI builds the capacity of refugees to work within their communities on mental health issues; reinforces referral and case management systems in conjunction with UNHCR, other NGO implementers, and medical facilities; and conducts mental health outreach and screening.

International Catholic Migration Commission (ICMC): The mission of ICMC is to reduce the risk of GBV among refugee women and children in Kuala Lumpur’s Klang Valley and surrounding areas and to improve access to emergency support services for survivors. ICMC works in collaboration with UNHCR and four local NGOs to meet its objectives, and it coordinates with other organizations to identify and assist GBV survivors. ICMC and its partners provide emergency support services for GBV survivors, including emergency shelter and auxiliary services such as interpretation, transportation, and psychosocial support. ICMC and partners also raise awareness about GBV-related issues and promote community involvement in preventing and responding to GBV.

Women’s Aid Organization (WAO): WAO is a local organization based in Kuala Lumpur that provides mental health services for refugees and asylum-seekers and is under contract with UNHCR to provide counseling for GBV survivors on the UNHCR compound. It also maintains a shelter to protect women who are in danger. ICMC has a Memorandum of Understanding (MOU) with WAO to provide PRM- funded shelter and mental health services to survivors referred by ICMC.

Evaluation Design, Methods, and Limitations

This performance evaluation employed the standard rapid appraisal methods of document review, key informant interviews (KIIs), focus group discussions (FGDs), site visits, and direct observation of program activities. The Malaysia performance evaluation complements, and builds upon, findings from the Desk Review Report submitted to DoS/PRM in July 2013 by providing primary information on best practices, lessons learned, and directions for future programming, support, and PRM engagement. The evaluation team identified the following five categories of target groups as data sources for the field evaluation:

Donor/U.S. Government Partners: DoS/PRM/Washington, DoS/PRM/Bangkok, DoS/Kuala Lumpur
NGO Implementers: International Catholic Migration Commission (ICMC), Refugee Women’s Protection Corps (RWPC), Health Equity Initiatives (HEI), Community Health Workers (CHWs), Women’s Aid Organization (WAO)
Sub-Grantees: Community Empowerment and Livelihood Center (CELC), Dai Community Organization FOCUS Project
Beneficiaries/Program Participants: Refugee community members, refugee leaders, teachers, students, GBV survivors
External Actors: International Organization for Migration (IOM), Kumpulan A Call To Service (ACTS), Pink Triangle Association

The main limitation that the evaluation team faced was the applicability of the PRM-required evaluation questions to the field evaluation. The team found several of the evaluation questions were best suited to yielding substantive findings within the context of the desk review, while others were more appropriate for, and necessitated, the primary data collection activities associated with the field evaluation.

Evidence and Findings

Evaluation Question 1: Did partners achieve the program activities defined in their project proposals? What were the barriers and facilitators to implementing program activities?

Part I: Achievement of program activities as defined in project proposals

Overall, the evaluation team found that PRM-funded NGO implementers successfully carried out the majority of proposed program activities, but the majority of activities involved responsive rather than preventive action and reflected a lack of understanding of GBV prevention-focused programming. Despite the fact that the programs evaluated were designed to respond to rather than prevent GBV, certain activities do contribute to the prevention of GBV. For example, the provision of counseling to children, such as ICMC’s “Keeping Me Safe” training, is important in preventing future violence against those children.

Part II: Barriers and facilitators to implementing program activities

KIIs with community members, beneficiaries, teachers, and students revealed that despite interviewees’ difficulty with describing different types of GBV, there was generally a strong understanding of domestic violence and its causes. Refugees were aware of most of the domestic violence services provided by NGO implementers. There was also clarity regarding the fact that men and boys can experience GBV, including child sexual abuse and rape. The number of reported domestic violence cases increased, suggesting that UNHCR- and PRM-funded programs, particularly ICMC, are identifying more cases and are better meeting the needs of domestic violence survivors and their children.

Evaluation Question 2: Were the objectives of the program based on evidence such as needs assessments or other forms of data? Were they realistic, measurable objectives? If not, how can the objectives be improved?

The design of program objectives and activities was generally informed by one-off surveys and assessments; however, many of those assessments relied upon secondary data and were not designed or carried out by the NGO implementers themselves. None of the implementers had conducted baseline surveys or needs assessments among their target beneficiaries, nor had they determined the incidence or prevalence of GBV within the refugee community, the types of GBV experienced by refugees, the places and persons at highest risk of GBV (risk mapping), or the likely perpetrators. There is a total lack of information about the level of GBV perpetrated by the police and other authorities, GBV committed by employers, the nature and extent of GBV among refugees and asylum-seekers in the forced labor market, the number of refugees and asylum-seekers forced by circumstances into survival sex, and the extent of needs and/or challenges faced by lesbian, gay, bisexual, transgender, intersex (LGBTI) refugees or asylum-seekers.

Evaluation Question 3: Did the GBV programming conform to internationally accepted GBV guidelines produced by the humanitarian community?

Few respondents, with the exception of ICMC, could identify any of the international guidelines by name or provide a precise description of guidelines. However, they were generally aware that guidelines exist. ICMC recalled the Gender Based Violence Information Management System (GBVIMS)[3] and said that it uses UNHCR guidelines for the sub-grantees for standard operating procedures.[4] WAO has its own protocols and develops its own modules for peer counseling training but has not used any international GBV guidelines. WAO does, however, refer to the Convention on the Elimination of All Forms of Discrimination Against Women (CEDAW), International Labor Organization (ILO) Convention, Convention on the Rights of the Child (CRC), and some regional instruments as guidance and advocacy tools.

Evaluation Question 4: Are the indicators produced by the humanitarian community for GBV programming appropriate for measuring the outcomes of PRM-funded GBV prevention programs? Are the indicators in the project proposals SMART? How can proposal indicators be improved? Do indicators from the GBV guidance documents effectively capture the impact of GBV prevention programs? Are some more useful than others and for what reasons?

Indicators used by NGO implementers to monitor program performance were weak (as noted in the Desk Review Report). Specifically, indicators were poorly designed and often included targets. Indicators should be neutral gauges of progress that can be compared against an objective or target. When used appropriately, targets can orient NGO implementers to tasks that need to be accomplished and provide guidance for monitoring whether or not program progress is being made on schedule and if results have been achieved over time. Generally speaking, implementers were confused about the difference between indicators and targets.

Evaluation Question 5: Were there any unexpected negative or positive consequences of PRM- funded GBV programs? Did organizations address negative consequences and how?

Few respondents were able to confirm unexpected consequences of GBV programs; however, some key informants made conjectures about unexpected consequences that might be happening and identified others that could possibly occur in the future. NGO implementers noted potential unexpected negative consequences in relation to shelters and livelihoods programs. The isolation of, and/or the feeling of isolation among, women in shelters may have unexpected negative consequences. DoS/PRM/Bangkok reported that it was, for the most part, unaware of negative consequences of the programs but said it was possible that some husbands may react negatively to their wives’ assertion of rights. A male refugee community leader explained that UNHCR’s refugee status determinations can cause domestic violence, often due to misunderstandings that become very difficult to clear up due to translation problems at UNHCR.

While few of the key informants had much to offer about unexpected consequences of GBV prevention programming, the evaluation team noted some possible negative consequences: 1) the push to teach children about what to do in sexual abuse cases may result in the problem of children disclosing abuse before a child protection system is put into place; 2) the emphasis of NGO implementers’ programs and services on Burmese refugees may lead other groups to feel that their concerns are less of a priority; and 3) several NGO refugee staff perceive that inquiring with community members about their experience with or knowledge of GBV is impolite, making it unlikely that anyone would disclose cases to those staff.

Evaluation Question 6: What factors explain expected and unexpected negative or positive consequences?

Several situational and contextual factors were identified as contributors to unexpected negative and positive program consequences. In general, NGO implementers and program beneficiaries offered relatively little information on factors that explained expected and unexpected program consequences. Predominate and unifying influences on GBV prevention program outcomes were difficult to identify.

Evaluation Question 8: To what extent have men and boys been included in GBV awareness campaigns? If they were not included, why was this? If they were, what was the impact and how was it measured? Do the GBV programs address the issue of the male survivors of sexual assault or domestic violence? If yes, how?

Part I: Male engagement in GBV awareness

In general, all NGO implementers made efforts to engage males in GBV prevention and awareness activities; however, the implementers did not measure the impact of male engagement. Implementers reported recruitment of men and boys into GBV prevention activities as a key challenge in Malaysia due to cultural norms; they also reported difficulty with reaching most males over the age of 12 years, as their time is partially consumed with employment.

Part II: Male survivors of sexual assault or domestic violence

PRM-funded GBV prevention programs in Malaysia did not address the needs of male survivors of sexual assault and violence, and KII respondents identified unmet needs in this area. Sexual assault is frequent among individuals who were incarcerated (often in their countries of origin), whereas domestic violence is common in Malaysia. Several respondents commented that locating shelters for boys who have been abused or are at high risk is a challenge, and UNHCR indicated a need for a shelter for boys due to traumatizing events that occurred during boat journeys, in particular for unaccompanied minors.

Evaluation Question 9: What were the short- and long-term outcomes of PRM-funded GBV prevention?

Understanding the short- and long-term outcomes of PRM-funded GBV prevention programs requires the measurement of outcome-level indicators such as the incidence of GBV, beliefs and attitudes about GBV, and the prevention of GBV. However, NGO implementers do not collect information at the outcome level in practice, which renders systematic assessment of PRM-funded program outcomes impossible. Furthermore, PRM has been funding NGO implementers in Malaysia for only three years and long-term outcomes are yet to be determined. When asked about short- and long-term outcomes of NGO implementers’ GBV prevention programs, respondents were unable to identify either.

Recommendations

The evaluation team provides the following evidence-based recommendations for continued progress in GBV prevention programming:

PRM should provide financial support for an extensive situational analysis to understand the needs and priorities of refugee communities that program interventions seek to benefit. The situational analysis should include participatory assessments to identify the protection concerns of men, women, girls, and boys. Data collected should be disaggregated by sex and age. Situational analysis should focus on risk-mapping, immediate and root causes of GBV cases, and the use of international and domestic legal standards as a framework for analysis and action.
PRM should convene a working group that aims to create and implement a five-year plan to include: standardization of GBV definitions in accordance with Inter-agency Standing Committee (IASC) guidelines; plans for a baseline study to characterize types of GBV and their incidence and prevalence; exploration of possible research projects in partnership with UNHCR and NGO implementers to conduct in-depth examination of certain issues; and technical support and/or collaboration with monitoring and evaluation (M&E) specialists to improve the development and reporting of GBV indicators.
NGO implementers should seek consistent, quarterly consultation with DoS/PRM/Washington and DoS/PRM/Bangkok, as well as with public health experts based in Kuala Lumpur, regarding the collection of GBV-related information that will be useful in risk-mapping and understanding GBV as it is experienced by refugee populations. For example, a confidential survey that conforms to standard research guidelines and best practices could be administered to a representative sample of refugees.
Programs that focus on healthcare, livelihoods, or other areas should be required by DoS/PRM to develop specific and measureable objectives that clearly relate to GBV activities. Additional mapping exercises of resources and GBV risk should be undertaken (ideally by ICMC due to the large scope of its activities across the refugee community in Malaysia).
DoS/PRM should make a concerted effort to develop and share a common understanding of what is meant by GBV prevention programming and ensure that NGO implementers funded by PRM are clear about prevention programming approaches.
NGO implementers should design, implement, and evaluate GBV primary prevention programs from a social norms perspective. Changing norms that legitimize GBV requires the collaboration of service providers across diverse sectors (e.g. health, protection, psychosocial, and livelihoods) to successfully engage influential individuals (e.g. husbands, religious leaders, youth, mother-in-laws, etc.) and groups (e.g. traditional leaders, religious leaders, peer groups) in identifying, discussing, and challenging social norms that legitimize GBV.
PRM should support the identification and dissemination of validated tools for measuring the impact of GBV primary prevention programs on changing social norms that legitimize GBV. Appropriate tools could include mapping risks of GBV, mapping networks of influential individual and groups to engage in changing norms that legitimize GBV, and tools to measure individual and community readiness for change in humanitarian settings.
PRM should develop an internal results-based management system to support the implementation of its Functional Bureau Strategy, including a logic model that demonstrates the sequence of cause-and-effect relationships between activities, outputs, outcomes, and goals. The logic model could explicitly cover GBV prevention programs, or the logic model could demonstrate how GBV prevention should be integrated into all PRM interventions.[5]
PRM, in consultation with UNHCR, should disseminate required GBV M&E methodologies to NGO implementers. Required M&E methodologies should allow flexibility related to context while supporting the need for standardization of GBV indicators, timeframes, tracking of unintended positive and negative consequences, and staff accountability in humanitarian settings. Programs that focus on healthcare, livelihoods, or other areas should be required by PRM to develop specific and measureable objectives that clearly relate to GBV activities. Use of common M&E methodologies—including standardized indicators[6]—will enable PRM to make comparisons across settings about the impact of GBV prevention programs, thereby providing monitoring data for PRM’s internal results-based management system.[7]
NGO implementers should develop engagement strategies that emphasize men as part of the solution, not the problem. Strategies should use positive messaging to encourage men and boys to develop their potential to act as agents of change. It is important to involve men who are influential in the community and held in esteem by younger men and boys. GBV prevention programs should engage men and boys where they congregate—at sporting events, religious gatherings, cafes, and even bars. It is also important that men and boys have a safe place in the community where they feel they can discuss sensitive issues and reveal their anxieties and vulnerabilities. Effective programs engage both males and females in separate age-appropriate groups with same-sex facilitators, later bringing the groups together for open discussion. Men and boys should have access to specially-designed assistance programs to meet their needs when they have experienced sexual violence. Establishing a shelter for boys in Kuala Lumpur is a good immediate step to address unmet needs; however, in the long term, a more coordinated effort is needed to support male sexual abuse and GBV survivors.

EVALUATION PURPOSE AND QUESTIONS

Evaluation Purpose

This performance evaluation examines the effectiveness of GBV prevention programming funded in Malaysia by DoS/PRM during fiscal years 2010-2012 (October 1, 2009 – September 30, 2012). Fieldwork conducted as part of this evaluation contributes to a one-year evaluation of GBV prevention programming supported directly by PRM or indirectly by its partner organization, UNHCR. The purposes of the evaluation are as follows:

Assess the effectiveness of GBV prevention programming for individuals and communities at risk;
Identify appropriate indicators for measuring the effectiveness of GBV prevention interventions in refugee settings; and
Characterize best practices and lessons learned in engaging men and boys in GBV prevention and response interventions in refugee settings.

The evaluation will provide DoS/PRM, multilateral organizations such as UNHCR, and NGO implementers with guidance about conducting priority GBV prevention initiatives; monitoring and evaluating field- based GBV prevention programs; and engaging host country, international, and local NGOs in best practices for GBV prevention.

Evaluation Questions

The evaluation seeks to answer the following questions:

1. Did partners achieve the program activities defined in their project proposals? What were the barriers and facilitators to implementing program activities?

2. Were the objectives of the program based on evidence such as needs assessments or other forms of data? Were they realistic, measureable objectives? If not, how can the objectives be improved?

3. Did the GBV programming conform to internationally accepted GBV guidelines produced by the humanitarian community?

4. Are the indicators produced by the humanitarian community for GBV programming appropriate for measuring the outcomes of PRM-funded GBV prevention programs? Are the indicators in the project proposals SMART? How can proposal indicators be improved? Do indicators from the GBV guidance documents effectively capture the impact of GBV prevention programs? Are some more useful than others and for what reasons?

5. Were there any unexpected negative or positive consequences of PRM-funded GBV programs? Did organizations address negative consequences and how?

6. What factors explain intended and unintended negative or positive consequences?

7. What outcomes did GBV awareness campaigns achieve? Are the indicators for these programs SMART? How can indicators be improved for GBV awareness campaigns?

8. To what extent have men and boys been included in GBV prevention programs? If they were not included, why was this? If they were, what was the impact and how was it measured?

9. What were the short- and long-term outcomes of PRM-funded GBV prevention programs?

PROGRAM BACKGROUND

Malaysia Country Background

Malaysia depends heavily on the cheap and readily available migrant labor force from the Asian region. The demand for labor has resulted in unwanted, irregular movements of economic migrants. Malaysia’s relative political and economic stability has also attracted refugees and asylum-seekers who are fleeing conflict and persecution. Low-skilled migrant workers have a marginal and semi-marginal existence in Malaysia. However, policies toward irregular migrants and refugees, tempered by issues of ethnicity and racism, are less tolerant.[8] Malaysia makes no distinction between refugees and undocumented migrants. There are an estimated 4 million migrants in the country, including 2 million who are undocumented and considered illegal. In January 2013, there were 90,185 registered refugees and 11,650 asylum seekers; the total population of concern to UNHCR was slightly less than 222,000. There are large numbers of undocumented, urban refugees—predominantly from Burma, but also from other countries throughout Asia. Refugees are subject to arrest, detention, physical punishment, and deportation. In the past several years, a decrease in arrests of documented refugees and asylum-seekers has been observed; however, large-scale arrests and detention of unregistered asylum-seekers persist.[9]

Malaysia has not acceded to most major human rights instruments. As Malaysia is not a signatory to the 1951 Refugee Convention or its 1967 protocol, refugees and asylum-seekers hold no legal status and are treated as illegal immigrants subject to harsh penalties such as arrest, detention, caning, and deportation. Malaysia ratified the CRC in February 1995, which requires member states to provide appropriate protection and humanitarian assistance to refugee children. Malaysia is also a signatory of CEDAW. However, the government maintains reservations on several CRC and CEDAW articles. The absence of a legal framework that regulates the status and rights of refugees and asylum-seekers leaves these populations vulnerable to human rights abuses. In a human rights report endorsed by 54 organizations working in Malaysia, key concerns included “the slow pace of the attainment of substantive equality and, in some instances, the roll-back in women’s rights; the failure of the legal system to ensure that perpetrators of violence against women are held accountable and sentenced appropriately; the discrimination faced by women, (especially Muslim women), during divorce and other matters related to marriage and family; and the escalation of discrimination and violence against lesbians, bisexual women, and the transgendered.”[10] The report noted that the CRC Committee for Malaysia recommended that the government accede to those human rights instruments that constitute a rights framework for migrant and refugee children. The NGO coalition also recommended that police, religious authorities, prosecutors, and the judiciary receive gender sensitization and human rights training. Domestic NGOs have limited capacity to support asylum-seekers and refugees, while international NGOs face significant difficulties in operating in the country. The achievements made by human rights and humanitarian assistance and protection programs should be considered in light of the difficult working environment in Malaysia.

Program Response

DoS/PRM’s mission is to provide protection, ease suffering, and resolve the plight of persecuted and uprooted people around the world. PRM provides life-sustaining assistance, works through multilateral systems to build global partnerships, promotes best practices in humanitarian response, and ensures the integration of humanitarian principles into U.S. foreign policy. The work of NGO implementers is instrumental to ensuring that PRM achieves its humanitarian objectives and fulfills its mandate. PRM directly funds NGO programs designed to fill critical gaps, such as GBV prevention in humanitarian assistance and protection programs. PRM relies on NGO implementers and benefits from their commitment to assist refugee and other populations in protracted situations. Not only are these implementers crucial for service delivery, they also provide critical information for policy development and advocacy.

PRM programming goals in Malaysia include the primary prevention of GBV. Primary prevention aims to prevent violence before it happens, whereas secondary and tertiary prevention focus on response to violence that has already occurred immediately (secondary prevention) or in the longer-term (tertiary prevention).[11] Based on definitions used by the World Health Organization, Centers for Disease Control and Prevention, and United Nations, primary prevention can be understood as follows:

Carried out before violence first occurs;
Aims to prevent initial perpetration or victimization;
Addresses social norms and environmental factors that contribute to violence; and
Appears to be most successful when carried out as part of comprehensive, multi-sectoral efforts

In Malaysia, the evaluation team examined programs implemented by three NGOs with PRM funding. Two of the programs, implemented by ICMC and WAO, had explicit objectives for the prevention of GBV. The third NGO, HEI, provides medical and mental health services that indirectly treat GBV survivors as well as refers survivors to the appropriate GBV-specific treatment and prevention services.

Health Equity Initiatives: HEI is a local organization based in Kuala Lumpur with the objective of improving mental health and medical services for Burmese and other refugees and asylum-seekers, with an emphasis on victims of torture, forced labor, human trafficking, and exploitation. To achieve this objective, HEI builds the capacity of refugees to work within their communities on mental health issues; reinforces referral and case management systems in conjunction with UNHCR, other NGO implementers, and medical facilities; and conducts mental health outreach and screening.

International Catholic Migration Commission: The mission of ICMC is to reduce the risk of GBV among refugee women and children in Kuala Lumpur’s Klang Valley and surrounding areas and to improve access to emergency support services for survivors. ICMC works in collaboration with UNHCR and four local NGOs to meet its objectives, and it coordinates with other organizations to identify and assist GBV survivors. ICMC and its partners provide emergency support services for GBV survivors, including emergency shelter and auxiliary services such as interpretation, transportation, and psychosocial support. ICMC and partners also raise awareness about GBV-related issues and promote community involvement in preventing and responding to GBV.

Women’s Aid Organization: WAO is a local organization based in Kuala Lumpur that provides mental health services for refugees and asylum-seekers and is under contract with UNHCR to provide counseling for GBV survivors on the UNHCR compound. It also maintains a shelter to protect women who are in danger. ICMC has an MOU with WAO to provide PRM-funded shelter and mental health services to survivors referred by ICMC.

EVALUATION DESIGN, METHODS, AND LIMITATIONS

Evaluation Design and Data Collection Methods

This performance evaluation employed the standard rapid appraisal methods of document review, KIIs, FGDs, site visits, and direct observation of program activities. The Malaysia performance evaluation complements, and builds upon, findings from the Desk Review Report submitted to DoS/PRM in July 2013 by providing primary information on best practices, lessons learned, and directions for future programming, support, and PRM engagement. The evaluation team identified the following five categories of target groups as data sources for the field evaluation:

Donor/U.S. Government Partners: DoS/PRM/Washington, DoS/PRM/Bangkok, DoS/Kuala Lumpur
NGO Implementers: ICMC, RWPC, HEI, CHWs, WAO
Sub-Grantees: CELC, Dai Community Organization FOCUS Project
Beneficiaries/Program Participants: Refugee community members, refugee leaders, teachers, students, GBV survivors
External Actors: IOM, Kumpulan ACTS, Pink Triangle Association

Document Review

The evaluation team conducted a document review for the Malaysia field evaluation in conjunction with work performed for the July 2013 Desk Review Report. The review included the following sources:

Guidelines on global GBV prevention and response in humanitarian settings;
Publications and reports on best practices and lessons learned for GBV prevention; and
Proposals, reports, program evaluations, and indicator data submitted by the NGO implementers.

Key Informant Interviews

The evaluation team conducted in-person KIIs in Bangkok, Thailand en route to Kuala Lumpur, Malaysia, where fieldwork was completed from June 9-22, 2013. Key informants were identified in each of the five target groups described above based on input and guidance from PRM/Washington, NGO implementers, and UNHCR/Malaysia. The evaluation team conducted a total of 35 KIIs.[12] The team conducted the KIIs on an individual basis or in groups to maximize efficiency, depending on circumstances, appropriateness, and availability of resources. KIIs were structured around the nine evaluation questions and aligned with the three directives in the Statement of Work (Annex I). KIIs were semi-structured and based upon the questionnaire presented in Annex II: Data Collection Instrument; interviews combined both closed- and open-ended questions. The evaluation team developed a series of sub-questions to complement the nine SOW questions, which allowed for deviations from the established script to pose follow-up questions and explore additional areas of inquiry. Gathering information from some of the key informants required more than one interview; follow-on interviews enabled the team to deepen inquiries, particularly as data collection and analysis proceeded during the second week of fieldwork.

Focus Group Discussions

The team conducted FGDs with groups of individuals who were either a) direct beneficiaries of the NGO implementers, such as recipients of GBV treatment services, b) participants in awareness-raising activities, or c) beneficiaries of training programs, including community organization members and leaders, teachers, and students. FGDs were organized to include individuals who possessed unifying characteristics that might distinguish their responses to interview questions from those of other groups with different characteristics. Key characteristics included sex, age, and exposure to specific program interventions or activities. The team conducted seven FGDs with a total of 56 participants from the following groups:

Refugee Women’s Protection Corps (RWPC)
Community Health Workers (CHWs)
Community Empowerment and Livelihood Center (CELC)
Kachin Refugee Learning Center
Myanmar Refugee Learning Center
Dai Community Organization
Dai Community FOCUS Project

The evaluation team facilitated the FGDs by adapting the questions presented in Annex II: Data Collection Instrument. Additionally, FGDs with recipients of services provided by NGO implementers focused on the following topics: perceptions of services offered; changes in knowledge resulting from participation in programs; and perceptions of the value and impact of services or support offered.

Site Visits and Direct Observation

The evaluation team visited several sites throughout Kuala Lumpur and the Klang Valley including refugee community centers, refugee learning centers, schools, shelters, and livelihood and sub-grantee offices. The team was fortunate to observe a selection of activities being implemented including an ICMC student training on respect for the opposite sex called Respect/Respek developed by the Women's Centre for Change Penang, as well as a teacher training focused on helping children to understand their rights and responsibilities and identify instances of sexual abuse, molestation, and violence.

Limitations

For example, findings about unexpected negative and positive consequences of GBV prevention programming (Evaluation Question 6) were essentially nonexistent during the desk review. Conversely, when applied to the field evaluation, this evaluation question yielded some interesting findings. In light of the varying applicability of the evaluation questions to the field evaluation context, several evaluation questions did not yield much data. As such, this evaluation report hones in on the evaluation questions that yielded the most interesting and extensive data (Evaluation Questions 1, 2, 5, and 8). Findings, conclusions, and recommendations are presented for Evaluation Questions 3, 4, 6, and 9; however, these questions did not provide significant data beyond that which was presented in the Desk Review Report. Due to the lack of findings, this evaluation report does not discuss Evaluation Question 7. Readers should refer to the Desk Review Report, which presents comprehensive data on this question.

EVIDENCE AND FINDINGS

Evaluation Question 1: Did partners achieve the program activities defined in their project proposals? What were the barriers and facilitators to implementing program activities?

Part I: Achievement of program activities as defined in project proposals

Nonetheless, the volume and diversity of quality services that NGO implementers provided was impressive, despite limited human and financial resources. However, KIIs with NGO management and FGDs with NGO staff and refugee workers underscored the challenges faced by implementing partners in accurately tracking program performance. While NGO implementers have constructed indicators to measure progress against program objectives, as well as targets for some of the activities and indicators, the evaluation team did not find evidence of routine activity-tracking. In the absence of a clear system to track planned versus actual implementation of activities over the life of a project, NGO implementers struggled to definitively report whether or not proposed activities had been successfully implemented. When asked whether or not NGO partners had successfully implemented their project activities, one DoS staff person reported, “PRM has been funding these programs for three or four years. There has been a definite impact but it is hard to say precisely what that impact has been.”

Health Equity Initiatives

HEI received direct funding from DoS/PRM in June 2010 for programs to support refugees’ mental and physical health. Program activities as outlined in the proposal, and current progress toward achievement of these activities, are summarized as follows:

Community Based Mental Health Program: The foundation of this program is the training of refugee community members as Community Health Workers (CHWs) on mental health. The trained CHWs are then able to undertake a wide range of activities that include psycho- education, mental health screening, making referrals to service providers, undertaking simple supportive interventions, translation during mental health sessions, making home visits to support treatment adherence, and supporting fellow refugees to navigate the public health system. HEI focuses on building a gender perspective into the work performed by the CHWs and in their engagement with other partners.
Mental Health Services: HEI reported 248 active case files, out of which 50 are clients in psychiatric treatment and 190 are receiving psychotherapy, including Cognitive Behavioral Therapy (CBT) and Interpersonal Therapy (IPT). HEI also provides clients with supportive counseling and group therapy. HEI uses a validated mental health screening tool with all of the individuals it serves. The psychological screening test measures depression, stress, and anxiety for all clients. The tool provides HEI with a score that directs staff to take and/or recommend a clinical intervention if required. It further uses validated assessment tools to make a diagnosis and adopt a treatment plan for each client. HEI’s mental health screening tool does not ask clients about whether they have experienced GBV. If GBV happens to come up naturally during the intake and screening process and the client is showing symptoms of stress, anxiety, depression, and Post-Traumatic Stress Disorder (PTSD), the individual will be referred for mental health treatment and further referrals will be made for GBV case management to partner NGOs. However, if symptoms of mental health distress are not detected, HEI reports that staff “leave them alone so as not to re-traumatize them.”
Case Management Services: HEI facilitates access to health and mental healthcare; liaises with detention authorities; and works to build and strengthen the referral network for mental and physical health services for refugees. Coordination and referrals of clients between HEI and WAO was very strong and occurred on a more limited basis between HEI and ICMC. In the course of case management, if a client cites GBV, the provision of services will be facilitated and referrals will be made according to HEI’s GBV protocol.
Research and Advocacy: HEI’s research-driven advocacy builds links between social marginalization and ill-health. HEI advocates through national and regional coalitions, and through the professional networks of the directors and steering committee members. A key advocacy message is calling for the Malaysian government to sign the Refugee Convention enabling refugees the right to work.

The proposed HEI program activities do not focus on GBV. Among the 52 referral cases that HEI received through April 2013, six were GBV-related. HEI only handles cases of GBV if the problem comes to light in the context of routine program encounters; HEI does not routinely screen its clients for GBV. HEI reported “we are not running a GBV program and we don’t have the resources to do this—this is not what we do; we will take referral cases from UNHCR that have mental health issues due to GBV.”

International Catholic Migration Committee

ICMC has received funding from DoS/PRM since 2010 for the implementation of activities seeking to reduce the risk of GBV among refugee women and children in Kuala Lumpur and to improve access to emergency support services for refugee survivors of GBV. ICMC exceeded its targets for the number of beneficiaries reached by more than 1,000 people over the last few years. Despite its small staff size and lack of office space, ICMC leverages its strong connections within the refugee community and service provider network to expand services for GBV treatment, referral, and prevention. Program activities as outlined in the proposal, and current progress toward achievement of these activities, are summarized as follows:

Building capacity among Burmese refugee communities via training: ICMC’s RWPC provides GBV prevention training for adults, covering the topics of sex and gender during sessions two-three hours in length. The youth training, for ages 13-17, started last year and covers the topics of healthy relationships; respecting the opposite sex; how to characterize your friends using a spectrum of acquaintance to close friend; misplaced trust; and how to say no, to protect yourself, and to avoid being forced into an unwanted relationship. ICMC also provides children ages 8-12 with safety training. This training used to be conducted by Protect & Save the Children, but due to language challenges, it is now taught by the RWPC. Finally, ICMC approaches community leaders to arrange community training for 30-50 community members. The youth training targets eight schools with an average number of 15-20 people trained per session.
Raising awareness about GBV through information campaigns, home visits, and community meetings: ICMC conducts awareness-raising activities throughout the various refugee communities, including distributing brochures and other materials at community events, escorting victims to providers and clinics, partnering on student and teacher training in the informal community schools, following up on inquiries and contacts, and reaching out to identify and assess victims or those deemed to be at high risk of GBV. Additionally, the RWPC staff members typically visit three different houses each day to perform their outreach activities on GBV awareness-raising.
Working with partner service providers to create a referral system for GBV survivors: ICMC’s starting point is always referral to UNHCR. One ICMC staff member reported: “We don’t go into the depths of the case, so that people are not re-traumatized by telling the case over and over; we want to make sure someone is under the care of the Community Development Unit (CDU), which has trained professional counselors and is more equipped to suggest certain responses to survivors. In low risk cases we can provide peer counseling, emergency funding, or shelter. In some instances, registered refugees can receive a 50 percent discount at the hospital via an arrangement with UNHCR. For non-registered refugees, ICMC provides emergency funds to assist with hospital bills. We try to fill in the gaps between the referral to UNHCR, which can take some time for them to respond to—when there is an emergency and there are immediate needs.”
Training teachers about child sexual abuse and training youth about their rights and responsibilities: ICMC introduced the teacher trainings, provided by Childline, into its project this year after learning from post-tests conducted among teachers who attended the children’s safety training that they were not well informed about how to support and respond to children who confide in them about experiences with GBV. In response to this gap, ICMC proposed the two-day teacher training and has thus far covered 80 teachers from 22 schools through two sessions. ICMC’s target is to reach 100 teachers from 30 different schools.
Responding to the safety, protection, and medical needs of GBV victims by providing victims with, or referring them to, necessary and appropriate services: All of ICMC’s survivors are referred immediately to shelter services with either WAO or Pusat Kebijikan Good Shepard (PKGS). One ICMC staff person reported: “All people in the shelter get in-house counseling sessions, medical services, police reporting, interim protection orders—all of this takes place first before we even think of employment. We have a nice female taxi driver who can transport the survivors and most of the time they are escorted by RWPC and then we do an initial intake session and then hand over to a social worker who manages her case.”

Women’s Aid Organization

WAO receives funding from DoS/PRM via ICMC (since 2010) for the following proposed activities:

Provision of shelter to victims of GBV (and their dependent children): Most clients who require shelter from WAO are victims of domestic violence. All WAO shelter clients receive in-house counseling sessions; medical services; and support with police reporting, obtaining interim protection orders, and job searches.
To train members of the RWPC, peer counselors, and community leaders on various aspects of GBV: WAO provides all members of the RWPC with 12 hours of GBV awareness training; five-six selected members are trained to become peer counselors and receive 30 hours of training and six hours of progress supervision. WAO also trains refugee community leaders in GBV awareness and reaches as many as 15 different communities or refugee organizations per training.

KIIs and FGDs with refugee leaders, program beneficiaries, teachers, and community members revealed strong recognition of PRM and ICMC activities, while HEI and WAO were less well-known. The majority of refugees interviewed said they had either received services from ICMC or had reached out to ICMC for assistance. Citing the responsiveness and resourcefulness of ICMC staff, many refugees expressed a preference for working with ICMC over UNHCR—even in instances when UNHCR provides the service they required.

Part II: Barriers and facilitators to implementing program activities

Barriers identified by NGO implementers and refugees

Shame: Shame prevents many victims from reporting cases of GBV. When asked how often she thinks refugee women experience sexual harassment, a refugee NGO worker said, “I think it happens a lot but the problem is that the refugee women don’t know what to do. They think maybe it’s their fault, and whatever happens they keep it to themselves until they cannot keep it inside anymore.” One community leader told the team that, “Even though we have had rapes we do not go to the police; the women do not want to report because the community and the other people will see how they are and they are ashamed so we do not report.”

Police extortion: One NGO told the evaluation team that there is a concern about GBV whenever the police interrogate or extort money from refugees; nearly every refugee interviewed raised the issue of extortion. The risk of extortion, arrest, and detention decreases the likelihood of reporting GBV. The majority of refugees interviewed, including those registered with UNHCR, expressed fear of arrest and detention; many raised the issue of extortion by the police. Extortion is so pervasive that NGO implementers report they have difficulty contacting refugees as their phones are often stolen by the police. During shakedowns, police often rob refugees of everything and instill fear in them; it can take up to two weeks for registered refugees to be released from detention. The experiences of several refugee NGO workers are described below:

“I have [had] cases where police have [harassed and threatened] the victim about reporting and say she will be sent back to her own country… [A]buse of power happens and [refugees] have to face the violence and they don’t know where to go to seek help—it’s the fear of being arrested and then the fear of being sent back to their home of origin… I think [women] who have been raped don’t have the opportunity to report, don’t have enough [of a] support system… I think it’s happening a lot…”

“The problem is we cannot report to the police… I tried to report a robbery to the police but they refused to help me. They said I was rubbish. Another time I had to pass one of the bridges to cross the road [from work] and the police were always waiting there to take money from me… they search[ed] my whole body… they took my phone. Women are afraid of the police... If I don’t have a UNHCR card, then I am totally powerless.”

“The [police] are always waiting for me—police asking [me to pay money]. Some of our clients have a financial problem [and] if they [walk outside] the police will really pester them. We are really afraid to go out. [If we don’t have money the police will] drive around and around until we can find someone to pay.”

Arrest and detention of NGO staff and clients: One NGO expressed major concern about hiring unregistered refugees, stating that to do so would place those refugees at substantial risk. To address cases of police harassment of voluntary community health workers, the organization established a protocol to respond to law enforcement and trained coordinators to prevent refugees’ arbitrary arrest and detention.

Risks and costs of medical care: In addition to acting as a deterrent against GBV reporting, risk of arrest and detention is also a barrier to medical care. Refugee women are unable to access the Malaysian government’s One Stop (Rape) Crisis Centers, as treatment is dependent on lodging a police report. One NGO worker noted that, “GBV survivors can go through the Emergency Room, but then we don’t know how well the Emergency Room staff will manage the case.” Hospital personnel sometimes report refugees to immigration authorities when they cannot pay their bills, resulting at times in immediate arrest of patients at discharge, including women with newborns. For example, a man diagnosed with traumatic gallbladder perforation was arrested when he could not pay his hospital bill and held for more than one month (HEI and UNHCR were able to advocate and arrange for monitoring the client’s health condition). Some hospitals are open to providing services to refugees, depending in part on the attitudes of the doctors and medical staff; however, doctors who are willing to provide services without police reports have said that they face challenges within their hospital systems.

HEI said that for clients whose mental health is affected by their physical health, a volunteer CHW will take the client to the hospital, translate for him/her, and obtain a note from the doctor about diagnosis and treatment. HEI then works with a panel of volunteer doctors who advise on how the person can obtain assistance; more often than not, payment for services is the central issue. HEI has regular meetings with medical staff at the hospital who deal with these cases. “One person had no identification card at all and we were able to advocate for her to get the treatment she needed,” HEI said, but also acknowledged that hospital administrators are reluctant to provide care when there is no identification or UNHCR registration card. With a UNHCR registration card, a person can receive care at half of the foreigners’ rate, but the price is still extremely high. For example, obstetrical delivery for a Malaysian costs about 15 Ringgit (~$1.50 USD) and for a foreigner 800 Ringgit (~$250 USD), so it would cost 400 Ringgit ($125 USD) for a refugee—still beyond what a refugee could afford.

Registration delays and lack of legal protection: Due to resource constraints, UNHCR is unable to register refugees and asylum-seekers within a reasonable period (at the time of fieldwork, initial appointments to register with UNHCR were being offered for Fall 2015—and initial appointments do not usually result in obtaining a UNHCR card). The recent arrival of many new refugees, especially Rohingyas from Burma, increases concerns about the abuses they may face. Refugees report that sometimes only some family members are registered for reasons that are entirely unclear and that this can translate into resettlement delays. There is a nearly complete lack of legal aid available for refugees who want to challenge decisions about their resettlement.

ICMC, WAO, and HEI all work with UNHCR to expedite registration for GBV survivors who are unregistered at the time of disclosure. The coordinator at ICMC maintains case files of all disclosures and all shelter cases, conducts follow-up with UNHCR’s Protection Unit, generates shelter referrals and coordinates services for shelter clients including discharge and transition back to their communities, facilitates access to medical care and counseling as needed, and coordinates provision of interpreters at counseling sessions. The informal system of alerts between NGOs, UNHCR, and U.S. Embassy officials appears to work relatively well, but there are still some cases that involve serious protection concerns and do not receive a rapid response due to an overburdened UNHCR system. At the same time, UNHCR does appear to attempt to fast track high-risk GBV cases and the U.S. Embassy and others working with UNHCR try to facilitate third country resettlement.

Cost of transportation to access services: HEI and other NGO implementers face the problem that refugees in need of services cannot afford the transportation costs to avail themselves of those services. For HEI in particular, this means that some refugees in major need of counseling and/or psychiatric treatment are hindered in receiving help. Not only is transportation expensive, but seeking services also requires taking the day off from work or other responsibilities. Experiments with reimbursing clients for travel claims have demonstrated that this seems to alleviate much of the problem, but such arrangements are made on an ad hoc basis. HEI has a system of assessing the needs of clients in order to provide transportation subsidy for those whose conditions are chronic and who have low social support. For example, a client whose husband was in the detention center and is the sole caretaker of her children as she has no community support was provided with transportation subsidy to HEI to receive treatment for her Major Depressive Disorder. HEI has also done some phone counseling to alleviate this problem for clients in particular need of services.

Difficulty maintaining contact with clients and participants: Several NGO implementers indicated that it is often difficult to maintain contact with refugees and asylum-seekers due to the constant theft of mobile phones by taxi drivers, thieves, local gangs, and even the police. Most often, refugees are asked if they are from Malaysia or Burma; if they are suspected of being refugees or asylum-seekers, they are robbed of everything, often even their UNHCR registration cards. Refugees working as CHWs or RWPC members are at constant risk because they are more mobile due to the nature of their work.

Resettlement of trained staff: Maintaining a staff roster with an adequate skill set can be difficult for many NGO implementers. The primary challenge is retaining enough volunteers, as many are in the resettlement pipeline. Further, cultural norms about gender roles may become an issue in terms of attracting male volunteers. For example, the first group of ICMC’s highly-trained peer counselors has almost entirely been resettled. To establish a new group of peer counselors and volunteer CHWs is a tremendous human resources undertaking in terms of necessary recruitment and training.

Lack of childcare for women in counseling or working: GBV survivors reported that a shortage of childcare and nursery options make it difficult for them to access services. ICMC has noted this and is considering childcare as a possible mini-grant project.

Barriers identified by the evaluation team

Near total impunity for perpetrators of GBV: The evaluation team obtained only one example of a case where a perpetrator of domestic violence was jailed, and the police failed to take action until the NGO became heavily involved in the case; the NGO made seven reports before the police arrested the perpetrator, and then only after he nearly killed the couple’s child.

Lack of protocol for managing child sexual abuse cases: Although ICMC’s Child Safety Program (designed to teach young children how to protect themselves against sexual violence) is highly appropriate, no protocol has been developed for the management of cases should children disclose sexual abuse. This is especially concerning if the perpetrator lives in the home, as the immediate protection of the child is critical.

Political sensitivities of refugee work: Working with refugees and asylum-seekers is a particularly sensitive and complicated issue in Malaysia given the fact that these populations are not afforded legal status by the Malaysian government. UNHCR noted that NGOs have difficulty obtaining the necessary agreements and permissions from the government to allow them to operate in Malaysia. Thus, NGOs working in Malaysia are typically quite small and originate from charitable organizations or from several individuals with a shared interest. In terms of capacity, UNCHR reported possessing funding for programs but not being able to find appropriate NGOs to implement them. NGOs are limited in number and capacity, as one UNHCR official observed:

“There is very little variety of NGOs and to complicate matters is the fact that refugees here are considered to be the same as undocumented migrants and are subject to arrest. And there are also problems in the legislation directed at employers, by [put in general terms] saying that an individual or a group housing undocumented migrants can be subject to prosecution. This scares NGOs off as well because they think they will get in trouble [by working with refugees].”

The authorities view organizations that work with refugees with suspicion, regarding them as aiding and abetting lawbreakers rather than acting to ensure the protection of human rights. Although there are no direct threats made to NGOs for working with refugees, there is a thin patina of tolerance of their efforts and some have expressed concerns about possible difficulties with the government.

Lack of data-driven knowledge about types of GBV faced by refugees: Information on GBV in refugee communities in Kuala Lumpur is extremely limited, which leads to the risk of incorrect assumptions. Very little information is available on the actual incidence or prevalence of any type of GBV.[13] Key informants interviewed by the evaluation team believe that domestic violence is the most common form of GBV, but they had limited knowledge about the incidence and prevalence of other forms of GBV within the refugee population. UNHCR maintains quarterly statistics on reported GBV cases; the evaluation team’s review of these statistics revealed confusing terminology and a lack of analysis of certain critical factors.

Lack of clarity about types of GBV and their implications: Key informants struggled to describe types of GBV other than domestic violence. For example, many female interviewees did not realize that frotteurism is a form of sexual assault and referred to it as “harassment.” There was a lack of clarity regarding rape (and a prevailing belief that there must be penile penetration) and some confusion about what constitutes child sexual abuse. For example, one teacher reported that ICMC advised adults to first provide children with counseling, then watch their behavior, and ask the children what really happened; “if it’s a serious case, not just touching private parts, we have to inform the community and call ICMC and UNHCR or HEI… We should take the rape case more serious, but if a student came and had been touched in their private parts we should teach them and tell them to say no or run away because they may have not known what to do.”[14] An NGO refugee worker noted that many children are not aware that what is happening to them is a kind of abuse. He acknowledged that children do not like it but eventually learn to tolerate it and stop resisting. After adults participate in the PRM-funded awareness sessions, they start to understand which behaviors are unacceptable. “They accept when I talk about child abuse and the private body parts that should not be touched, and we told them to spread the word at school and among their friends.”

Lack of access to clients in detention: The few organizations with access to detention centers face numerous restrictions. They are prohibited from advocating for refugee rights in the community and are obliged to be discrete about their activities. For this reason, HEI, ICMC, and WAO elected not to seek access to detention centers. HEI has attempted mobilizing lawyers’ visits to detention centers, but it was unable to obtain a commitment from the lawyers. Follow-up with clients who had been receiving HEI mental health services and were detained was initiated through the Individual Assistance Desk of UNHCR.

Non-halal shelters: Some of the shelter facilities are not halal, meaning that Muslim refugees cannot accept and inhabit them. This should be examined more closely to determine if it is a real barrier for services or if other shelters can provide halal services for Muslim refugees.

Facilitators

Assistance from UNHCR: Although not a perfect system, the lines of communication are open between UNHCR and NGO implementers regarding protection cases that require immediate engagement. ICMC noted that UNHCR has intervened at times to help with practical matters such as encouraging teachers to attend training sessions. “We need this kind of support... and with the mini-grants, when we started we had a lot of guidance from the UNHCR Social Protection Fund. We had a lot of discussions with them. We keep them updated about the projects we are running as well. Certain projects have failed miserably for UNHCR due to fraud, so it’s [helpful to get their opinion] regarding whom we can trust or who we might not want to trust]. They have also been very helpful in terms of how to approach the Rohingya community. So for our mini-grants, a facilitator has been UNHCR.”

Partnerships with refugee community organizations: ICMC, HEI, and WAO all work closely with refugee community organizations, which have been appreciative and supportive of their efforts. ICMC’s close cooperation with WAO, HEI, and the Women’s Centre for Change Penang has enabled ICMC to strengthen its capacity to work with women and youth on gender affairs in Malaysia and to become familiar with counseling and the sensitivities that are involved. In a very creative spirit, ICMC has used spaces and times that might not ordinarily be considered to provide information to refugees. For example, ICMC worked with IOM to conduct awareness-raising sessions in a hospital waiting area where IOM conducts its pre-departure medical examinations. Largely due to the work of ICMC, refugees know a great deal about where to get help and will often call ICMC before UNHCR as they may get a faster response and referral. ICMC has, in that sense, become a clearinghouse of information and referral.

High quality staff at implementing partner organizations: A DoS/PRM official said that, “One thing that facilitates programs is the retention of good staff… we talk with the refugees as well as with staff when we go to evaluate programs, so we feel we have a good idea of what goes on. Staff seem well aware of the issues faced by refugees.”

The number of refugees and asylum-seekers forced by circumstances into survival sex is not known. One NGO implementer reported: “No, we don’t see this. I have not heard of any cases, mainly because they have opportunities to work. Almost all of them are able to find some form of employment, in the restaurants and farming, so therefore we don’t get cases of survival sex; we get cases of rape or sexual harassment.” UNHCR, however, stated that “There are definitely some cases of survival sex, but capacity to address the problem is very low.” Except for UNHCR, which said that there were fewer than 10 cases of trafficking last year, interviewees were not aware of the incidence or prevalence of sex trafficking.

The incidence and prevalence of GBV perpetrated by the police and other authorities, as well as GBV happening in detention, is unknown. Although there have been allegations of rape, sexual assault, and humiliation of refugees and asylum-seekers in detention, none of the NGOs interviewed have documented the number of allegations. There are no recent reports regarding the severity of the problem. One key informant who has direct contact with detainees reported that while she is sure that such abuse occurs, it is very difficult to “get at” the problem; most people are very reluctant to report abuses in detention. An NGO implementer staff member reported that she heard about police asking for sexual favors from women who want to see their husbands or relatives in detention. One female key informant reported to the evaluation team that she had been taken out of the detention center and raped by a male guard. A 16-year-old male beneficiary of a PRM-funded program reported to a CHW that he observed sexual abuse of boys and men while in detention. A female NGO implementer staff member said, “My sense is that sexual violence [by police] is occurring, but it is hard to know how much it is happening… [T]he power dynamics are so unequal and citizens are very aware, and so are refugees, of the possibility of malicious intent.”

According to UNHCR, child trafficking for early marriage and forced begging is “a rampant issue right now on the peninsula,” but neither UNHCR nor any of the NGO implementers had any specific information. Street children are at high risk for abuse and exploitation, but there seems to be a dearth of information regarding the numbers of refugee street children, which refugee children are most at risk, and the specific challenges faced by street children. UNHCR reported:

“Refugee children are being exploited and refugees are the perpetrators. Begging is a big money- making enterprise… The children were smuggled for begging. Around 11:00 pm in Changkat Bukit Bintang, an area where there are a lot of eateries, there is [also] a syndicate operating (kids pedaling flowers in Berjaya Times square). [We] think that there is no GBV among these kids; sex tourism is not a big thing here as it is in other areas. [But] the danger is there.”

Allegations of trafficking for early marriage and/or arranged early marriage seem to focus on the Rohingya community, but little research has been conducted on the nature and extent of the practice.

NGO implementers lacked conclusive and comprehensive information about GBV committed by employers. While they stated that it happens, they said that it often goes unreported. A male community leader explained, “Some of the boss[es]… show their private parts or physically touch [female employees] ...they can do what they want, since the men cannot be arrested. Even if there is a rape [no one] can do anything.” Another NGO implementer staff member said: “The Chin women are vulnerable to GBV because they work outside the home; we have heard some stories about this, but have not verified it. Some of the young Chin women were promised jobs here; they come in as refugees and think they will get a job in a restaurant but then they end up being kept in the house and raped.”

The nature and extent of GBV among refugees and asylum-seekers experiencing forced labor is largely anecdotal. A 2012 HEI study of 1,074 Burmese refugees and asylum-seekers from eight ethnic groups found that about one-third of the study population had experienced forced labor. Both men and women experienced forced labor conditions, and physical confinement was reported by nearly 25 percent of the study population.[15] Sectors involved included plantations, construction, restaurants, hotels, and shops. The two in-depth case descriptions involving women revealed that they both had experienced rape, sexual enslavement, and sexual assault (one of the women was trafficked for sex as a child). The study did not explore the issue of GBV in any more depth except to say that sexual exploitation is common among forced laborers and that agents offering work or smuggling for work are often perpetrators.

Some studies revealed specific problems but have yet to result in significant changes to the NGO implementers’ programming. For example, HEI’s study on gender and GBV in school-aged children revealed that 29 percent of male children report that they have been “touched below the waist.”

Results of the study were used to advocate for the sex education module developed with the Ministry of Education to be available to refugee children and to engage with the medical professionals who are involved in the public hospital’s Suspected Child Abuse and Neglect (SCAN) Team. While not contributing to GBV prevention or program planning, HEI reported developing a rapid assessment tool for care providers at government-run shelters to facilitate their response to the psychological needs of victims of trafficking. Similarly, WAO reported development of an assessment tool for children at its childcare center and shelter. WAO currently does not have staff members with the requisite skills to conduct the assessment but is in the process of hiring new staff for the childcare center who will be responsible for administering the assessment tool. ICMC plans to develop assessment instruments tailored to refugees, youth, teachers, and RWPC members to analyze knowledge gained, understanding of rights, and GBV prevention in different settings. While ICMC has not conducted any risk mapping, it has mapped where refugees live. This exercise was carried out in the Klang Valley to determine where NGOs provide support and where services are available (i.e., accessibility of healthcare clinics). ICMC used the findings to determine whether and how to expand its programming outside of Kuala Lumpur.

Information on the types of GBV refugees face is extremely limited, which may lead to incorrect assumptions. NGO implementers’ programming appears to emphasize domestic violence over other types of GBV; however, the decision to focus resources on this area does not appear to be evidence- based. As presented in the Desk Review Report, PRM-funded activities were specific, measurable, and realistic for programs that focused directly on GBV prevention and response (ICMC). In comparison, the relationship between activities and objectives was less specific in programs where protection and/or GBV prevention and response were one of many components or service areas (HEI and WAO).

Evaluation Question 3: Did the GBV programming conform to internationally accepted GBV guidelines produced by the humanitarian community?

Few respondents, with the exception of ICMC, could identify any of the international guidelines by name or provide a precise description of guidelines. However, they were generally aware that guidelines exist. ICMC recalled the GBVIMS and said that it uses UNHCR guidelines for the sub-grantees for standard operating procedures.[16] WAO has its own protocols and develops its own modules for peer counseling training but has not used any international GBV guidelines. WAO does, however, refer to CEDAW, ILO Convention, CRC, and some regional instruments as guidance and advocacy tools.

Specific GBV prevention strategies outlined in IASC and UNHCR guidelines include mobilizing the community to establish a system for survivors to access safe shelter if places of residence are unsafe; establishing community-based protection activities and mechanisms to prevent abuses in places where children gather for education; conducting routine spot checks and discussions with community members to reduce exposure to sexual violence due to poor shelter conditions or inadequate space and privacy; and working with host communities, local authorities, and other partners to reduce tensions relating to scarce resources. To fully involve the refugee community, guidelines recommend supporting community leaders to continuously strengthen prevention strategies by maintaining GBV risk awareness, engaging in problem-solving discussions, and supporting community groups to share information about GBV risks and incidents through formal and informal networks with stakeholders engaged in GBV prevention.

PRM-funded NGOs in Malaysia appear to have mobilized the community to establish a system for domestic violence survivors to access safe shelter if places of residence are unsafe, undertaken some significant work to prevent abuse in places where children gather for education, worked toward the full involvement of the refugee community, and supported groups (Burmese community organizations, in particular) to share information about incidents through formal and informal networks. Issues such as confidentiality, medical care, and provision of psychosocial services appear to be in compliance with international standards—with excellent outreach and counseling by HEI and WAO, and a prominent profile in the refugee community for ICMC. None of the NGOs or refugees mentioned use of complaint mechanisms to ensure accountability.

Codes of conduct are important for protecting refugees and asylum-seekers from humanitarian aid workers who engage in abusive or exploitative behavior. In discussing international standards and guidelines for GBV treatment and prevention, HEI mentioned that it has a code of conduct, which is exemplary. ICMC reported that all RWPC members must sign the ICMC Code of Conduct, which has been revised to incorporate the six IASC core principles relating to sexual exploitation and abuse and was distributed to all ICMC staff in May 2011, including to those who do not have direct access to e-mail. To ensure that staff members are sensitized to the importance of sexual exploitation and abuse, all ICMC employees and volunteers (including all RWPC members) are required to complete the InterAction online training: Addressing Sexual Exploitation and Abuse. ICMC managers supervise the training and collect signatures from staff who attend. This course, which gives an overview of the international standards related to sexual exploitation and abuse— and the steps to address its occurrence—is mandatory for all ICMC staff (employees and volunteers).

Indicators used by NGO implementers to monitor program performance were weak (as noted in the July 2013 Desk Review Report). Specifically, indicators were poorly designed and often included targets. Indicators should be neutral gauges of progress that can be compared against an objective or target. When used appropriately, targets can orient NGO implementers to tasks that need to be accomplished and provide guidance for monitoring whether or not program progress is being made on schedule and if results have been achieved over time.

Generally speaking, NGO implementers were confused about the difference between indicators and targets, as evidenced by the following quotes: “So far, even without staff positions being filled, we are still going to fulfill all the indicators” and “We follow the indicators—for example, we need to achieve 30 schools that receive the children’s safety program. If it’s 8 youth centers to receive the curriculum, then we plan accordingly.” In addition to conflating indicators with targets, many implementers confuse indicators with activities. Examples include: publication and distribution of 5,000 self-help strategies; a community screening tool updated to include a section on torture, forced labor, trafficking, and exploitation; and updated training modules for community health workers.

There was an absence of standardized indicators across NGO implementers—despite sharing common GBV prevention goals, each implementer developed its own indicators. Indicators were largely output oriented and capable of showing progress at the activity level but incapable of demonstrating program outcomes. DoS/PRM/Bangkok reported that it did not observe any improvements in M&E capacity among implementers following PRM’s 2011 shift from one-year to three-year funding cycles (which was an objective of the shift). Interviews with NGO implementer and sub-grantee staff revealed the shortcomings of indicators, as key informants were unable to describe GBV prevention outcomes:

“Most of the things that ICMC engages us to do are on the awareness training, so our evaluation is mostly based on whether [participants] know the knowledge. Before we start the training we give them some tests and then also verbally ask them, but we don’t have a proper way to see what we want to achieve. Our objective is for recipients to have more awareness, but to actually see if this happens we don’t have much thought on that. We never thought about how to develop something to assess the attitudes and see what changes in the process.”

NGO implementers reported that PRM guidance on M&E strategies was limited to the proposal submission process and that it would be helpful to receive guidance and support on indicator development. PRM is aware of the challenges in understanding the effectiveness of implementers’ programs. One DoS/PRM/Washington staff member reported that UNHCR and NGO implementers do not provide PRM with substantive, evidence-based conclusions about the effectiveness of GBV prevention programs. The impact of GBV response interventions is easier to measure than the impact of GBV prevention interventions. It is difficult to prove a negative.

Evaluation Question 5: Were there any unexpected negative or positive consequences of PRM-funded GBV programs? Did organizations address negative consequences and how?

NGO implementers noted potential unexpected negative consequences in relation to shelters and livelihoods programs. The isolation of and/or feeling of isolation among women in shelters can be due to a variety of factors including location, lack of interpreters, and rules requiring women to temporarily hand over their phones and money until they leave the shelter. The evaluation team noted that at one shelter, which was perceived to be the safest due to its secret location, it seemed that women were leaving more quickly—potentially as a result of isolation. At the same time, rules are necessary to prevent the identification of shelter locations. WAO reported that one man learned the location of a shelter from police reports and went there to look for his wife. When he did not find her there, he killed himself in front of the shelter, traumatizing everyone there. One NGO implementer staff member noted that livelihoods programs that provide women with greater independence may increase tension within the home due to anger over lack of control, competitiveness, or insecurity; the abuser may become more controlling as he feels threatened.

DoS/PRM/Bangkok reported that it was, for the most part, unaware of negative consequences of the programs but said it was possible that some husbands may react negatively to their wives’ assertion of rights. A male refugee community leader explained that UNHCR’s refugee status determinations can cause domestic violence, often due to misunderstandings that become very difficult to clear up due to translation problems at UNHCR:

“I have a brother and he is not registered yet. Because he is my brother, I have to wait another two or three years for [resettlement]. Because we are refugees, we cannot find jobs and we don’t have money, so we fight and say, ‘it’s because of your brother or your mother that we cannot resettle’...pressure and frustration [builds] up. I see a lot of [family members] fighting and I ask them why...they say it’s ‘because UNHCR says my wife is lying.’ They cannot tell their story [due to language challenges and so] UNHCR rejects the case or postpones the case and the husband and wife fight... We have a hard time explaining to UNHCR how we are related. Is he my cousin or my brother or my cousin’s brother? We call them all ‘cousin,’ [but] if we don’t explain it properly, maybe UNHCR will say that I am lying and then that delays the case.”

The evaluation team found that this was not an unusual complaint. Other key informants said they were confused because one family member or another was denied resettlement when others were not, and there was no clear reason given. In one case, a man’s sister was refused resettlement and never received an explanation; she was denied at 12 years old and is now 15 or 16 years old). In another case, a man and his family are to be resettled, but his parents have been refused.

One NGO implementer identified an unintended positive consequence related to refugees who work for implementers; these individuals gain work experience and skills such as English language, computer literacy, knowledge of health and social service systems, and networking—which will likely serve them well when they seek future employment upon resettlement.

While few key informants had much to offer about unexpected consequences of GBV prevention programming, the evaluation team noted some possible negative consequences:

The push to teach children what to do in sexual abuse cases may result in the problem of children disclosing abuse before a child protection system is put into place. This may lead to ad hoc arrangements that are not in the best interest of the child.
The emphasis of NGO implementers’ programs and services on Burmese refugees may lead other groups to feel that their concerns are less of a priority (although the evaluation team notes that all implementers referred non-Burmese clients to the shelters).
When asked during a FGD about the issue of sexual violence in detention, refugee workers responded, “We don’t know for sure, but we have never heard of it.” When asked if they thought it could be happening and people might not be reporting it, at least two key informants said, “it would be impolite for us to ask.” This is concerning because often in GBV cases, unless the question is asked (and asked in a safe environment), many survivors will not talk about the violence. Both ICMC and HEI refugee staff members make home visits, which provide an important opportunity to raise such issues when conditions are deemed appropriate. But refugee workers’ own discomfort with the topic makes it highly unlikely that anyone would disclose to them.

Evaluation Question 6: What factors explain expected and unexpected negative or positive consequences?

NGOs identified the feeling of isolation among women in shelters as a contributor to negative program consequences. Women in shelters often feel isolated and suffer due to separation from family and/or home; inability to adequately communicate due to language barriers; and incomplete understanding of reasons for rules such as no phones or personal money while in shelters. A contextual challenge is that many refugees have either no work or unskilled work and have not had the advantage of a complete education (or have had no education)—leaving them at a distinct disadvantage for employment. Finally, refugee concerns about open discussions related to GBV as being “impolite” possibly indicates they have not had prior experience discussing their own reactions to and beliefs surrounding GBV, a situational factor that should be accounted for in program planning.

Part I: Male engagement in GBV awareness

Refugees predominantly belong to male-dominated societies (regardless of religion), UNHCR explained, “and the way to get them involved in GBV prevention successfully has not yet been identified.” One female refugee said that during the time she had been in the ICMC program, “Some of the men will turn away—they won’t listen, they will leave; [but] others are quite positive and open to the discussion, even admitting they [hold myths] about these issues… There is no specific effort that we make to get the message to the men, but we think it’s really important for men to be involved because most of the time perpetrators [are] men.”

ICMC has made a concerted effort to hire males for the RWPC, noting that male volunteers facilitate outreach training when groups have more male participants—especially when dealing with male leaders is required. Additionally, males may be more persuasive when training on gender roles and responsibilities. In mid-2011, ICMC noted that it aimed to “better meet the needs/gaps by involving more Rohingya and Burmese Muslims (including men) in [RWPC] activities to improve GBV prevention and awareness among this large and very insular cohort.” In a subsequent meeting, Rohingya community members welcomed an intense awareness-raising session for men. One refugee community leader said:

“We know that the focus of what we do here is to tell what GBV is and then they can prevent it from happening to themselves. The other thing is that we have a different traditional background. We want to share the knowledge so that [people] know what is GBV—sometimes they think that when some men touch the men they think this is not GBV and the boss does this to them but they don’t understand… what we are doing here is to open the discussion to know what is GBV and how can we prevent [it]. We have to go to another country – in America we cannot beat our children, this is child abuse… we have to overcome our traditional knowledge and our assumptions, this is what our focus is now.”

WAO conducts innovative work that is consistent with GBV prevention programs that are believed by researchers to be the most effective. WAO training with community groups involves up to 70 percent men, although there are no specific programs for men and boys.

“We are very clear that we train a community that is made up of men and women, but we do not put our [limited] resources into boys and men. We have public education campaigns; there have been initiatives to reach out to men and boys, but this is outside of our funding. The programming we do for men and boys is through advertisements, poster campaigns, radio campaigns with messages like ‘Do you really care?’ targeting English-speaking and Malay-speaking audiences, middle and upper- middle class people specifically on domestic violence… We keep reaching out to women, [and we know] we need to reach out to men too, but we think that men need to reach out to men… I am very skeptical about reaching out to men in three days—it should be a 365-day effort, it’s not just the one-off mindset… we try to do it through policy, laws.”

The CELC reported that work with men and boys is critical: “I even spoke with ICMC and the youth group because in the GBV situation it’s not only women; it includes both men and women in the situation and the effect is not only for women. It can be for men too.” NGOs and sub-grantees explained that many people did not realize that domestic violence is a crime and that if people knew such violence was criminal, they might be reluctant to commit acts of violence. A number of respondents said they would like to move forward with providing counseling for perpetrators of domestic violence, and several women expressed a desire for their husbands to receive counseling on domestic violence. UNHCR has also expressed an interest in working with perpetrators of domestic violence.

Part II: Male survivors of sexual assault or domestic violence

PRM-funded GBV prevention programs in Malaysia did not address the needs of male survivors of sexual assault and violence, and KII respondents identified unmet needs in this area. Sexual assault is frequent among individuals who were incarcerated (often in their countries of origin), whereas domestic violence is common in Malaysia. One community leader, when speaking about male victims and survivors of abuse, noted that “In Malaysia most men don’t report sexual violence or GBV, but what we have is domestic violence—sometimes the man reports that the wife is beating him or that the wife married another man and ran away.”

Several respondents commented that locating shelters for boys who have been abused or are at high risk is a challenge, and UNHCR indicated a need for a shelter for boys due to traumatizing events that occurred during boat journeys, in particular for unaccompanied minors.

Evaluation Question 9: What were the short- and long-term outcomes of PRM-funded GBV prevention?

One respondent was aptly able to summarize the complexity of outcome-level reporting within the context of a GBV prevention program:

“Of course we hope that the most important thing is that the members of refugee communities will be more supportive when they hear about a case—that is the first thing. And then [we hope] to create some changes in the next generation so we constantly, when we do a one-on-one training, tell them that this will be the way that you train your children so your children can be free from violence. So I’m not entirely sure how to measure the impact in the short term; even in the long term the indicators of measuring [are] very difficult to [determine]. But one of the things I could say is that definitely over the years, not just through [our] program… they do have a better understanding of [what types of GBV occur in the community, and] members [will] actually also know what they need to do if a case happens. They can go to the police and we have cases where when we train the community leaders… they can call us to get advice about how to handle situations in their communities and they will then sometimes mediate… now it’s more in terms to tell the perpetrator that it’s not right, as opposed to before when they encouraged the victim and perpetrators to remain as a family and work it out.”

Other respondents discussed program activities and process- or output-level measures when asked about short- and long-term outcomes. Some respondents reported that their programs were still too new to identify long-term outcomes.

CONCLUSIONS AND RECOMMENDATIONS

Evaluation Question 1

NGO implementers demonstrated commendable achievements in terms of activities. Members of the refugee community were more familiar with ICMC and more likely to reach out to ICMC than to HEI for assistance (in part because ICMC and WAO services are much more focused on GBV treatment and prevention). Although each partner discussed the frequency with which it refers clients for services, improved coordination in both staff and community training and awareness-raising activities would strengthen each partner’s respective program. The length of time required for refugees and asylum- seekers to obtain UNHCR cards is a major contributing factor to increased risk of abuse and has resulted in pervasive fear of extortion by the police, arrest, detention, and deportation.

Reporting and analysis of GBV cases is insufficient, and NGO implementers have little to no understanding of whether and how their work prevents GBV. There has been no risk mapping, and NGO implementers lack an overall picture of the risk, incidence, and prevalence of rape and other forms of violence among refugees in Kuala Lumpur. Without this information, NGO implementers lack the foundation to measure program performance and are unable to determine whether or not they are achieving program objectives such as GBV prevention. NGO implementers and refugees do not know where GBV is mostly likely to take place, and they lack a profile of perpetrators.

Recommendations for PRM

PRM should support provide financial support for an extensive situational analysis to understand the needs and priorities of refugee communities that program interventions seek to benefit. The situational analysis should include participatory assessments to identify the protection concerns of men, women, girls, and boys. Data collected should be disaggregated by sex and age. Situational analysis should focus on risk-mapping, immediate and root causes of GBV cases, and the use of international and domestic legal standards as a framework for analysis and action.
DoS/PRM/Washington should convene a working group that aims to create and implement a five-year plan across all GBV prevention programming. It became clear to the evaluation team that such a framework would be a welcomed initiative to all stakeholders in Malaysia. The framework should include: standardization of GBV definitions in accordance with IASC guidelines; plans for a baseline study to characterize types of GBV and their incidence and prevalence; exploration of possible research projects in partnership with UNHCR and NGO implementers to conduct in-depth examination of certain issues; and technical support and/or collaboration with M&E specialists to improve the development and reporting of GBV indicators.
PRM should fund partnership projects to address the gaps in knowledge about GBV. In order to develop and implement effective prevention measures and strategies, it is essential to obtain a clear understanding of the problem in a particular setting. International guidelines recommend periodic participatory assessments to gather information and understand GBV-related issues. Standardized tools are now available with the introduction of the GBVIMS. A ProCap or GenCap representative could assist in the development of new ways to measure GBV prevention and to coordinate this effort; UN Office for the Coordination of Humanitarian Affairs experts are free to UNHCR and will remain in-country for six months or more. Providing an advisor to assist the NGO implementers and UNHCR in developing a standardized, confidential questionnaire that includes questions about types of GBV, perpetrators, places where GBV occurs, etc. that can be distributed during home visits or health clinic visits could also help to address knowledge gaps. Other forms of participatory research, such as focus groups, are also recommended.
PRM and U.S. Government partners should engage Malaysian police, detention personnel, and immigration authorities to address issues related to their behavior toward (un)registered refugees and asylum-seekers. Previous U.S. Government interventions had significant results in terms of decreased deportations and trafficking by authorities. HEI, WAO, IOM, and Transparency International are potential NGO implementers for projects targeting police, detention personnel, and immigration authorities.
PRM, in conjunction with NGO implementers, should develop a livelihoods strategy for all GBV prevention programs to identify income-generating opportunities for at-risk populations.[17] PRM-funded NGO implementers should use the livelihoods-oriented GBV prevention strategy to engage and collaborate with livelihoods-focused NGOs—providing increased opportunities for refugees to strengthen their livelihoods as a mechanism to prevent GBV.
PRM should expand resources for ICMC’s GBV prevention programming. UNHCR supports this recommendation and stated “because ICMC has such a phenomenal relationship with the community and they are often our eyes and ears on the ground, they have the time to sit down and build the strong relationships and trust that we don’t have the time to do. ICMC is best placed to work on GBV prevention.” The evaluation team believes that ICMC cannot stand to lose funding given its exceptionally small staff and resources, which are already working above and beyond the funded mandate.
PRM should work with UNHCR to address the backlog and delay in refugee status determinations.[18]

Recommendations for NGO Implementers

NGO implementers should seek consistent, quarterly consultation with DoS/PRM/Washington and DoS/PRM/Bangkok, as well as with public health experts based in Kuala Lumpur, regarding the collection of GBV-related information that will be useful in risk-mapping and understanding GBV as it is experienced by refugee populations. For example, a confidential survey that conforms to standard research guidelines and best practices could be administered to a representative sample of refugees. By developing a set of standard “risk-mapping indicators,” PRM could standardize risk-mapping protocols across country programs for learning and program improvement.
NGO implementers should co-facilitate a GBV protection working group with UNHCR. Guidance for such a group could be similar to that used by the UN Global Protection Cluster approach and include guiding principles.[19] The group should focus on specific activities to prevent GBV and improve response and could leverage unique strengths of the NGO implementing partners to address identified problems.
NGO implementers should adapt the referral pathway developed by ICMC and WAO in shelter cases to address child sexual abuse cases.
HEI and ICMC should 1) combine efforts to train their refugee staff members (ICMC RWPC staff should receive HEI staff trainings on mental health, and HEI CHW staff should receive ICMC peer counselor and GBV awareness training); and 2) request UNHCR Refugee Status Determination staff to refer refugees with the requisite skills for RWPC and CHW positions.
HEI should 1) conduct information sessions with UNHCR, IOM, ICMC, and WAO to better inform them about its work and the research it has conducted; 2) institute GBV screening as a standard protocol for all clients; and 3) frame its stress and anxiety reduction activities in the context of GBV prevention efforts.
ICMC should provide all members of the RWPC with the 30-hour peer counselor training provided by WAO so they are prepared to handle the stresses they encounter in their work and to provide psychological first aid to members of their communities.

Evaluation Question 2

The lack of comprehensive needs assessments focusing on GBV or baseline assessments of program beneficiaries undermines the learning potential of PRM-funded programs in Malaysia. Information on GBV remains largely anecdotal and there is no accurate characterization of the problem at large. NGO implementers lack a clear understanding of the nature of GBV within the refugee community; this not only threatens the development and implementation of effective programs, but also hinders reliable understanding about the outcomes of GBV prevention programs. The lack of information for NGO implementers to target prevention efforts toward high-risk geographic areas and sub-groups results in a failure to translate knowledge into practical prevention strategies and reduces the effectiveness of GBV prevention programs. Baseline assessments are necessary to evaluate the effectiveness of programming, and mapping of available services is important to identify gaps and develop referral pathways for survivors. NGO implementers’ plans and efforts to develop data collection tools and better understand their beneficiary populations are laudable. However, to maximize the benefits of this information, the efforts need to be well coordinated and findings should be used to inform the design and implementation of current and future programs.

Recommendations

Programs that focus on healthcare, livelihoods, or other areas should be required by DoS/PRM to develop specific and measureable objectives that clearly relate to GBV activities. Additional mapping exercises of resources and GBV risk should be undertaken (ideally by ICMC due to the large scope of its activities across the refugee community in Malaysia).
PRM and UNHCR should request that ICMC and HEI collaborate to administer a survey (in light of their extensive access to different community members) to gather information about types of abuse, knowledge about GBV, and areas in the community that are not safe. PRM and UNHCR should support the administration of the survey, and findings should be used to inform programming.

Evaluation Question 3

PRM-funded GBV response programs in Malaysia appear to generally meet international guidelines, while GBV prevention programs fall short of international standards and have largely been limited to awareness-raising through didactic methods and one-off trainings. Although ICMC and the other implementers have made considerable progress in addressing the GBV “knowledge gap,” GBV prevention requires the transformation of socio-cultural norms in gender relations—a lengthy process of deepening inquiry and discussion over time. Little is known about the actual changes in knowledge, attitudes, and behaviors that have occurred due to the GBV prevention efforts of NGO implementers in Malaysia thus far.

Recommendations

DoS/PRM should make a concerted effort to develop and share a common understanding of what is meant by GBV prevention programming and ensure that NGO implementers funded by PRM are clear about prevention programming approaches.
NGO implementers should design, implement, and evaluate GBV primary prevention programs from a social norms perspective. Changing norms that legitimize GBV requires the collaboration of service providers across diverse sectors (e.g. health, protection, psychosocial, and livelihoods) to successfully engage influential individuals (e.g. husbands, religious leaders, youth, mother-in-laws, etc.) and groups (e.g. traditional leaders, religious leaders, peer groups) in identifying, discussing, and challenging social norms that legitimize GBV.
PRM should support the identification and dissemination of validated tools for measuring the impact of GBV primary prevention programs on changing social norms that legitimize GBV. Appropriate tools could include mapping risks of GBV, mapping networks of influential individual and groups to engage in changing norms that legitimize GBV, and tools to measure individual and community readiness for change in humanitarian settings.
PRM, in coordination with UNHCR, should provide NGO implementers with training and resources to address social norms that legitimize GBV. NGO staff and community members, including service providers across diverse sectors in humanitarian settings, require additional training and resources to build their capacity to understand norms in the community and develop the skills to discuss and challenge those norms with influential individuals and groups.
NGO implementers should hold regularly-constituted focus groups or discussion groups on GBV and other problems in the community to both promote awareness and create dialogue between refugees. ICMC’s RWPC program and HEI’s CHWs actively engage refugees and interact with community leaders using didactic approaches. Routine use of participatory methods of GBV-awareness raising and prevention would be beneficial.

Evaluation Question 4

NGO implementers are struggling with M&E, and the evaluation team was not able to effectively demonstrate GBV prevention program outcomes or learn from PRM-funded programs. In particular, the absence of standardized indicators used across PRM’s implementing partners prevents PRM from learning about the overarching outcomes of the GBV prevention initiatives that it funds.

Recommendations

DoS/PRM/Washington should provide PRM Regional Refugee Coordinators, many of whom are generalists, with information and guidance on M&E. Recommendations for GBV-specific indicators and M&E tools can then be provided by PRM regional offices to NGO implementers.
PRM should develop an internal results-based management system to support the implementation of its Functional Bureau Strategy, including a logic model that demonstrates the sequence of cause-and-effect relationships between activities, outputs, outcomes, and goals. The logic model could explicitly cover GBV prevention programs, or the logic model could demonstrate how GBV prevention should be integrated into all PRM interventions.[20]
PRM, in consultation with UNHCR, should disseminate required GBV M&E methodologies to NGO implementers. Required M&E methodologies should allow flexibility related to context while supporting the need for standardization of GBV indicators, timeframes, tracking of unintended positive and negative consequences, and staff accountability in humanitarian settings. Programs that focus on healthcare, livelihoods, or other areas should be required by PRM to develop specific and measureable objectives that clearly relate to GBV activities. Use of common M&E methodologies—including standardized indicators[21]—will enable PRM to make comparisons across settings about the impact of GBV prevention programs, thereby providing monitoring data for PRM’s internal results-based management system.
PRM should support NGO implementers to build capacity in required M&E methodologies. NGO implementers are using multiple methods for M&E as well as diverse GBV indicators within and across countries. A PRM-funded capacity building workshop would provide NGO staff with increased understanding of required M&E methodologies and important tools to collect and report evidence about the successes of GBV prevention programs in humanitarian settings.

Evaluation Question 5

NGO implementers do not appear to regularly consider unintended consequences of GBV prevention and response programming, despite the likelihood that both positive and negative unintended consequences abound.

Recommendations

NGO directors should encourage their staff members to consider and discuss possible unintended consequences of programming on a regular basis, perhaps in weekly staff meetings.
PRM may wish to raise the issue of the resettlement of divorced women with UNHCR to determine exactly how these cases are handled.
PRM should encourage NGO implementers to place more emphasis on outreach to refugees other than the Burmese, if resources permit.
PRM should discuss refugee complaints detailed in the Evidence and Findings section of this report with the UNHCR Refugee Status Determination unit.

Evaluation Question 6

In general, the occurrence of unintended program consequences was the result of not thinking through possible negative or positive consequences beforehand. Several specific contextual factors were identified as contributing to unexpected program outcomes.

Recommendations

NGO implementers should fully brief each woman entering a shelter about the precise conditions before agreement is reached that she will go to the shelter. It is important for someone familiar with her case and who has appropriate language skills to check on her daily (this policy is currently practiced by some NGOs).
Refugees who are working with GBV issues should be given ample time and space to discuss their own reactions to GBV and to explore their own discomfort as an initial reaction to a difficult problem. If they are not openly and explicitly asked to explore cultural and personal beliefs about GBV, they will be unlikely to be comfortable raising GBV issues with others.

Evaluation Question 8

There is openness among both NGO implementers and refugee communities to engage men and boys in GBV programming. However, most implementing partners have yet to identify specific engagement strategies. Programming for male survivors of sexual violence and GBV is limited and could be expanded to address unmet needs, especially with respect to boys and unaccompanied minors.

Recommendations

NGO implementers should develop engagement strategies that emphasize men as part of the solution, not the problem. Strategies should use positive messaging to encourage men and boys to develop their potential to act as agents of change. It is important to involve men who are influential in the community and held in esteem by younger men and boys. GBV prevention programs should engage men and boys where they congregate—at sporting events, religious gatherings, cafes, and even bars. It is also important that men and boys have a safe place in the community where they feel they can discuss sensitive issues and reveal their anxieties and vulnerabilities. Effective programs engage both males and females in separate age-appropriate groups with same-sex facilitators, later bringing the groups together for open discussion. Men and boys should have access to specially-designed assistance programs to meet their needs when they have experienced sexual violence. Establishing a shelter for boys in Kuala Lumpur is a good immediate step to address unmet needs; however, in the long term, a more coordinated effort is needed to support male sexual abuse and GBV survivors.

Evaluation Question 9

NGO implementers are not connecting their program goals (as proposed in requests for PRM funding) with how they understand their programs in practice. Across the board, NGO implementers struggled to verbally articulate their short- and long-term goals. However, program documents revealed a great deal about short-term goals of providing a protection response to GBV survivors and longer-term goals of bringing about changes in the community at large.

Recommendations

PRM should require logic models that link program goals to specific indicators and data collection methods (at the process, output, and outcome levels) as part of all NGO implementer proposals for funding.
PRM should require all NGO implementers to report quarterly and/or annually on all indicators specified in logic models, including on outcome measures.
NGO implementers should be encouraged to use information that is collected for M&E purposes for routine program management and decision-making purposes.

ANNEXES

Annex I: Evaluation Statement of Work

STATEMENT OF WORK

U.S. Department of State
Bureau of Population, Refugees and Migration

Evaluating the Effectiveness of Gender-Based Violence (GBV) Prevention Programs with Refugees in Chad, Malaysia, and Uganda

NATURE AND PURPOSE

The purpose of this solicitation is to obtain the services of a contractor to carry out an evaluation, lasting up to 12 months, of Gender Based Violence (GBV) programs supported either directly by the Bureau of Population, Refugees and Migration (PRM) or indirectly through one of its multilateral partners, the United Nations High Commissioner for Refugees (UNHCR) in targeted countries. The evaluation will consist of: (1) a comprehensive desk review and analysis of GBV program reporting by PRM and UNHCR; and (2) field-based evaluations in three countries (Chad, Malaysia, and Uganda) where PRM and UNHCR support GBV prevention programming. Both the desk review and the field- based evaluations should prioritize identifying: (1) the effectiveness of GBV prevention programming; (2) appropriate indicators for measuring the effectiveness of GBV prevention interventions in refugee settings and (3) best practices and lessons learned in engaging men and boys in GBV prevention interventions in refugee settings. Evaluation recommendations should include guidance that PRM can consider when: (1) writing requests for GBV proposals; (2) when reviewing GBV proposals; (3) monitoring GBV programs in the field; and (4) engaging host governments, International Organizations (IOs), and Non-Governmental Organizations (NGOs) on GBV issues. The contractor will coordinate with PRM, UNHCR, and NGOs.

BACKGROUND

PRM’s mission is to provide protection, ease suffering, and resolve the plight of persecuted and uprooted people around the world on behalf of the American people by providing life-sustaining assistance, working through multilateral systems to build global partnerships, promoting best practices in humanitarian response, and ensuring that humanitarian principles are thoroughly integrated into U.S. foreign and national security policy. PRM is the largest bilateral funder to UNHCR and other multilateral humanitarian responders. PRM funds NGOs to fill critical gaps in programming by UNHCR and host governments.

Preventing and responding to GBV in refugee settings is a PRM priority. PRM’s Multilateral Coordination and External Relations (MCE) Office oversees PRM-supported GBV prevention and response activities. Prior to FY 2010, MCE maintained a central pot of funding for GBV prevention/response programs. On an annual basis, MCE would issue a Request for Proposals (RFP) through which NGOs could apply for any region with PRM populations of concern. After FY 2010, MCE instead made the majority of these funds available to regional PRM offices, reserving only a small amount of central funding to promote research, capacity-building, and innovation concerning GBV prevention/response in humanitarian settings. For this reason, the scope of the evaluation will be projects carried out between FY 2010 to the present.

MCE is the main source of expertise on GBV related issues for the Bureau, complemented by technical assistance from USG partners such as the Centers for Disease Control and Prevention’s International Emergency and Refugee Health Branch (CDC/IERHB) and the United States Agency for International Development’s Office of Foreign Disaster Assistance (USAID/OFDA).

There seems to be an inherent challenge in measuring the impact of GBV programs, particularly where prevention activities are concerned. In a humanitarian context especially, GBV interventions tend to focus on health, legal and psychosocial response activities, given the urgency of the situation, funding constraints by donors (PRM generally funds activities 12 months at a time, for example), and the ability to measure impact more quickly, while the understanding of how to best support and measure the impact of GBV prevention activities in humanitarian contexts continues to be a challenge. As part of GBV prevention, PRM has raised the importance of determining how best to engage men and boys to reduce gender inequalities and prevent violence through questioning traditional norms associated with femininity and masculinity, and reinforcing positive masculine behavior, rather than behaviors that harm women. Although more has been done in the development context on this issue, the humanitarian community still has much to learn in identifying best practices on engaging men and boys in GBV programming. Strong monitoring and evaluation contributes to the identification of best practices that can be promoted in future GBV prevention and response programs, and we hope that this evaluation will identify appropriate indicators for measuring the effectiveness of GBV prevention interventions, as well as best practices on engaging men and boys in GBV prevention interventions in humanitarian settings. In addition to best practices, we should learn from mistakes that we and our partners have made so they are not repeated.

Monitoring the performance of PRM partners is a responsibility shared by MCE, regional offices, PRP and PRM’s Regional Refugee Coordinators based at embassies throughout the world. The Bureau’s Office of Policy and Resource Planning (PRP) will oversee administration of the evaluation and be the primary point of contact. Upon award, PRP will work closely with the contractor for the duration of the evaluation. In accordance with the standards of good management and performance-based results, the contractor will be held accountable for cost, schedule, and performance results.

SCOPE OF WORK

The contractor will:

Conduct a comprehensive desk review and analysis of selected NGO GBV projects supported by PRM and UNHCR between FY 2010-2012 with an emphasis on measuring the effectiveness of prevention interventions.
Carry out field-based evaluations in three countries where both PRM and UNHCR fund GBV prevention programs with refugee populations. For this study, the research sites would include refugee camps in eastern Chad, refugees living in settlements in western Uganda, and neighborhoods with high concentrations of urban refugees in Kuala Lumpur, Malaysia.
The evaluations should answer the following questions, with an emphasis on developing best practices, lessons learned, and actionable recommendations that can inform PRM supported GBV programming in the future.
- Did partners achieve the program activities defined in their project proposals? What were the barriers and facilitators to implementing program activities?
- Were the objectives of the program based on evidence such as needs assessments or other forms of data? Were they realistic, measureable objectives? If not, how can the objectives be improved?
- Did the GBV programming conform w/ internationally accepted GBV guidelines produced by the humanitarian community? Relevant guidelines include: (1) IASC Guidelines for GBV in Humanitarian Settings; (2) UNHCR Handbook for the Protection of Refugee Women; (3) UNHCR Guidelines on the Protection of Refugee Children; (4) GBV AoR Handbook for Coordinating GBV Interventions in Humanitarian Settings; and (5) IASC Gender Handbook in Humanitarian Action.
- Are the indicators in the above guidance documents (where available) appropriate for measuring the outcomes of PRM funded GBV prevention programs? Are the indicators in the project proposals Specific, Measurable, Achievable, Realistic or Timely? How can proposal indicators be improved? Do indicators from the above guidance documents effectively capture the impact of GBV prevention programs? Are some more useful than others and for what reasons?
- Were there any unexpected negative or positive consequences of PRM funded GBV programs? Did organizations address negative consequences and how?
- What factors explain intended and unintended negative or positive consequences?
- What outcomes did GBV awareness campaigns achieve? Are the indicators for these programs specific, measurable, achievable, realistic or timely? How can indicators be improved for GBV awareness campaigns?
- To what extent have men and boys been included in GBV prevention programs? If they were not included, why was this? If they were, what was the impact and how was it measured?
- What were the short and long term outcomes of PRM funded GBV prevention programs?

Fieldwork Component: Malaysia

International Catholic Migration Commission (ICMC): The objective of ICMC is to reduce the risk of GBV amongst the refugee women and children in Kuala Lumpur’s Klang Valley and surrounding areas and to improve access to emergency support service for survivors. ICMC is co-located with UNHCR on its Kuala Lumpur compound and partners with four local NGOs to meet its objectives. Beyond the local organizations, ICMC also coordinates with international agencies such as the International Rescue Committee (IRC) and the International Organisation for Migration (IOM) in identifying and assisting GBV survivors. ICMC and its partners provide emergency support services for GBV survivors, including emergency shelter and auxiliary services such as interpretation, transportation and psycho-social support; raise awareness about GBV related issues, and promote community involvement in preventing and responding to GBV, including with vulnerable children.
Health Equity Initiatives (HEI): HEI is a local organization based in Kuala Lumpur with the objective of improving mental health and medical services for Burmese and other ethnic minority refugees and asylum seekers, with an emphasis on victims of torture, forced labor, human trafficking, and exploitation. In order to do so, HEI builds the capacity of Burmese refugees to work within their communities on mental health issues; reinforces referral and case management systems in conjunction with UNHCR, other NGOs, and medical facilities; and conducts mental health outreach and screening in refugee communities.
Women’s Aid Organization (WAO): WAO is a local organization based in Kuala Lumpur which provides counseling for female GBV survivors on the UNHCR compound twice a week. It also maintains a shelter to protect women who are in danger. WAO is a sub- contractor with ICMC, along with Women’s Centre for Change Penang, MCTF Childline Malaysia, and Pusat Kebajikan Good Shepherd. WAO is also an important strategic partner for UNHCR and the two organizations have an MOU for referral of survivors for counseling and shelter-related services.

Annex II: Data Collection Instruments

Evaluation Questions Numbered for Coding of Interviews General/background questions to start interview

PRM/RRC/USG Other/International Org./UNHCR/NGO Grantees:

1. How did you become involved in the field of refugee protection?

2. What is the definition of GBV prevention that your organization uses?

3. To UNHCR: What kinds of “code words” are used by the community you work with to imply that someone has experienced GBV?

4. What information do you have about sexual violence/exploitation of men or boys in the communities you work with?

5. What do you know about GBV and gay/transgender members of the community?

6. Please provide a brief overview of the GBV prevention programming your organization supports.

7. What is your experience working on GBV either in the refugee community or otherwise?

8. What are the most critical aspects of GBV among refugees in Malaysia?

9. What do you know about the refugee situation in Malaysia? PRM’s work with refugees in Malaysia?

Direct beneficiaries/Indirect beneficiaries/Community leaders:

10. How long have you been in Malaysia?

11. Are you a member of a community group?

12. How long have you been a participant in this program?

Evaluation Directive 1: Effectiveness of GBV Prevention Programming for Individuals and Communities at Risk – findings, best practices, lessons learned

Evaluation Question 1a: Did partners achieve the program activities defined in their project proposals?

PRM/MCE (DC)

1.a.1. What are the main GBV prevention and treatment activities your grantees have proposed to carry out this fiscal year with PRM funding?

1.a.2. Are your grantees on track to achieve their proposed activities?

1.a.3. How do you determine whether or not your grantees have achieved the activities defined in their proposals?

1.a.4. Were there any changes to planned activities? If so, what were the changes?

1.a.5. How were you informed about these changes?

1.a.6. How did/have the changes affect program success/achievements?

RRC BKK

1.a.1 What are the main GBV prevention and treatment activities your grantees have proposed to carry out this fiscal year with PRM funding?

1.a.2. Are your grantees on track to achieve their proposed activities?

1.a.3. How do you determine whether or not your grantees have achieved the activities defined in their proposals?

1.a.4. Were there any changes to planned activities? If so, what were the changes?

1.a.5. How were you informed about these changes?

1.a.6. How did/have the changes affect program success/achievements?

UNHCR

1.a.1. What are the main GBV prevention and treatment activities your grantees have proposed to carry out this fiscal year with PRM funding? 44

1.a.2. Are your grantees on track to achieve their proposed activities?

1.a.3. How do you determine whether or not your grantees have achieved the activities defined in their proposals?

1.a.4. Were there any changes to planned activities? If so, what were the changes?

1.a.5. How were you informed about these changes?

1.a.6. How did/have the changes affect program success/achievements?

NGO Grantees/Implementers (HEI/WAO/ICMC/RWPC)

1.a.1. What are the main GBV prevention and treatment activities you have proposed to carry out this fiscal year with PRM/UNHCR funding?

1.a.2. Are you on track to achieve your proposed activities?

1.a.3. How do you determine whether or not you are making progress toward the achievement of the activities defined in your proposal to PRM/UNHCR?

1.a.4. Have you made any changes to planned activities? If so, what were the changes?

1.a.5. How have you informed UNHCR/PRM about these changes?

1.a.6. How did/have the changes affected the success/achievements of your program?

Mini-grantees/Community Orgs. (CELC/PKGS/WCC/MCTPF-Childline)

1.a.1. What are the main GBV activities you have proposed to carry out this fiscal year with ICMC funding?

1.a.2. Are you on track to achieve your proposed activities/objectives?

1.a.3. How do you determine whether or not you are making progress toward the achievement of the activities defined in your proposal to ICMC?

1.a.4. Have you made any changes to planned activities? If so, what were the changes?

1.a.7. How do these activities prevent GBV?

Direct Beneficiaries (including those who received GBV awareness training)

1.a.8. How did you learn about HEI/WAO/ICMC?

1.a.9. What is your understanding of what this program is/was intended to do?

1.a.10. What types of services do you receive from HEI/WAO/ICMC? (OR) What kind of program did you participate in?

1.a.11. How did the services meet your needs? Are there other services you would like to receive from this program? OR What did you learn from your participation in this program? (GBV Awareness)

1.a.12. How do you think that HEI/WAO/ICMC could improve its services?

1.a.13. How do you think violence against women/girls can be prevented in families/communities? Please give specific examples.

1.a.14. How does this program prevent violence against women/girls? Please give specific examples. Are there specific services received from the program that have helped to prevent violence against women and girls?

International Orgs./Other:

1.a.15. What can you tell us about the challenges refugees in Malaysia face in terms of GBV?

1.a.16. Are you aware of the programs/services provided by HEI/WAO/ICMC? If yes, what is your understanding of what the program(s) is/are intended to achieve?

1.a.17. How well do you think the program has met the objectives you have identified?

Evaluation Question 1b: What were the barriers to implementing program activities?

PRM/MCE (DC)

1.b.1. What are some of the challenges to program implementation that you observed among the grantees?

1.b.2. Do you know of instances where your grantees were unable to implement their programs or activities?

1.b.3. Can you provide a specific example of a program that was unable to implement its activities?

RRC BKK

1.b.1. What are some of the challenges to program implementation that you observes among the grantees?

1.b.2. Do you know of instances where your grantees were unable to implement their programs or activities?

1.b.3. Can you provide a specific example of a program that was unable to implement its activities?

UNHCR

1.b.1. What are some of the challenges to program implementation that you observed among the grantees?

1.b.2. Do you know of instances where your grantees were unable to implement their programs or activities?

1.b.3. Can you provide a specific example of a program that was unable to implement its activities?

NGO Grantees/Mini-grants/Implementers (HEI/WAO/ICMC/RWPC)

1.b.4. Did you experience any difficulties implementing your program activities? Please describe.

Mini-grantees/Community Orgs. (CELC/PKGS/WCC/MCTPF-Childline)

1.b.4. Did you experience any difficulties implementing your program activities? Please describe.

Direct Beneficiaries

1.b.5. Did you experience any difficulties in obtaining services from the program? Describe.

1.b.6. Did you talk to the staff about the problems you were having?

1.b.7. If you told the staff about the problems you were having, how did they respond?

1.b.8. If you did not tell anyone about the problems you were having , why not?

International Orgs./Other

1.b.9. Are you aware of any difficulties that beneficiaries might have had in accessing the services of ICMC/WAO/HEI?

Evaluation Question 1c: What were the facilitators to implementing program activities?

PRM/MCE (DC)

1.c.1. Can you identify some programs/activities that have been easiest for your grantees to implement?

1.c.2. What aspects of these programs/activities made them easy to implement?

1.c.3. Can you provide a specific example of a factor or characteristic that helped to facilitate program implementation?

RRC BKK

1.c.1. Can you identify some programs/activities that have been easiest for your grantees to implement?

1.c.2. What aspects of these programs/activities made them easy to implement?

1.c.3. Can you provide a specific example of a factor or characteristic that helped to facilitate program implementation?

UNHCR

1.c.1. Can you identify some programs/activities that have been easiest for your grantees to implement?

1.c.2. What aspects of these programs/activities made them easy to implement?

1.c.3. Can you provide a specific example of a factor or characteristic that helped to facilitate program implementation?

NGO Grantee/Implementers (HEI/WAO/ICMC/RWPC)

1.c.1. Which of your activities have been the easiest for you to implement?

1.c.2. What aspects of these activities made them easy to implement?

1.c.3. Can you provide a specific example of a factor or characteristic that helped you implement the activity?

Mini-grantees/Community Orgs. (CELC/PKGS/WCC/MCTPF-Childline)

1.c.4. Were there things that made it easier for you to implement your GBV prevention activities?

1.c.5. Were there things that made it easier or more possible to include men and boys in GBV activities?

1.c.6. Did ICMC take any action that made it easier for you to implement your activities? If so, please describe.

Direct Beneficiaries

1.c.7. Were there actions taken by (the NGO) to encourage your participation in the program or to make it easier for you to use their services? (Some examples might be providing interpreters, transportation, child care).

International Orgs./Other

1.c.8. Are you aware of any actions taken by HEI/WAO/ICMC/RWPC that have made it easier for program participants to access services? Please describe.

Evaluation Question 2a: Were the program objectives based on evidence such as needs assessments or other forms of data?

PRM/MCE (DC)

2.a.1. How did the grantees develop their program objectives?

2.a.2. What data or information did the grantees consult in the design of their program objectives?

2.a.3. Were the objectives informed by needs assessments?

2.a.4. If needs assessments were conducted, were they conducted specifically for the program or did they already exist?

2.a.5. What evidence exists to substantiate/support the need for the program?

RRC BKK

2.a.1. How did the grantees develop their program objectives?

2.a.2. What data or information did the grantees consult in the design of their program objectives?

2.a.3. Were the objectives informed by needs assessments?

2.a.4. If needs assessments were conducted, were they conducted specifically for the program or did they already exist?

2.a.5. What evidence exists to substantiate/support the need for the program?

UNHCR

2.a.1. How did the grantees develop their program objectives?

2.a.2. What data or information did the grantees consult in the design of their program objectives?

2.a.3. Were the objectives informed by needs assessments?

2.a.4. If needs assessments were conducted, were they conducted specifically for the program or did they already exist?

2.a.5. What evidence exists to substantiate/support the need for the program?

2.a.6. What kind of work do you think it would be good to expand?

NGO Grantees/Implementers (HEI/WAO/ICMC/RWPC)

2.a.7. How did you develop your program objectives?

2.a.8. What data or information did you consult in the design of your program objectives?

2.a.9. Did you conduct any needs assessments before designing your program?

-If so when?

-If so, who was included in your assessment?

2.a.10. Did you include men and boys in your assessment?

-If so, how?

2.a.11. What were the major findings of the assessment?

2.a.12. What other forms of information did you use to design your program?

2.a.13. Was the design informed by your prior or ongoing work in this area?

Mini-grantees/Community Orgs. (CELC/PKGS/WCC/MCTPF-Childline) FOCUS

2.a.14. Have you conducted any safety/risk mapping in your program? Please describe.

Direct Beneficiaries

2.a.15. What kinds of problems do you think led ICMC/HEI/WAO to decide to start this program?

2.a.16. Do you know whether HEI/WAO/ICMC conducted any safety or risk mapping for this program?

2.a.17. What types of violence or harm (psychological/emotional/physical) do women/girls face in your community?

2.a.18. What types of violence do boys or men in your community face?

International Orgs./Other

2.a.19. What types of needs assessments or other analyses were conducted prior to the planning and implementation of this program?

2.a.20. What suggestions would you have about modifying the objectives of the GBV program?

Evaluation Question 2b: Were the objectives realistic and measurable?

PRM/MCE (DC)

2.b.1. Do you think that the objectives are realistic or achievable within the timeframe of the project?

-If so, please provide a specific example?

-If so, do you have information or data that substantiates your position?

2.b.2. Are there indicators in place to measure the objectives?

2.b.3. If so, are the indicators appropriate measures for the objectives? RRC BKK

2.b.1. Do you think that the objectives are realistic or achievable within the timeframe of the project?

-If so, please provide a specific example?

-If so, do you have information or data that substantiates your position?

2.b.2. Are there indicators in place to measure the objectives?

2.b.3. If so, are the indicators appropriate measures for the objectives? UNHCR

2.b.1. Do you think that the objectives are realistic or achievable within the timeframe of the project?

-If so, please provide a specific example?

-If so, do you have information or data that substantiates your position?

2.b.2. Are there indicators in place to measure the objectives?

2.b.3. If so, are the indicators appropriate measures for the objectives? NGO Grantees/Implementers (HEI/WAO/ICMC/RWPC)

2.b.1. Do you think you will achieve your program objectives are realistic?

-If so, please explain why.

2.b.4. Do you think that you will achieve your program objectives within the timeframe of your project?

-If so, please provide a specific example.

2.b.5. Do you have any information or data to show your progress to date in achieving your program objectives?

2.b.6. Do you have indicators that you use to measure your progress in achieving your program objectives?

-If so, please provide some examples.

Mini-grantees/Community Orgs. (CELC/PKGS/WCC/MCTPF-Childline)

Direct Beneficiaries

2.b.7. Have you been asked your opinion about whether or not you think the program is effective?

2.b.8. What kind of things do you think would show that the program is working well?

2.b.9. How do you think the staff could tell if the program is working well?

International Orgs./Other

2.b.1. Do you believe that the objectives of this program relevant to GBV (ICMC/HEI/were/are realistic and measurable? (Discuss examples).

Evaluation Question 2c: If the objectives were not realistic and measureable, how could they be improved?

PRM/MCE (DC)

2.c.1. Please explain why the objectives are not achievable within the timeframe of the project.

2.c.2. Has the objective always been unrealistic or unachievable, or has there been a change in the project that affected the objective?

2.c.3. Please provide a specific example that demonstrates why/how the objective is unrealistic or unachievable. Do you have information or data that substantiates your position?

2.c.4. How would you change the objective to make it more realistic or more likely to be achieved within the timeframe of the project?

2.c.5. What could the grantee do differently to make the objective more realistic or more likely to be achieved?

2.c.6. What other factors would need to change to make the objective more realistic or more likely to be achieved?

RRC BKK

2.c.1. Please explain why the objectives are not achievable within the timeframe of the project.

2.c.2. Has the objective always been unrealistic or unachievable, or has there been a change in the project that affected the objective?

-Please provide a specific example that demonstrates why/how the objective is unrealistic or unachievable.

-Do you have information or data that substantiates your position?

2.c.4. How would you change the objective to make it more realistic or more likely to be achieved within the timeframe of the project?

2.c.5. What could the grantee do differently to make the objective more realistic or more likely to be achieved?

2.c.6. What other factors would need to change to make the objective more realistic or more likely to be achieved?

UNHCR

2.c.1. Please explain why the objectives are not achievable within the timeframe of the project.

2.c.2. Has the objective always been unrealistic or unachievable, or has there been a change in the project that affected the objective?

-Please provide a specific example that demonstrates why/how the objective is unrealistic or unachievable.

-Do you have information or data that substantiates your position?

2.c.4. How would you change the objective to make it more realistic or more likely to be achieved within the timeframe of the project?

2.c.5. What could the grantee do differently to make the objective more realistic or more likely to be achieved?

2.c.6. What other factors would need to change to make the objective more realistic or more likely to be achieved?

NGO Grantees/Implementers (HEI/WAO/ICMC/RWPC)

2.c.1. Please explain why your objective is not achievable within the timeframe of the project.

2.c.2. Has the objective always been unrealistic or unachievable, or has there been a change in the project that affected the objective?

2.c.3. Please provide a specific example that demonstrates why/how the objective is unrealistic or unachievable.

-Do you have information or data that substantiates your position?

2.c.4. How would you change your objective to make it more realistic or more likely to be achieved within the timeframe of the project?

2.c.5. What would you do differently in the future when developing project objectives to better-ensure that they will be realistic and achievable?

2.c.6. What other factors would need to change to make your objective more realistic or more likely to be achieved?