Completed Program Evaluations for FY 2003 - Strategic Goal 11: Public Diplomacy and Public Affairs

FY 2003 Performance and Accountability Report
Bureau of Resource Management
December 2003
Report

Strategic Goal 11: PUBLIC DIPLOMACY AND PUBLIC AFFAIRS
Title or Topic American Corners
Evaluator U.S. Department of State
Department Contact Cynthia Borys, Office Director, Bureau of International Information Programs
Major Findings
  • American Corners are the logical, market driven, cost-effective "reinventions" for the information age of the American library network that had been so popular abroad. For this new initiative posts identify a non-USG public space (libraries, universities, etc.) to provide a multifunctional programming platform to tell America's story, especially to the young. They are staffed by the host institution and allow interaction with the public without compromising mission security concerns.
  • There are now 79 American Corners abroad with another hundred in various stages of development.
Major Recommendations
  • From Changing Minds Winning Peace: A New Strategic Direction for U.S. Public Diplomacy in the Arab & Muslim World, October 1, 2003: "A rapid expansion of the scope of the American Corners program for local institutions should be undertaken, especially given the decreased access to American facilities."
  • Continue funding to help support the spread of American Corners through post initiatives.
Major Actions Taken or That Will be Taken
  • Develop accepted standards that can be modified to meet local needs and that allow for accurate measurements of success.
  • Regularize the sharing of information through a central clearinghouse to which 79 American Corners have already contributed data.
Expected Result
  • The development of fully functioning American corners can repair the harm cited in Changing Minds Winning Peace: " In the mid-1990s, after the fall of the Berlin Wall, the United States abandoned many of the tools of public diplomacy that had helped win the Cold War. ... Until the past decade, easily accessible facilities housed our public diplomacy activities ... (which) served as a venue for a wide range of activities, including reference services, book circulation and presentation, ...lectures, film and television screenings, ...satellite television reception and Internet access. ... The Advisory Group has heard abundant testimonials from foreign opinion makers on the positive impact of these institutions on their education and outlook in formative years."
  • American Corners can restore that public presence and the trust and understanding which they brought and which was lost over the past decade. They can go a long way toward providing an antidote to international hostility towards the U.S.
Title or Topic "hi" Magazine
Evaluator U.S. Department of State
Department Contact Christopher J. Datta, Assistant Deputy Coordinator, Bureau of International Information Programs
Major Findings
  • Launched in July, 2003, the monthly magazine "hi," and its associated website, seek to establish a constructive interactive dialogue with a young Arab readership as part of our broader agenda of developing civil society and engaging younger audiences. The magazine is designed to serve as a "window on America," highlighting American culture, values and lifestyles.
  • The magazine is for sale in Lebanon, Jordan, West Bank/Gaza Strip, Israel, Iraq, Morocco, Egypt, United Arab Emirates, Bahrain, Qatar, Kuwait, Oman, Yemen, Cyprus and Greece, with an initial print run of 50,000 copies. As of the end of FY 2003, we are still awaiting permission to distribute in Saudi Arabia, Syria, Tunisia, Algeria and Sudan.
  • Initial sales figures will not be available until January 2004 due to the absence of computer inventory sales systems in the Middle East. As a start up venture, we anticipate it will take a minimum of a year and a half to determine how well the magazine sells in the market. Even in the U.S., start-up magazines take two years to determine true market penetration.
Major Recommendations
  • Constantly review and adjust content with a view to attracting the widest possible audience.
  • Develop a strategy to engage Arab youth in a debate on issues of concern to them.
  • Enhance website capabilities to better respond to e-mail messages from readers.
  • Invest more in a marketing/promotion campaign in a bid to inform and attract more readers.
  • Send the publisher/editor, The Magazine Group, to the Middle East on an assessment trip and to develop new story ideas.
  • Develop and evaluate strategy to determine whether the magazine is accomplishing its goals through regular market studies and focus groups.
  • Develop a CD to be included in every issue of hi with games and information on the United States geared for 15- to 18-year olds.
Major Actions Taken or That Will be Taken
  • Marketing/Promotion campaign launched in late September 2003.
  • Additional marketing/promotion campaign to be organized in 2004.
  • Content to be enhanced by giving the magazine more depth, diversity and controversy.
  • Use focus groups or some other evaluation mechanism to ensure that the magazine is having an impact on target readership's perception of American culture and values.
Expected Result
  • hi magazine influences Arab youth to have a more positive perception of American culture and values.
  • Readership (paper and Internet versions) of the magazine grows.
Title or Topic INFOCENTRAL
Evaluator Computer Technologies Consultants, Inc.
Department Contact Joel Fischman, Senior Policy and Planning Officer, Bureau of International Information Programs
Major Findings
  • INFOCENTRAL, a joint initiative of the State Department's Bureaus of International Information Programs and Public Affairs (IIP and PA), was launched in FY2003 as a "one-stop" source of information, updated daily, for USG foreign affairs and security officials working with the public. The passworded (restricted access) website is the first single source ever developed for quick, up-to-the-moment official guidance, texts, reference sources and media reaction, readily available to the official foreign affairs community.
  • Currently, dedicated pages within the INFOCENTRAL site focus on Iraq, the War on Terrorism, Plan Colombia and Cuba. Over 2,500 subscribers are regular users to INFOCENTRAL, an impressive figure given the narrowly defined universe, and growing daily. INFOCENTRAL delivers instantly and efficiently, what overseas posts once had to ask colleagues in Washington to deliver.
  • INFOCENTRAL uses the evaluation software Webtrends to provide data on user access, demographics, usage, etc., for management analysis of the site.
Major Recommendations
  • A full time content manager ensures that all content in the relatively large website is kept current, critical for synchronized around the world accuracy in official public statements. INFOCENTRAL's continued success depends upon at least one permanent FTE for this function. This would have to be augmented by a surge capability, adding additional staff during times of crisis.
  • We need to examine other major international issues for INFOCENTRAL focus, including the Middle East peace process and reaching out to Arab and Muslim communities around the world.
  • Re-evaluate Webtrends to ensure we are getting sufficient data for informed site development.
Major Actions Taken or That Will be Taken
  • A search engine has been purchased to facilitate efficient user interface with the entire website and its contents. This will overcome the largest current weakness of INFOCENTRAL, providing for an aggregated search across InfoCentral content, including its links.
Expected Result
  • Informed officers in the field and in Washington, including those at high levels, with a menu of all the proprietary information they need to engage international publics at their fingertips. This is an efficient means of compensating for insufficient staffing.
  • InfoCentral subscribers are being provided with up to the minute information critical to the decision making process.
Title or Topic Persian Website
Evaluator U.S. Department of State
Department Contact William Armbruster, Deputy Office Director, Bureau of International Information Programs
Major Findings
  • In the few months since the Department's website in Persian was launched on May 12, we have evidence of a steady 2,000 viewers daily. We have reports that Iranian journalists monitor the site and have seen articles published based on material only available on the site.
  • With no official presence in Iran, the site serves in many ways as a virtual U.S. Embassy and cultural center for the growing number of Iranians who have Internet access.
  • The website includes both current policy texts and extensive publications on topics such as democracy, human rights, American institutions, and culture. Although the government in Tehran is critical of U.S. engagement in the Middle East and U.S. policies, its population is clearly open to learning more about the U.S., its democratic processes, open deliberation and free media and institutions.
Major Recommendations
  • Broader interest needs to be kept through increased posting of new materials and supporting graphics.
  • With additional funding, the Department could explore developing more creative and interactive Internet appropriate techniques that serve US interests while addressing specific questions from our audience.
Major Actions Taken or That Will be Taken
  • We are developing a system to look at readership for individual pages and categories of pages to better target our resources. We are beginning to translate major publications on the issues facing a new federal government and discussions on the meaning of democracy, human rights and how American institutions developed.
  • We are developing a companion listserv to the site.
Expected Result
  • A 40% Increase during FY 2004 in page views and placement of links and articles on other sites dealing with Iranian-American politics and Iranian discussions of democracy, human rights, and future political structures. Analysis will also be applied to the nature of users.
  • This site may well be the basis for a more open and unrestrained American-Iranian dialogue.
Title or Topic Content Management System for Public Diplomacy Websites
Evaluator U.S. Department of State
Department Contact Martha Chaconas, Acting Office Director, Bureau of International Information Programs
Major Findings
  • IIP's Content Management System project (CMS) is the vehicle by which IIP's core mission is delivered: "To get the right information, to the right people, at the right time". The CMS uses specially designed content management software to provide the delivery platform and infrastructure that facilitates the development, maintenance, and hosting of a centralized multilingual content management system to streamline the creation and delivery of high quality public diplomacy content for intended users.
  • Until the CMS was implemented, the process of using and repurposing information was labor intensive and highly inefficient -- economies of scale and leveraging resources were non-existent. The CMS now enables the efficient and effective creation, management and delivery of web-based content, currently over 70,000 web pages on the Department's international website (http://usinfo.state.gov) and enhances the timeliness of daily syndication of foreign policy news of almost 100 articles per day, to embassy affiliates (over 200 embassies) worldwide on a 24x7 operation. The CMS also enables users to create and deploy new products with the turn-around time demanded by quick response to world events.
Major Recommendations
  • By centralizing all publications and textual and graphical material and storing it in fluid XML format, we can offer increased efficiencies in producing electronic and print publications.
Major Actions Taken or That Will be Taken
  • Site conversion and new product implementations of Department and embassy websites, including foreign language websites, and new products.
Expected Result
  • Higher quality materials;
  • Increased workloads without the need for increased resource levels;
  • New products and services within days instead of weeks or months;
  • Delivery of critical information to the right sources in a fraction of the time it currently takes;
  • Syndicated content to embassies and missions thereby eliminating a significant part of their workload in repurposing content; and
  • A readily available infrastructure for embassy websites with easy access to ready-to-use content from a central data repository.
Title or Topic Outcome Assessment of the Institute for Representative Government (IRG)
Evaluator American Institutes for Research (AIR)
Department Contact Ted Kniker, Chief, Evaluation Section, Bureau of Educational and Cultural Affairs
Major Findings
  • IRG Alumni reported an increased understanding of U.S. democracy and democratic principles (80%) and the U.S. government and U.S. Congressional structure (73%).
  • 82% reported that the IRG program experience helped provide them with a more informed and broader perspective of the United States, and 60% indicated the program helped them develop a greater appreciation of alternative opinions and political processes.
  • Alumni indicated sharing what they learned about the United States and its government and people with professional colleagues (97%), family and friends (90%), and their community (85%).
  • Alumni who maintained contact with their delegation members (83%) and with people in the United States (76%) indicated these contacts helped them reinforce or sustain democratic practices in their country.
  • Alumni reported sustained contacts from the United States helped them inform their colleagues and community about the U.S. system of democracy (77%), initiate or develop democratic practices in their country (65%), and open or sustain political dialogue with the United States (60%).
  • Alumni reported successfully legislating or enacting policies at the national level (39%) and at the regional or local level (18%) that were partly influenced by what they learned during the IRG program.
Major Recommendations
  • Encourage and facilitate communication among program alumni.
  • Make the program schedule more flexible so participants can explore the United States on their own.
  • Consider a variety of factors in choosing program participants (delegates).
  • Encourage a more open atmosphere between delegates and U.S. professional contacts during lectures, presentations, and tours.
  • Make the program content more specific and less comprehensive.
Major Actions Taken or That Will be Taken
  • The IRG organization was being reorganized, following the death of the Executive Director, just as the evaluation was concluded.
  • The evaluation has been given to the IRG Board Members and to the re-organized IRG organization and program adjustments will be made based upon the recommendations cited above.
Expected Result
  • The evaluation results and recommendations are expected to inform the future directions and focus of the IRG program, improve the administration of the program, and contribute to the successful implementation of the IRG program in the future.
Title or Topic Evaluation of the Freedom Support Act (FSA) Undergraduate Program
Evaluator Aguirre International, Inc.
Department Contact Ted Kniker, Chief, Evaluation Section, Bureau of Educational and Cultural Affairs
Major Findings

Note: Each percentage is reported with alumni first, followed by semi-finalists and the random cohort of university undergraduates

  • Alumni give significantly stronger self-assessments on a battery of life skills associated with leadership abilities than either the semi-finalists or the random cohort.
  • Alumni are significantly more active in community service (43%, 30%, 20%) and significantly more likely to be employed (75%, 69%, 58%) following the program.
  • Alumni are significantly more likely to be working in the international arena (64%, 16%, 4%).
  • Alumni believe they have a much better understanding of U.S. politics (95%), culture (100%), and economics (89%) as a result of the exchange.
  • Alumni are significantly more likely to support equal rights for all citizens (74%, 71%, 61%).
  • Alumni are more likely to be in contact with Americans (64%, 48%, 15%), U.S. businesses (41%, 25%, 3%), and significantly more likely to use U.S. websites for information (93%, 76%, 29%).
  • Within the last year, 90% of alumni received a letter from an American friend and 76% received more than 10 letters. Twenty-four percent of alumni hosted one American visitor and 4% hosted more than 10 visitors.
Major Recommendations
  • Further review to ensure that the community colleague component of the program provides sufficient academic experience for participants.
  • When matching students to institutions, place a high priority on ensuring that the institution provides a strong program in the student's field of study.
  • As part of new alumni programming, consider involving program alumni as volunteers to assist the grant organization in planning program orientations and debriefings in order to better ensure participants are well prepared for the program and post-program return to their country.
  • Provide participants more pre-program English language training, with specific focus on those students with lesser competency.
  • Enhance communications and coordination between program coordinators and students/alumni to better ensure that the needs of students and alumni are being addressed.
  • Consider offering students an opportunity to express their living/housing preferences, while being clear about the options/limitations; be more selective with potential host families for homestays.
  • Provide more information on the U.S. health care system and practices during orientations, and develop/distribute informational brochures advising of common illnesses and medications.
  • Regarding the internship component, better guide students in the selection of their internship site and provide them with more information about the organization they will intern with; possibly develop a database of past intern organizations to incorporate vital organizational information.
  • Consider making the internships longer and to be undertaken when classes are not in session.
  • To ensure that the community service component is observed, program coordinators should check with participants periodically for the specifics of their community service activities.
  • Consider providing more support, resources and funding for alumni's professional development, job fairs and a job bank. In addition, consider incorporating semi-finalists in alumni activities.
  • Make an ongoing effort to ensure that more student participants will have the coursework completed in the United States recognized as formal academic study meriting college credit at their home institutions.
Major Actions Taken or That Will be Taken
  • Reviewed last year's community colleges being used, specifically looking at academic capacity, and recruited academically stronger community colleges for current year.
  • Reviewed host university/college's fields of study for potential new fields (non-duplicative of fields at home institutions); added criminal justice as a community college field and are better scrutinizing fields where material already received at home institution may be repeated in U.S. host institution.
  • Held re-orientation in-country seminars this summer for students returning home - to help them better assimilate back into their home countries/institutions. Alumni were used in recruitment for this fall's participants and in the pre-departure in-country orientations - to give current students a real view of the program and studying/living in the United States.
  • Continuation of providing additional English language training to students who need it, including ESL classes at the host or a nearby university/college.
  • A special program was started for alumni this year that allows alumni direct access to the alumni coordinator at the grantee organization in Washington, D.C. In addition, a print/electronic journal was developed to increase the outreach to both alumni and current students.
  • This year more students were placed in dorms versus host families. Host families are now required to sign diversity statements that they will respect students' religious preferences and cultures.
  • A list of common ailments and over-the-counter medication names have been included in the orientation materials.
  • Detailed information about internships is now provided via the FSA Ugrad listserv, including details about potential internship sites (also covered in the fall issue of the FSA Ugrad newsletter).
  • New reporting mechanism requires students to submit brief reports upon completion of their community service projects, including photos.
  • This past summer, semi-finalists were invited to attend Welcome Home Receptions for the returning students - hosted by the program organization and involving the Public Affairs Section.
Expected Result

Increased program administrative efficiencies through:

  • Increase in number and quality of community colleges.
  • Improved student-institution matching with a focus on placing students in fields of study not already available to them at their home institution.
  • Improved engagement and use of program alumni to better benefit both the alumni and current student participants.
  • Students will increasingly be better prepared to study and live in an English-speaking environment, and thus, acclimate better in their program and being in the United States.
  • Increased outreach to and engagement of current students and alumni through direct and indirect methods.
  • Improved selection and monitoring of host families and increased living/housing options for students.
  • More informed student participants regarding health/medicine.
  • More informed students in selected internship sites.
  • Increased monitoring of student compliance with the community service component.
  • Increased engagement of semi-finalists which also allows them to meet and network with program participants and alumni.
Title or Topic Evaluation of the Russia-U.S. Young Leadership Fellows for Public Service Program
Evaluator Aguirre International, Inc.
Department Contact Ted Kniker, Chief, Evaluation Section, Bureau of Educational and Cultural Affairs
Major Findings
  • Program alumni seemed to understand the important elements of civil society, but only 80% of Russians and 72% of Americans interviewed have continued their participation in public service.
  • Those that have continued their public service, however, appear to be very active, with 80% of those Russians and 70% of those Americans who have continued public service activities reporting themselves as more active than before the program.
  • 84% of Russians and 82% of Americans supported equal rights for all citizens regardless of ethnicity or circumstance.
  • Alumni overwhelmingly believe the program improves their leadership abilities (91% of Russians and 81% of Americans) and helps them acquire new skills for professional development (95%; 82%).
  • Almost all alumni (100% of Russians and 97% of Americans) agreed that they better understand the host country's culture as a result of the program.
  • The program successfully engenders lasting ties between the alumni and host country natives and institutions. Over 90% of Russians and 95% of Americans had monthly or more frequent email contact with friends in the other country.
  • Alumni taught their hosts about the culture of their home country and arguably reached a much wider audience than other programs that do not mandate significant public service.
  • Many alumni moved from their home communities following their program (42% of Russians and 45% of Americans), however, some Russian alumni who relocated were still maintaining contact with their home communities and several were sponsoring community-building projects.
Major Recommendations
  • Many Russian alumni recommended that the program result in the receipt of a diploma or a Master's degree. Alternatively, they would like to see the program expanded to a 2-year program so they can earn a Master's degree.
  • Review the selection criteria and the preparation of academic advisors; and consider allowing the students to choose the university that they will attend based on a "short list" provided to them by program organizers.
  • All participating universities should provide an in-depth orientation and debriefing session for participants to better prepare them for the start of their program and their transitioning back into their home country and culture. Develop an appropriate checklist of topics and issues that should be covered at each critical time period.
  • Re-examine the appropriateness of the computer science and business/corporate affairs fields of study options. Also, public and community service and internship organizations should be more carefully vetted to ensure that those chosen stress volunteer service in response to community and that the community service activities undertaken can be applied to the development of the public sector.
  • The community service and internship components of the program should be well planned, take place in an organization that complements the student's field of study, and clearly be designed as a service component to the program and not as work.
  • The American side of the program was not meeting all of its legislative and programmatic goals. Some changes would need to be made to the U.S. program to make it more useful and applicable.
Major Actions Taken or That Will be Taken
  • The decision to eliminate American participants from the program was made while the evaluation was in process - in part due to some of the preliminary findings of the evaluation.
  • Medical illness and over-the-counter medication names are now included in the orientation.
  • In-depth orientations were held in August 2003 for the Russian students. Some sessions focused specifically on what is community service and how to find community service projects in the United States.
Expected Result
  • With the elimination of the U.S. portion of the program, there will be a better focus on the Russian participants and their needs and the implementation of their programs.
  • Russian participants are better prepared for their U.S. program experience, and as a result, will have a more successful program.
Title or Topic Outcome Assessment of International Professional Exchange Programs in a U.S. Community (Philadelphia)
Evaluator T.E. Systems, Inc.
Department Contact Ted Kniker, Chief, Evaluation Section, Bureau of Educational and Cultural Affairs
Major Findings
  • 94% of U.S. host/resource respondents reported having "basic to advanced knowledge" about the culture and country of the foreign visitors immediately after the hosting experience, compared to 75% before the experience.
  • 80% reported that the hosting experience allows them to share U.S. values and beliefs with foreigners.
  • 86% of respondents shared information about their hosting experience with family, friends and/or colleagues, and as a result, family/friends (72%) and colleagues (67%) developed a greater appreciation of foreign visitors and countries.
  • 97% of respondents reported the hosting experience promotes mutual understanding among Americans and foreigners.
  • 87% reported the hosting experience increases their appreciation of foreign exchange visitors and countries.
  • 82% of respondents reported that hosting promotes peaceful relations between the United States and other countries.
  • For every dollar allocated in Fiscal Year 2001 by the State Department to support these exchange programs, an additional $2.64 was generated by the Philadelphia hosts/resources in monetary and in-kind contributions. This $1.00 to $2.64 ratio is the most conservative baseline calculation.
Major Recommendations
  • Develop a more comprehensive and thorough analytic methodology to better determine the economic impact on a local community.
  • Expand outreach activities while concentrating on a core group of host/resources.
  • Schedule more informal or social meeting time to promote interactions between foreign exchange visitors and U.S. hosts/resources.
Major Actions Taken or That Will be Taken
  • A follow-on study will be initiated in FY2004 to more rigorously collect, analyze and report on the economic benefits that these exchange programs bring to the Greater Philadelphia area. This methodology will then be applied to other communities.
  • A second follow-on study is in process in three Iowa cities to assess the impact of seven ECA exchange programs in a multiple-city geographic area. Lessons learned from the Philadelphia study are informing the design and methodology of the more complex study in Iowa.
  • A more streamlined host/resource database - comprised of updated contact information for the local hosts/resources located during this study - has been forwarded to the the International Visitors Council of Greater Philadelphia.
  • ECA's Office of Policy and Evaluation, the two ECA Program Offices involved in this study, and ECA's private sector partner organizations (the IVC of Greater Philadelphia and the National Council for International Visitors (NCIV)) have all been involved in a variety of outreach activities since the completion of this study in March 2003. Copies of the final report summaries have been distributed at national and regional conferences (NCIV conferences and NAFSA conferences), and information about this study and its findings has also been incorporated into ECA presentations at these conferences.
  • Study information and products have also been included in ECA promotional materials distributed at training sessions for a variety of ECA's private sector partner organizations. The study's products have also been widely disseminated to and used by the 95 Councils for International Visitors in approximately 45 states to better promote the three ECA programs included in this study (the International Visitors Program, Community Connections Program and the partially ECA-funded Sister Cities Program) and to improve the level of support for international exchanges, in general, in their communities.
  • Information from the study has been used to design follow-on exchange programming to further promote the interaction of American resources and visitors they host.
  • ECA is working with partner organizations to better track a wider range of private sector cost-sharing.
Expected Result
  • The more rigorous economic impact study in the Philadelphia area will provide ECA with better and more sophisticated evidence of the impact of ECA exchange programs on a community's local economy.
  • The expanded and more complex study in Iowa will provide ECA with key information about the impact of multiple ECA exchange programs on multiple cities in Iowa. This study will also allow ECA to better assess a broader view of why individuals, organizations and communities are supportive of international exchange activities in their communities and how ECA programs help advance international activities already in place in communities.
  • A more streamlined host/resource database will allow the IVC of Philadelphia to contact and engage their local constituents in a more efficient and effective manner.
  • Continued outreach activities and dissemination of study products by several ECA offices and ECA's private sector partner organizations will reach and engage a higher volume ECA and CIV stakeholders, and a broader portion of the general public.
  • ECA stakeholders and the general public are more informed of ECA exchange activities and the impact and benefits of these activities to U.S. communities, hopefully resulting in more support and understanding of ECA exchanges and/or international exchanges in general.
  • Further promotion of mutual understanding through enhanced exchange follow-on activities.
  • Increased cost-sharing and cost-leveraging from private sector organizations.