POINT OF VIEW

Colleges Need Better Ways to Assess Study-Abroad Programs
By JOAN GILLESPIE

In the wake of September 11, those of us who help students study abroad have been caught up in questions about the safety of our students and the long-term impact of terrorist attacks on our programs around the world. In fact, according to unofficial surveys, more than 100,000 American undergraduates studied abroad this year without incident, and few came home right after September 11. However, we still need to address another question that a number of programs and organizations had begun to ask before this past fall. It is an important question that provides the context for considering the future of study abroad.

What do we really know about the worth of these programs? Put another way: How do we evaluate them, including their provisions for student safety?

Ask a student about his semester in Spain. You'll hear about the color of the sky in Madrid or bullfighters. About the passion of Spanish professors and students for what they study, or a class that rekindled interest in Spanish literature and helped develop an intimate relationship with the language. Over and over again: My life has been changed.

The nature of that change, and the elements in a program that contribute to it, go largely undocumented, however, because formal assessment of study-abroad programs lags behind the assessment of other kinds of programs on college campuses. While institutions are likely to apply their own standards to their programs, no national standards exist to guide them. Perhaps international-education ventures have grown too quickly for formal evaluation to keep pace. Perhaps some institutions have not valued off-campus education as much as they have on-campus programs. Perhaps the exceptions to the rule of measurable standards have been so numerous in the case of study abroad that educators have simply savored the relief of not having to rigorously appraise them.

What is needed is a set of minimum standards that apply to every program, and comprehensive standards for different types of programs.

Granted, the unique character of international education, with its potential for both significant cognitive and noncognitive growth, demands a unique set of policies and procedures regarding student assessment. But it is also true that information gained from assessment would serve many of the same purposes as assessment on the home campus -- in particular, helping us to design programs to give students the maximum benefit.

The distinct questions include: What are intercultural competencies, those abilities and skills that distinguish an academic experience abroad from one in an American classroom? Are there reliable methods of assessing them? In bringing American standards of measurement to an international program, do we impose our expectations on those programs?

The starting point to finding answers is to remind ourselves of the factors that stimulated the assessment movement among U.S. colleges and universities more than 25 years ago. At that time, emphasis on defining quality in education began to shift from looking at the institution to looking at the student, from teaching to learning. In study-abroad programs, we already know enough to know that our focus needs to be on students as "intercultural learners" -- as opposed to academic tourists. (Of course, that doesn't mean that resources and a carefully designed infrastructure are not important.)

There are, indeed, already some guidelines that we can follow. Two of the regional accrediting groups, the Middle States Association of Colleges and Schools Commission on Higher Education and the Southern Association of Colleges and Schools, evaluate study-abroad programs as part of an institution's accreditation review. They ask about program objectives, methods of evaluating student performance, and types of follow-up studies after students return home.

At the Institute for the International Education of Students, a consortium of American colleges and universities that runs semester, full-year, and summer programs abroad for undergraduates, we have developed the Model Assessment Practice, a comprehensive process of evaluating student environment, resources for academic and student support, and student learning. In the section on student learning, we look at academic achievement (defined by mastery of course material, development of critical-thinking skills, and language proficiency), as well as at intercultural competencies (defined by measures of cognitive, personal, and interpersonal growth). Quality is measured by both qualitative data -- for example, students' awareness of culturally shaped knowledge -- and quantitative data, like the number and kinds of opportunities that expose students to the local intellectual and material culture.

In the categories of intellectual and personal development, it is particularly important to gather both types of data, testing the premise that international education enlarges students' abilities to recognize, understand, and respect cultural differences. Creative works like essays, videos, films, music, and selections of studio art can serve to document both intellectual and personal growth, thus bridging the academic and extracurricular elements.

One key part of our evaluation of academic achievement asks how much a program links its course work to fieldwork: Opportunities to engage their surroundings place students in a better position to reflect on cultural values and differences and to synthesize their insights with ideas they garner from written course materials. "Internationalization and globalization don't magically happen by sending ever-higher numbers of students abroad," William Anthony, a lecturer in the German department and director of the Study Abroad Office at Northwestern University, writes in an IES MAP manual. "They take place in the minds of individual students as a result of challenging and active academic study and interaction on site."

Language development, another set of standards, is the primary reason for many students to study abroad. So an assessment of language-intensive programs must evaluate those skills. We recommend quantitative measurement of linguistic progress through entry and exit tests, like the oral-proficiency interviews developed and conducted by the American Council of Teachers of Foreign Languages or the language-examination systems administered by the Association of Language Testers in Europe.

When we offer some, or all, of our courses through institutions abroad, we confront another issue: whether we accept the grades, standards, or assessments of that institution as part of a student's transcript at our own college, or whether we record the credit simply as "pass/no pass." Some U.S. institutions that practice the former believe that students take a program more seriously if the grade is recorded, while those that practice the latter want to reward students who take up the challenge of study abroad by not punishing them with potential low marks. However, just recording credit as pass/no pass does eliminate a quantitative measure of assessment.

Other practical questions must also be addressed: Are there any core experiences that every student in a study-abroad program should share? What level of integration in the host culture is ideal? What is possible? How can a program compensate for beginning-language skills?

I'm not suggesting that we must have one single set of standardized criteria for assessing study-abroad programs. Programs simply vary too much, and assessment methods must reflect those differences. What is needed, however, is agreement in the field on a set of minimum standards that includes recommendations for student assessment.

A well-constructed assessment program should make clear to students what they can expect and what we can expect from them. It also would make advising easier. The major responsibility of on-campus advisers is matching a student with a program that promises to meet his or her goals, then ascertaining that the program makes good on that promise through a reliable method of quality control. And finally, by building assessment into educational policy, we would announce a commitment to long-range planning and continuous improvement. Our programs have grown enormously in popularity and variety. Now it's time to build and adopt the tools that will demonstrate their integrity.

Joan Gillespie is assistant vice president for academic programs and program dean at the Institute for the International Education of Students. 

http://chronicle.com
Section: The Chronicle Review
Page: B20