Assessing Intercultural Competence in Experiential Learning Abroad: Lessons for Educators
ANDREA PARAS & LYNNE MITCHELL
University ofGuelph      University ofGuelph
Abstract. The old adage that travel broadens the mind may not hold true for every student studying abroad. While pre-departure and in-country support for intercultural learning can help, some students still fail to develop their intercultural competence and some even go backwards. Using a combination of quantitative (Intercultural Development Inventory) and qualitative measures, this study examines what happens in intercultural learning when students participated in a four-week experiential field school in India, preceded by a twelve-week preparation course on the ethics of international voluntourism. Results found that, while students' pre- and post-trip Intercultural Development Inventory results varied, qualitative data gave insights into student learning and revealed important lessons for educators.
One might assume that taking a group of students on a community-engaged, experiential learning course abroad would result in considerable intercultural learning, an increase in cultural sensitivity and the general development of intercultural awareness in participants. But does the age-old assumption that travel broadens the mind really hold true? As many institutions bolster their overseas programs, it is important to challenge the assumptions that surround an overseas experience. Does simply being in another country increase students' intercultural competence? Are some
© 2017 Southern Utah University Press & Design.
Correspondence should be sent to Andrea Paras, Department of Political Science, University of Guelph, and/or Lynne Mitchell, Centre for International Programs, University of Guelph, 50 Stone Rd. E., Guelph, ON NIG 2W1, Canada. Email: aparas@uoguelph.ca and/or lmitchel@uoguelph.ca
ELTHE: A Journal for Engaged Educators, Vol. 1, No. 1 (2017), pp. 45-64
Paras & Mitchell
students better positioned by their previous experience, learning style, or attitude to glean more intercultural competence from an academic sojourn abroad? What do educators need to know to nurture intercultural competence in combination with other discipline-based academic learning?
This study examines what happened in intercultural learning when students participated in a four-week experiential field school in India, preceded by a twelve-week preparation course on the ethics of international voluntourism. It builds on a previous, unpublished study conducted by the Centre for International Programs at the University of Guelph in 2011, which showed that, without intervention, most students make few gains in their intercultural competence, and some even go backwards in intercultural scores after a four-month study abroad experience (Blenkinsop & Mitchell 2011). This is in keeping with other studies which found that, without guided intercultural learning, students can return from a program abroad with, at best, very little increase in intercultural sensitivity or, at worst, reinforced negative stereotypes and strengthened ethnocentrism (Bateman 2002, Hammer 2012, Jackson 2008, Patterson 2008).
In an era where university programs claim to be producing global citizens and where a more globalized world makes intercultural skills a necessity (Gambino & Hashim 2016), educators need to carefully consider how to purposefully develop intercultural skills in learners regardless of discipline. We would be foolish to assume that intercultural skills can be obtained by mere immersion (Vande Berg, Paige, & Lou 2012). As Mitchell (2013) points out in the Canadian Bureau for International Education's 2013 report, we wouldn't lock a group of undergraduates in a microbiology lab for four months on their own and expect them to emerge having made major scientific discoveries, so why do we think osmosis is an appropriate method of instruction when it comes to intercultural skills? If we want to intentionally develop intercultural competence in students who study abroad, we need to know how students view culture and cultural issues and how different students learn differently when it comes to intercultural competence.
46
ELTHE: A Journal for Engaged Educators
Assessing Intercultural Competence
Methodology
This study utilized a University of Guelph community-engaged experiential learning program called the India Field School as a vehicle to examine the development of intercultural competence in senior undergraduate students. The majority of student participants were International Development Studies majors, in addition to one student each from Environmental Governance, Geography, and Criminal Justice. The India Field School consisted of two parts: a pre-departure preparation seminar and a four-week immersive field school. During the pre-departure seminar students examined the ethics of international voluntourism within the context of broader critiques of international development. A significant portion of the pre-departure seminar was also devoted to understanding the concept of intercultural competence and providing students with a toolkit of reflection skills. During their time in India, students worked in full-time volunteer positions at a variety of Tibetan and Indian NGOs in Dharamsala, which included a range of human rights and development organisations. The students also had the opportunity to interact with a number of guest speakers and visit numerous cultural sites.
Our research employed a mixed quantitative and qualitative methodology that analyzes students' Intercultural Development Inventory (IDI) survey results alongside approximately seventeen written reflections from each student.1 All students were required to take two IDI surveys, as well as submit written reflections, as part of their coursework. Out of fifteen enrolled students in the class, there was a high participation rate in the study, with thirteen students consenting to participate. The investigators also collected general background information from the students, including program of study and semester level.
The originator of the IDI survey, Mitchell Hammer, defines intercultural competence as "the capability to shift cultural perspective and adapt behavior to bridge cultural difference" (Hammer 2012, 116). The IDI survey is a psychometric instrument consisting of fifty questions designed to measure intercultural competence as a set of knowledge, skills, and attitudes along a development continuum. Along this continuum (depicted in Figure 1), Denial and Polarisation signify monocultural mindsets, Minimisation is a transitional phase, and Acceptance and Adaptation signify more intercultural mindsets.
Vol. 1, No. 1 (2017) 47
Paras & Mitchell
Figure 1. IDI Stages of Intercultural Development (IDI LLC 2015)
The survey measures individuals' perceptions of how in terculturally competent they perceive themselves to be (their "Perceived Orientation"), as well as their actual level of intercultural competence (their "Development Orientation"). The difference between the former and the latter is the "Orientation Gap."
Data collection took place over a period of six months, as depicted in Figure 2. The IDI survey was administered to each student within two weeks of the beginning of the pre-departure seminar (the "pre-IDI"), then again within two weeks after the completion of the in-country field school (the "post-IDI"). After the completion of both IDI surveys, students received their individual results and aggregate class results. After the completion of the first IDI, a qualified IDI survey administrator visited the pre-departure seminar to provide a full debriefing about the aggregate results, and students had the opportunity to meet with her individually to get further clarification about their individual results. After the completion of the second IDI, there was no opportunity for a group IDI debriefing, although students had the opportunity to contact the IDI survey administrator about their results. The results of the two IDI surveys were analyzed to identify whether there were any changes in the students' Perceived Orientation, Development Orientation, and Orientation Gap at the beginning of the program compared with the end of the program.
48
ELTHE: A Journal for Engaged Educators
Assessing Intercultural Competence
Figure 2. Data Collection
In Canada
T
Pre-trip Seminar Course N=13
6 in-class reflections
6
6 take-home reflections
IDI administered and group debrief
In India
In-country Field School Course N=13
In Canada
T
Final Field School Requirements
Final post-trip reflection
>-
IDI administered
The written reflections were submitted by students both in-class and as formal assignments. Twelve of the reflections were submitted during the pre-departure seminar, four of the reflections were submitted during the in-country field school, and one final course reflection was submitted approximately a month after the completion of the program. NVivo qualitative data analysis software was used as the primary tool for analyzing the written reflections. In order to protect anonymity, all of the students were given pseudonyms after their reflections were uploaded to NVivo.2 The reflections were then coded using a combination of deductive and inductive coding. For the former, the reflections were coded for the intercultural competency orientations identified by the IDI survey: Denial, Polarisation (Defense and Reversal), Minimisation, Acceptance, and Adaptation. The investigators also employed inductive coding whereby they identified recurring themes that were not captured by the IDI intercultural competency orientations. After the coding of the written reflections was complete, each student's IDI survey result was compared with the coded reflections. Withholding the IDI survey results from the researchers until the end of the qualitative data analysis ensured that results did not unduly influence the investigators during the analysis and coding of the data.
Vol. 1, No. 1 (2017) 49
Paras & Mitchell
Figure 3. Aggregate Pre- and Post-IDI Orientations
Pre-IDI
Post-IDI
[| Perceived Orientation
Development Orientation
55     70    85   100   115   130 145
Using the IDI to Assess Intercultural Competence
The two sets of IDI results suggest that, in the aggregate, both the students' Perceived Orientations and their Development Orientations increased as a result of their participation in the field school (Figure 3). In the pre-IDI survey, students perceived themselves to be at an intercultural mindset of Acceptance (125.32 points), whereas their actual orientation was at the lower transitional phase of Minimisation (105.15 points). After completing the field school, students perceived that their orientation had increased to the highest intercultural mindset of Adaptation (130.31 points), whereas their actual orientation had increased by a statistically significant amount to the border area between Minimisation and Acceptance (113.75 points). In other words, the aggregate results for the class optimistically suggest that the field school helped students to improve their intercultural competence, although students still perceived themselves to be significantly more competent than they actually were.
Nevertheless, when we look more closely at students' individual results, we can see that changes in individual students' IDI orientations are uneven. In Figure 4, we see that seven out of thirteen students experienced a statistically significant improvement in their Development Orientation, two experienced a decline, while the remaining four did not change at all.4 The results suggest that, even with extensive pre-departure cultural preparation, educators cannot assume that students will improve their intercultural competence when they study abroad. Interestingly, five out of thirteen students also experienced a
50
ELTHE: A Journal for Engaged Educators
Assessing Intercultural Competence
Figure 4. Individual Pre- and Post-IDI Orientations
	Major	PO-Pre	PO-Post	Change in	DO-Pre	DO-Post	Change in DO	Change in OG
AN	IDev	Acceptance	Adaptation	2.07	Minimisation	Minimisation	-1.93	3.01
Bailey	IDev	Acceptance	Acceptance	12.09	Minimisation	Minimisation	23.882	-11.79
Bobo	Geography	Adaptation	Adaptation	2.47	Minimisation	Acceptance	3.05	-0.58
Casey	Env. Governance	Acceptance	Adaptation	9.32	Minimisation	Minimisation	22.731	-13.41
Devin	IDev	Acceptance	Acceptance	7.65	Minimisation	Minimisation	20.58	-12.93
Jaime	IDev	Acceptance	Acceptance	-1.35	Acceptance	Minimisation	-13.99	12.64
Jazz	IDev	Acceptance	Adaptation	12.48	Minimisation	Acceptance	29.225	-16.74
Kennedy	Criminal Justice	Acceptance	Acceptance	10.5	Minimisation	Minimisation	13.79	-3.29
Logan	IDev	Acceptance	Adaptation	5.5	Minimisation	Minimisation	14.48	-8.98
Morgan	IDev	Acceptance	Acceptance	-1.26	Minimisation	Minimisation	-10.858	9.59
Rory	IDev	Adaptation	Adaptation	-2.49	Acceptance	Acceptance	-4.385	1.9
Terry	IDev	Adaptation	Adaptation	0.78	Acceptance	Acceptance	-3.577	4.35
Tyler	IDev	Minimisation	Acceptance	4.73	Polarization	Minimisation	9.089	-4.35
PO = Perceived Orientation DO = Development OG = Orientation Gap
statistically significant change in their Perceived Orientation, all of which were increases, which suggests that participation in a study abroad might actually augment the tendency of some students to over-estimate their intercultural skills. What mitigates this finding, however, is that four of the same five students also reduced their Orientation Gap. In other words, even though their perception of their own competence increased, these students were still able to arrive at a more realistic assessment of their intercultural capacities. Jazz, for instance, had both the largest increase in Perceived Orientation, as well as the most significant narrowing of her Orientation Gap. Finally, we see that academic program was not a predictor of intercultural learning. The International Development Studies majors, who comprised the majority of the class and who might have been predicted to have the largest increases in intercultural learning based on the international focus of their academic subject matter, experienced both significant increases and declines in international competence. Overall, the individual IDI results demonstrate that intercultural learning during study abroad takes place along multiple dimensions.
However, it is impossible to tell the entire story about the students' learning during the field school simply by looking at the numbers contained in their pre- and post-IDI survey results. While the IDI survey is a powerful
Vol. 1, No. 1 (2017) 51
Paras & Mitchell
tool that enables us to see where changes in intercultural competence occurred, as well as provides information about the nature of that change, it does not provide any information about why or how it took place. In order to gain a more nuanced and deeper understanding of student learning, we must turn to an analysis of the students' written reflections. These provide us with greater qualitative insights about the students' learning processes as they grappled with various experiences during the India Field School.
Using Written Reflections to Assess Intercultural Competence
Using the students' reflective writing samples, we are able to get a much richer understanding of what contributed to the development of students' intercultural competence, as well as identify areas of learning that are not adequately captured through the IDI survey. The reflections demonstrate that students perceive they have an understanding of what intercultural competence entails and what is required to develop it. Nevertheless, studying a flight manual or understanding how an engine works doesn't mean you can fly an airplane. Likewise, theoretical knowledge about intercultural competence does not necessarily translate into practice. Even though students understood what was required for them to improve their intercultural competence and perceive that they have those skills, they frequently faced challenges in knowing how to behave in unfamiliar contexts or challenging situations, thus reflecting the gap between their Perceived Orientation and their Development Orientation. As Rory observed in one of her in-country reflections, "Before, I believed that simply being aware of these challenges would help me to walk around them. Instead, I walked straight into them and became quite stuck!" It was incredibly disorienting for the students who had already engaged in extensive pre-departure preparation to arrive in India and still find themselves struggling with challenges they had anticipated being able to address or circumvent. The sense of disorientation was heightened by the emotional intensity of being in a new and unfamiliar intercultural context. Continuing with the above metaphor, it is one thing to read about pulling an airplane out of a nose dive, but quite another matter to actually do so. In such cases, it was not enough to assume that the pre-departure seminar, or their prior knowledge about intercultural competence, would
52
ELTHE: A Journal for Engaged Educators
Assessing Intercultural Competence
provide the students with the appropriate supports and resources they will need to address challenging intercultural situations. Rather, in order to turn these challenges into valuable learning opportunities, in-country support was necessary to build on the information and training that students received prior to departure.
Often, it was in these kinds of contexts during the field school that students' Development Orientation, that is, their actual level of intercultural competence (as opposed to their perceptions of their competence), was demonstrated in their reactions to such situations. For instance, on one occasion, the students had an opportunity to attend an important religious teaching by the Dalai Lama at a monastery in the valley close to Dharamsala. The temperature was hot, the students were unused to sitting on the ground, the radios that provided English translation did not work properly, and most of the students chose to leave within thirty minutes of the start of the teaching. In a debriefing the following day, the course instructor engaged the students in a conversation about how the early departure may have been insensitive to local cultural and religious norms. Kennedy reflected after the incident,
While I do believe I have gained more insight into how I may go about adjusting my behaviour in various settings, there were still times during the India Field School where I felt disoriented and unsure how to proceed. The primary example that comes to mind is my early departure from the Dalai Lama's teaching. I did not fully account for how my actions may be perceived by a member of the Tibetan community. What I did however consider is that I could appreciate the gravity of the situation and how much it meant to Tibetans to listen to the Dalai Lama. This is an example of Acceptance over Adaptation.
As the student mentions, one of the features of an Acceptance Orientation is that it is possible to identify and appreciate cultural differences, but often people with this orientation do not know how to shift or adapt their behaviour or perspective in culturally appropriate ways. Likewise, individuals with an Acceptance Orientation often find it difficult to deal with moral differences or dilemmas. For instance, Rory had prior experience working in sexual health education in Canada, but she found it difficult to mediate between her own values and the abstinence-focused campaign of the organisation with which she volunteered in Dharamsala. In her efforts to improve her orientation from Acceptance to Adaptation, Rory struggled with how to shift her perspective and behaviour without compromising her own beliefs.
Vol. 1, No. 1 (2017) 53
Paras & Mitchell
In another example frequently cited in reflections, students wrote about their discomfort at being photographed by other Indian visitors to the Golden Temple in Amritsar, which is one of the most important sites in the Sikh religion. For instance, Kennedy expressed her discomfort at the attention: "How [is] one to prepare for potentially having their picture taken while doing seemingly regular day-to-day activities?" In this example, by equating her presence as a tourist at the Golden Temple with "regular day-today activities," she demonstrates a Minimisation Orientation response to the situation. In other words, she minimises the presence or relevance of cultural difference, rather than shifting her perspective to identify different cultural norms around personal space or what actually entails a regular daily activity in Amritsar {i. e., it might not be a regular daily activity for locals to see a large group of Western university students in the Golden Temple).
Throughout the field school, students identified a wide range of challenges in their reflective writing, which ranged from logistical challenges related to unfamiliar living conditions to more complex challenges related to understanding culturally appropriate behavior or dealing with morally sensitive topics. The three top-cited challenges included (1) fear of ineffectiveness as a volunteer; (2) dealing with the gap between expectations and reality; and (3) lack of ability to understand or communicate. All of these had the potential to paralyse students, particularly in the context of their volunteer placements. For instance, Casey writes,
It has also been brought to my attention that being a native English speaker is probably very beneficial for these students. While I agree with this and see the merit of this, I wonder, is this enough? Is it enough to teach these children English because I have been speaking English for my whole life? I do not think that this is enough. Is there also an assumption around the colour of my skin? That because with the colour of my skin there is an assumption of my level of education and with the assumption of my level of education there is an assumption of my ability, in this case, in teaching? Is this an ignorant question?
It is plausible that, without studying the limitations of short-term volunteerism prior to departure, students would not have been attuned enough to the potential pitfalls or limitations of international volunteer
54
ELTHE: A Journal for Engaged Educators
Assessing Intercultural Competence
work to identify these complex dynamics in their own placements. That the students had the capacity to apply the theoretical knowledge they had gained through the seminar to help them identify complex practical issues within their own placements should be considered strong evidence of learning.
Nevertheless, while the pre-departure seminar helped the students to identify some of the challenges and limitations of volunteer placements, the students varied in their abilities to transform these challenges into learning opportunities. In his contributions to transformative learning theory, Mezirow (1991) has argued that disorienting dilemmas are a crucial component of experiential learning, which are defined as problems that awaken curiosity because of their intrinsic importance to the learner, and which produce levels of perplexity, doubt, or disorientation (Perry, Stoner & Tarrant 2012). When knowledge is combined with hands-on learning, Mezirow argues, reflection can help students make meaning of their experiences in potentially transformative ways (1997). However, what is required for this to happen is that students experience the discomfort of the disorienting dilemma and then move through stages of reflection that enable the students to integrate new perspectives and re-frame their learning objectives (Perry, Stoner and Tarrant 2012). Alternatively, students may become paralyzed if they cannot successfully integrate their reflections or have no opportunity to do so.
In some cases, students in the field school were able to recognise and accept that deep learning is often accompanied by deep discomfort. Two students, Casey and Bobo, felt inadequately prepared for their positions teaching English to children at a local school, especially in light of critiques that they had previously read about the negative impacts of short-term volunteers working with schoolchildren. Because they had found themselves in the position of potentially replicating problematic volunteer practices, they worked hard to find other ways to contribute to the organization and determined that they could be more useful writing grant templates for the school administrators. Casey reflects,
While I did not prepare for a full day of teaching, or the level of discomfort and disorientation, I realize that a challenging experience is perhaps a more beneficial learning experience, and that I have to make the best of my experience, for both myself and the school.
Vol. 1, No. 1 (2017) 55
Paras & Mitchell
Casey and Bobo were able to turn this uncomfortable situation into an opportunity to learn about how short-term volunteers can provide more sustainable forms of assistance.
Likewise, Bailey reflected on how her whiteness became an uncomfortable marker of her identity in a way that she had previously never experienced,
On the airplane on the way to India, I was aware of the fact that I was one of the only white people on the plane. This was perhaps the first time that that experience was so obvious to me, and it created a huge learning experience for me while we were in India—I was constantly struggling with the implications of skin tone. By this I don't just mean for me, I mean that if I could feel so uncomfortable at a place like the Golden Temple, where the environment surrounding the fascination with our whiteness was not hostile in the least, how did minorities in Canada feel all the time? How was it that I could come to India and be the minority, and still only experience my skin tone as giving me power and privilege?
Here, Bailey was able to use her own uncomfortable awareness of her visible cultural identity markers to come to a deeper understanding of how privilege functions both in India and back home in Canada, thereby integrating a new perspective as a result of this transformative learning experience.
Although no course instructor wants to see her students fail, especially since there are potentially high costs for failure in study abroad contexts for both students and local hosts, it is important to remember that even failure lends itself to learning. For example, Rory candidly remarks on how the group's failures contributed directly to her learning about the limitations of short-term international volunteerism,
Our group came in with the best intentions in particular areas and failed. We were loud, took up too much space, insulated ourselves, and did not fill in our blind spots. However, in seeing these failures come from a group dedicated to NOT perpetuating such harms, I've been challenged to consider a different
perspective on development and service learning overseas____     Sometimes, good
intentions are the catalyst needed to propel oneself to check their privilege, push their comfort zone, and move toward more accountable allyship. Sometimes, good intentions are used as a cop-out.
56
ELTHE: A Journal for Engaged Educators
Assessing Intercultural Competence
In all of the examples cited above, it was necessary for the students to reframe their pre-existing expectations and assumptions, as well as look beyond the details of their own discomfort to recognise the broader implications of their participation in an international volunteer program. We can see that students were able to use the "disorienting dilemmas" they encountered during the field school to enhance their theoretical and practical learning.
In other cases, some students reacted to the disorienting effects of a challenging placement with less capacity to transform the dilemma into a learning opportunity. For example, in one of her in-country reflections, Morgan expressed this heightened level of disorientation:
We learned enough in class prior to our departure about the downfalls of short term volunteering to make me skeptical about my ability to actually be useful, or
really be needed at my placement____I find that I am really questioning whether
or not I was actually useful, even though I completed my assignment, and if I wasn't useful does this mean that any short term volunteer program will ever be useful?
While having more questions than answers can be interpreted as a sign of learning {i.e., "the more you know, the more you don't know"), Morgan found herself closer to a place of paralysis when considering how she could circumvent the limitations of short-term volunteerism. She had less success in re-framing her own learning objectives or shifting her own expectations about what meant success or failure in her placement. Her fear of being an ineffective volunteer led her to question not only the benefit that she could bring to her own volunteer position, but the potential benefits of all short-term volunteer programs. Interestingly, this paralysis was reflected in her pre- and post-IDI scores: while her Development Orientation was identified as Minimisation in both surveys, her post-IDI score dropped by nearly 11 points, which is a statistically significant change. Thus, an important lesson from Morgan's case is that it is possible for students to become less interculturally competent as a result of their participation in study abroad programs. Moving backwards on the IDI continuum is common amongst study abroad students who experience high levels of stress or traumatic situations. In Morgan's case, the paralysis she experienced with regard to the value of her contributions to her volunteer organisation, in combination with specific factors related to her placement, may have contributed to the decline in her IDI score.
Vol. 1, No. 1 (2017) 57
Paras & Mitchell
Nevertheless, even in Morgan's case, her regression on the IDI continuum served as a valuable opportunity to learn about her own ability to adapt to an intercultural context, as was evident in her final course reflection submitted a month after her return to Canada. It is worth quoting her reflection at length to see how the decline in her IDI score ultimately helped to improve Morgan's understanding of her own intercultural skills and development,
Ultimately, however, despite lower scores on the IDI survey after the second test,
this was an important learning experience____     Looking back at my first in-class
reflection on what it meant to be interculturally competent I cannot believe how much my attitudes have changed. I even find my perspective towards what I thought it meant to be interculturally competent in my final portfolio to be naive. In both my previous reflections I was viewing intercultural competence as understanding superficial aspects of a culture, and by having cross-cultural interactions, but not necessarily reflecting on the significance of these interactions [sic].
Again, we see that even failures can be mobilised into teaching tools if students are offered appropriate support and opportunities for regular debriefing and reflection. Morgan's example also provides a caution against interpreting IDI results—or any other similar intercultural assessment tool— without digging more deeply into the thought processes behind the numbers. Analysis of the reflections revealed that, despite the lower IDI score, Morgan was ultimately able to make sense of her experience and learn from it. If this analysis had only looked at the IDI results, without considering the content of the reflections, only half of the story about her learning in the field school would have been told. Likewise, if Morgan hadn't been offered opportunities to debrief and reflect on her experiences, her assessment of the experience— and her learning—may have been less positive.
Lessons for Educators
What do students learn when we send them abroad to study, and how can we know that they are learning the things that we intend them to learn? From the outset, the field school's design was underlined by the assumption that intercultural competence is a skill that students must intentionally learn, rather than an inevitable outcome of studying abroad. A unique feature of
58
ELTHE: A Journal for Engaged Educators
Assessing Intercultural Competence
the field school, therefore, was that intercultural competence was a focus of class discussions and readings and was explicitly identified as one of the course's learning objectives. Following from this pedagogical design, one of the main goals of this research was to assess how the field school contributed to students' learning about intercultural competence and to identify lessons for educators who wish to support the intercultural development of their students.
Educators should not assume that intercultural competence will improve as a result of student participation in study abroad programs. As Paige and Vande Berg (2012) have demonstrated, educators should not assume that students who participate in study abroad programs will return with higher levels of intercultural competence, especially if appropriate interventions and supports are not provided to students before, during, and after their programs. In the case of the India field school, approximately thirty hours of pre-departure intercultural training was provided to students, and a full-time faculty member remained on site throughout the field school to provide intervention and support. Even then, only seven students improved their intercultural competence according to their pre-and post-IDI survey results, while four students had no statistically significant change, and two students moved backwards on the IDI continuum. Despite substantial focus in the course on understanding intercultural competence, and significant effort on the part of students to improve their intercultural competence, improvements were not uniform across the class. Educators, therefore, should not assume in any study abroad program—whether there is substantial intercultural training or not—that students' intercultural competence will improve as a result of their experiences abroad.
A second lesson follows from the above, which is that educators and students should be prepared for the possibility of moving backwards in intercultural competence. For many students who participate in study abroad programs, this is the first time that they have had any significant independent experience in a completely new cultural environment. Along with the novelty and excitement of these new experiences often comes a plethora of disorienting emotions, including anxiety, anger, discomfort, guilt, fear, frustration, and exhaustion. As identified in the discussion above, these can all be compounded by situations where students' expectations are unmet, or where communication is a challenge. In programs that also contain a volunteer or community service component, students not only
Vol. 1, No. 1 (2017) 59
Paras & Mitchell
have to manage themselves, they also have to consider their role and impact in an unfamiliar workplace. It should be no surprise, therefore, if all these compounding factors lead to an outcome where students have difficulty in knowing how to appropriately respond and adapt to the situation, much less improve their intercultural competence. Like Morgan above, they may cope with the challenges by searching out commonalities rather than engaging with cultural differences, or by reacting defensively or judgementally to cultural differences without adequately understanding how their own cultural values or assumptions figure into the situation. Educators and students alike should be prepared for the possibility that failures can occur, despite good intentions to the contrary. Course instructors and administrators should have appropriate support structures in place to mitigate the effects of failure, while students should be prepared for scenarios they might encounter that could spark backward movement in their intercultural competence. This is not to set students up for failure but rather to create an environment in which learning is still possible in the event of failure.
Course instructors and administrators should be equipped with appropriate tools and supports to maximize the opportunity for learning no matter where students are beginning along the IDI continuum or how successful they are at moving forward along the continuum. In other words, intercultural learning happens at various starting points, and educators must be prepared to tailor their teaching according to student learning needs. According to the IDI model, intercultural development takes place along the continuum, and individuals must move through all the stages along the spectrum—that is, skipping a stage is not possible. For instance, Tyler began the course with a Polarisation mindset, which tends to see cultural difference in terms of an "us" and "them" mentality. As she identified in one of her pre-departure reflections, her strategy for improving her intercultural competence was to "draw more similarities between cultures rather than differences." By the end of the course, her post-IDI placed her orientation at the subsequent stage of Minimisation, which indicates that she successfully met her learning objective of improving her intercultural competence. It would not have been realistic or appropriate to expect her to skip ahead to an Acceptance Orientation, nor would it have been effective to design pedagogical interventions intended to promote Acceptance. Similarly, students who are already working within an Acceptance Orientation would require different learning strategies than those that are one stage behind in Minimisation. Rory, for instance, already had an
60
ELTHE: A Journal for Engaged Educators
Assessing Intercultural Competence
appreciation for the cultural differences and similarities she encountered, and she was struggling to figure out how to shift mindsets without compromising her own values. In her case, the pedagogical strategy was to find ways for her to move towards Adaptation rather than simply help her identify patterns of difference in cultural values.
It is also important to note here that one's IDI development orientation is not necessarily a predictor of one's ability to engage in intercultural learning. By virtue of the fact that the IDI survey measures intercultural competence along a development continuum, it is assumed that an individual is capable of intercultural learning no matter her orientation along the continuum. Of the students mentioned above, Rory was at Acceptance pre- and post-program, Casey was at Minimisation pre- and post-program, while Morgan moved backwards along the continuum. Nevertheless, all of these students demonstrated deep insights into how the challenges they faced during the field school contributed to their learning processes. Therefore, educators can maximize their students' learning abroad by tailoring interventions that support students' individual intercultural learning needs.
Our research proposes that using IDI survey results in combination with written reflections provides an effective way of assessing intercultural learning. In the case of the India Field School, using written reflections alongside the IDI survey results had two essential functions. First, from a pedagogical perspective, it made it possible for the course instructor to make timely interventions that would assist with student development. For example, one of Jaime's reflections unconsciously judged workplace culture in an Indian and Tibetan NGO based on Western cultural norms. In the course instructor's written feedback to the reflection, she was able to point out this "blind spot," which in turn provided Jaime with an opportunity to identify and reflect on her own unconscious cultural assumptions. In other words, written reflections provide a medium for students to make meaning of their own experiences and learning, as well as a means by which educators can both assess student learning and intervene appropriately.
Second, from a research perspective, collecting written reflections in combination with IDI survey results made it possible to collect more detailed data about the process of learning that takes place abroad. Through written reflections, educators and researchers can learn more about how and why students' intercultural competence increases or decreases during study abroad programs and can even link student learning to specific events or challenges.
Vol. 1, No. 1 (2017) 61
Paras & Mitchell
Our research does not propose a major departure from those programs or studies that already use the IDI as a teaching or research tool, since the IDI collects contexting statements each time the survey is administered. However, the contexting statements only capture what students recall about a past event at the time of taking the survey, whereas regular written reflections capture students' thought processes while they are still in the midst of a study abroad experience. Written reflections are able to capture more accurately how students are responding to specific intercultural challenges or their fresh perceptions of a new situation. In short, our research suggests that using written reflections to capture intercultural learning has both pedagogical and research benefits.
Finally, if educational institutions are serious about helping students improve their intercultural competence during study abroad programs, it is imperative for there to be coordination and cooperation amongst course instructors, study abroad administrators, and ideally, senior levels of university administration. The teaching methodology described above was effective, but it also required significant commitment from multiple levels of the university. Reading and responding to multiple student reflections required time and resources from the course instructor. Launching a one-semester-long pre-departure seminar required cooperation at the department and college level, as well as coordination with the Centre for International Programs.5 In addition to administrative coordination, one factor that was crucial for the success of the program was pedagogical alignment among all the contributors to the program. In the case of the India Field School, the course instructor, the Department of Political Science,6 and the Centre for International Programs were all equally committed to a rigorous method of promoting students' intercultural competence through their participation in study abroad. This cooperation is signified by a unique feature of this research, namely that one of the co-investigators in this study is a faculty member and one is a study abroad administrator.
At a time when educational institutions are already scrambling for limited resources, our prescriptions may appear daunting. Indeed, study abroad faculty and administrators at different institutions may have varying capacities to modify the existing structures of their programs in order to rigorously promote intercultural learning among their study abroad students. Launching a one-semester preparation course for study abroad students, such as the one offered prior to the India Field School, may seem like a luxury
62
ELTHE: A Journal for Engaged Educators
Assessing Intercultural Competence
that educational institutions cannot afford. Being able to facilitate in-depth intercultural learning may also present challenges for instructors who may need to undergo intercultural training of their own before they are equipped to teach these skills to their students.
However, if educational institutions are truly committed to producing global citizens, and if intercultural skills really are essential in an increasingly globalized world, we cannot afford to simply hope that students will learn intercultural competence through osmosis. Especially considering that study abroad programs require significant financial investment from students and institutions, it seems foolhardy to leave the development of a major learning objective to chance. Rather, investing adequate resources to the promotion of improved intercultural competence in study abroad students is not a luxury, but a necessity.
Notes
1. Due to illness or other issues, some students did not complete all seventeen reflections.
2. However, as the course instructor was also one of the co-investigators, it was possible in some cases for her to identify the author(s) of the written reflections during coding, despite the pseudonyms.
3. A change of seven or more points is considered to be statistically significant.
4. Statistically significant changes have been highlighted in bold in Figure 4.
5. The Centre for International Programs at the University of Guelph is responsible for administering all of the university's study abroad programs.
6. The Department of Political Science is the course instructor's home department.
References
Bateman, Blair E. 2002. "Promoting Openness toward Culture Learning: Ethnographic Interviews for Students of Spanish." The Modern Language Journal 86, no. 3: 318-31.
Blenkinsop, Lisa, and Lynne A. Mitchell. 2011. "The Study Abroad Experience Impact on Intercultural Development." Unpublished internal study, Centre for International Programs, University of Guelph.
Gambino, Giacomo, and S. Mohsin Hashim. 2016. "In Their Own Words: Assessing
Vol. 1, No. 1 (2017) 63
Paras & Mitchell
Global Citizenship in a Short-term Study-Abroad Program in Bangladesh."
Journal of Political Science Education 12, no. 1: 15-29. Hammer, Mitchell R. 2012. "The Intercultural Development Inventory: A New
Frontier in Assessment and Development of Intercultural Competence." In
Student Learning Abroad: What Our Students Are Learning, What They're Not, and
What We Can Do About Lt, edited by Michael Vande Berg, R. Michael Paige, and
Kris Hemming Lou, 115-36. Sterling, VA: Stylus. Jackson, Jane. 2008. "Globalization, Internationalization, and Short-Term Stays
Abroad." International Journal of Intercultural Relations 32, no. 4: 349-58. Mezirow, Jack. 1997. "Transformative Learning: Theory to Practice." New Directions
for Adult and Continuing Education 74: 5-12. -. 1991. Transformative Dimensions of Adult Learning. San Francisco, CA:
Jossey-Bass.
Mitchell, Lynne. 2013. "The Qualities and Quantities of Study Abroad: Making It Count." In World of Learning: Canada's Performance and Potential in International Education, 41-45. Ottawa, ON: Canadian Bureau for International Education.
Paige, R. Michael, and Michael Vande Berg. 2012. "Why Students Are and Are Not Learning Abroad: A Review of Recent Research." In Student Learning Abroad: What Our Students Are Learning, What They're Not, and What We Can Do About Lt, edited by Michael Vande Berg, R. Michael Paige, and Kris Hemming Lou, 29-60. Sterling, VA: Stylus.
Patterson, Paula K. 2008. "Effect of Study Abroad on Intercultural Sensitivity." Ph.D. dissertation, University of Missouri-Columbia.
Perry, Lane, Lee Stoner, and Michael Tarrant. 2012. "More Than a Vacation: Short-Term Study Abroad as a Critically Reflective, Transformative Learning Experience." Creative Education 3, no. 5: 679-83.
Vande Berg, Michael, R. Michael Paige, and Kris Hemming Lou. 2012. "Student Learning Abroad: Paradigms and Assumptions." In Student Learning Abroad: What Our Students Are Learning, What They're Not, and What We Can Do About Lt, edited by Michael Vande Berg, R. Michael Paige, and Kris Hemming Lou, 3-28. Sterling, VA: Stylus.
64
ELTHE: A Journal for Engaged Educators
Copyright of Experiential Learning & Teaching in Higher Education (ELTHE): A Journal for Engaged Educators is the property of Southern Utah University Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.