Defining the Question Protocol 55
In my case, thanks to funding and other resource constraints, three years passed between two rounds of fieldwork for a single project (Cyr 2017b). If I had not carefully made note of my instruction protocol and moderator choices, I would have had reason to be concerned about the comparability across each set of focus groups.
Defining the Question Protocol (Question 12)
Developing a set of questions, sometimes called a question protocol or a questioning route, may seem easier than it actually is. In fact, a lot of careful consideration is required. For one, a researcher must strategize over the best use of the focus group's fairly limited length. In one and a half to two hours -the standard time of any single focus group - you will be hard-pressed to ask more than eight to twelve questions (Krueger 1998a). Ask more than that, and you will sacrifice the substantive depth that makes focus groups so unique as a method.
Questions, then, should be carefully thought out, with an emphasis on making the most of a limited time frame. They must also be pointed, that is, directed toward the research at hand. Finally, they should be engaging for the participants and even fun. In all, writing a question protocol is no easy task! Therefore, this section is oriented toward helping you create a useful set of focus group questions.
Above all else, when developing your focus group questions, you must keep your research goals in mind. What are you hoping to learn from the participants? What topics do you want them to address? The questions should evoke conversation, but they should also be one-dimensional. That is, each question should address one issue or topic. It is better to ask two questions, addressing two different dimensions of an issue (How was the program useful to you? How was it practical?), rather than fold both into one (How was the program useful and practical?). Overall, the questions should be concise and easy to understand. Ideally, they will adopt the language and idioms of the participants themselves (Krueger and Casey 2015, Chapter 3).
When developing a question protocol, most obviously, the questions should be open-ended in nature. If questions can be answered with a simple, "yes" or "no," then they are unlikely to generate much conversation. In Gamson's study, for example, the moderator asked participants to think about issues that were prominent in the news cycle at that time. She asked, "When you think about this issue of..., what comes to mind?" (Gamson 1992, 194-195). Phrased as such, citizens are invited to give their reaction, whatever it might be, to the topic.
Downloaded from https://www.cambridge.org/core. Pontificia Universidad Catolica de Chile, on 03 Aug 2020 at 23:12:53, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/9781316987124.003
56
Preparing for the Focus Groups
Keep in mind that it may even be advisable to avoid "why" questions, as they can inadvertently put individuals on the defensive. Rather than ask, for example, why your participants participated in a particular program, you might consider rephrasing it in the following way: "What prompted you to participate in ...?" (Krueger et al. 2001, 9). In the case of Gamson's project, a follow-up prompt to the initial issue-based question mentioned above might not be "Why did you have that reaction?" Instead, the moderator should be prompted to react, where appropriate, "That is interesting. What do you think causes you to have that reaction?"
Additionally, the kinds of questions you ask can (and should!) vary. For example, it may be appropriate to ask questions that compel participants to recall an experience or event - presumably, one that is related to the research question at hand. You might also use visual aids, stories, song lyrics, or advertisements to contextualize a particular question. Returning to Gamson (1992), the participants were presented with a political cartoon that tackled the same issue they had previously discussed. Here, participants were asked to comment on the cartoon specifically. These questions break up the monotony and engage participants in a different way (Krueger 1998a). Along these lines, participants may be asked listing, rating, or sorting questions, where they are asked to work together to categorize or classify a set of topics related to the research (Colucci 2007).
Note that the inclusion of a non-standard or traditional question can be a useful tactic for asking the same question in a different way. This strategy -asking a similar question in multiple formats - allows a researcher to check for the reliability of participant responses on certain, key questions. In most focus group research designs, the researcher will be particularly interested in assessing participant reactions to one or two driving concerns. Often these concerns are the justification for the focus groups themselves.
For example, I have used focus groups to understand why certain Peruvians identify strongly with or against a particularly controversial former leader, Alberto Fujimori. At the time of the research, not much was known about these individuals, who often identify as fujimoristas or anti-fujimoristas. I organized focus groups of either fujimoristas or anti-fujimoristas to understand how they thought about each group and why they identified as belonging to one or the other. I asked several questions to tap into the same notion: How individuals understand the meaning of "(anti-)fujimorista." One question included a group exercise, which required participants to work together and list the traits of an ideal politician. The purpose of this question allowed me to assess, first, whether that politician reflected the qualities that they associated with (anti-)fujimorismo, and, second, the extent to which there was agreement about what (anti-)fujimorismo was.
Downloaded from https://www.cambridge.org/core. Pontificia Universidad Catolica de Chile, on 03 Aug 2020 at 23:12:53, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/9781316987124.003
Defining the Question Protocol 57
Finally, it may be appropriate periodically to ask participants to write down their response on a particular question before voicing it to the rest of the group. There are two benefits to this. Most importantly, it can allow you, as the researcher, to check for the presence of group think (see Chapter 4) or of an overly influential participant. Do participants' vocal responses cohere with what they wrote? If so, then you can feel more confident that their spoken answers were not overly swayed by one particularly predominant idea or person. Additionally, a written record of certain responses may be useful when it comes to analyzing your data. When, for example, measuring how participants understand their (mti-)fujimorista identity, it was useful for me to ask each participant to assign three characteristics to that identity. The primary goal was to spark a group conversation about the characteristics that were listed. As a secondary objective, however, I sought to glean an actual count of the different descriptors used by participants and the extent to which they assigned the same descriptor to the identity. The written responses were useful in crafting a database on this point. They also allowed me to analyze data created at the individual level of analysis (see, e.g., Chapter 5).
Keep in mind that your research goals will also shape the question protocol. What do you wish to learn from the focus groups? Is the project largely exploratory in nature? Are you hoping to test a hypothesis? The goals of the research will help determine the kinds of questions you ask. This, of course, seems obvious. In a focus group about drug addiction, it will not make sense to pose questions about political preferences. Less obvious, however, is that your research goals can even influence the number of questions you ask. For instance, when your goal is largely exploratory, it may make sense to ask only one or two very open questions. When, however, your goal is to test a theory or hypothesis, it makes more sense to develop a full question guide with fairly structured questions (Morrison-Beedy et al. 2001, 48).
Finally, as with most things methodological, there are tradeoffs involved with crafting a usable question protocol. For one, questions should be open enough to generate discussion. Yet, there is also some utility to retaining and, where necessary, imposing the order and nature of the question protocol. This is especially true where you expect to compare responses across focus groups. Greater focus group standardization will make comparability feasible. At the same time, too much standardization can expose focus groups to the "fallacy of adhering to fixed questions" (Morgan 1996, 142). After all, participants may wish to take the conversation in a different direction than the questions allow. One possible solution to this tradeoff is to pre-test and/or run a pilot of the questions with one or two focus groups to see if the general tenor of the
Downloaded from https://www.cambridge.org/core. Pontificia Universidad Catolica de Chile, on 03 Aug 2020 at 23:12:53, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/9781316987124.003
58
Preparing for the Focus Groups
conversation veers too much from the protocol itself. This requires time and resources, however, which the researcher may not have. An alternative solution would be to establish a small set of core questions that must be asked and allow for greater flexibility beyond that (Morgan 1996).
The researcher may also wish to strategize about the best approach to question order. Sometimes it may make sense to begin with a general question and make each additional question increasingly pointed, such that the final questions address the researcher's most important concerns. This sequencing can allow participants to flex their "focus group muscles," if you will, before getting to the really crucial questions. It may also, however, generate participant fatigue and inadvertently constrain the quality of the discussion of those important questions. On this point, question variety can mitigate fatigue. Additionally, where the researcher hopes to ask the same (ostensibly important) question twice, she may place one version closer to the beginning of the focus group and the other nearer to the end.
Indeed, as a final note of caution, the researcher must keep the overall flow of the conversation in mind when devising a set of questions. Is the question order relatively seamless, such that one question is followed by a different but not unrelated question? If not, then the researcher may wish to allow for some sort of transition text, whereby the moderator explains where the group has been, figuratively speaking, and where they are going. This way, the turn from one topic to another is less disjointed.
Certainly, it makes sense to begin the focus group with an opening question. This question is easy and quick to answer. It is designed to get everyone talking from the get-go. Next, you may want to ask an introductory question - one related to the primary topic of interest but still pretty general in nature. This question serves to break the ice, introducing the participants to the topic without requiring them to engage too directly or specifically with the primary research concerns. From here, you may wish to include a transition question -one that brings the participants to the "heart" of the discussion. This would be followed by the key questions of interest, the figurative "heart" of your investigative inquiry. Finally, time permitting, you may wish to close the focus group with a reflective question, one that allows the participants to summarize their contribution and perhaps speculate about some future aspect of the topic at hand (Krueger and Casey 2015, Chapter 3). Box 3.3 provides a sample question protocol from a project on Venezuelan political parties that follows this format.
Question choices clearly matter when it comes to preparing your focus group design. Knowing what you want to ask how and when you wish to ask it, and where there might be wiggle room along the way will greatly facilitate a successful focus group experience. To be sure, understanding
Downloaded from https://www.cambridge.org/core. Pontificia Universidad Catolica de Chile, on 03 Aug 2020 at 23:12:53, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/9781316987124.003
Defining the Question Protocol 59
Box 3.3 Sample Question Protocol (see also Appendix 1)
1. (Opening question) As I mentioned earlier, we hope to learn a bit about your impressions of different political parties in Venezuela. Before we jump into the questions, however, I'd like each of you to tell us a bit about yourself. Please tell us your first name. Then, give us a sense of how interested you are in politics. Would you say that you spend a lot of time (maybe even daily!), some time, or not very much time thinking about politics?
2. (Introductory question) Now that we know each other, I want to start off with a bit of a memory exercise. Specifically, I'd like to ask you all to think back to the late 1990s. As you probably recall, in 1998, Hugo Chavez was elected to office in a fairly momentous election. At that time, two parties (AD and COPEI) were elected out of office. I would like each of you to think back to that period and tell us a story about AD that you remember. It can be an anecdote about a particular party member, or a general impression that you recall having about the party at the time. You can tell me anything from that period that you can recall about AD. I'll give you a minute or two to think about it, and then we can go around the table, and I'll ask each of you to tell your story.
3. Now that we have recalled a couple of stories from that time, I'd like to know what you remember thinking about AD, that is, what your opinion of the party was. In other words, at that time in the late 1990s, how would you described AD? Please use the paper and pen in front of you to write down three words or phrases that would have come to mind... three words or phrases that best describe what the party meant to you at that time. These can be anything. Once you've written them down, we'll go around the table and everyone can tell us what they wrote.
4. What about COPEI? I'm curious to know how you all would have described the party at that time. What words or phrases would you use to describe that party? As with AD, please write down up to three words or phrases that you might have used about the party in the late 1990s.
5. I'm now going to show you a series of images. I'll ask you to identify the images, if you can, and to share your reactions to these images. What does the image mean to you? What do you think it means for Venezuela? Remember that we don't have to agree about these images and what they mean to us. [In each case, the moderator will first show a picture of the political party's logo, a picture of the political party's founder, and a picture of the most recent president from the party.]
6. (Ranking exercise) As you all know, politics has changed since Hugo Chavez was elected. For one thing, Venezuela has a new constitution. Also, new political parties are competing, and new politicians are being elected into office. Let's talk about these changes. What has changed positively since Chavez was elected in 1998? To answer this question, I'd like us to come up with a list of the three most positive changes that the country has undergone.
Downloaded from https://www.cambridge.org/core. Pontificia Universidad Catolica de Chile, on 03 Aug 2020 at 23:12:53, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/9781316987124.003
60
Preparing for the Focus Groups
Box 3.3 (Cont.)
7. What about negative changes? Can you identify three or more negative changes that the country's political system has undergone?
8. Now I'd like us all to think about AD today. I'd like you to think about what you know or have seen about AD more recently, let's say over the last five years. What do you think about AD today? Has your attitude toward the party changed?
9. What about COPEI? What do you think about COPEI today? Has your attitude toward the party changed?
10. I have one last question for you all. As you know, neither AD nor COPEI has been very successful at fielding presidential candidates over the last ten years. Why do you think this is? Why do you think that Venezuelans stopped voting for the parties? In your opinion, what would each party have to do differently so that people would begin voting for it again? Let's begin first with AD and then consider COPEI.
11. (Final, reflective question) Is there anything else you'd like to mention about either AD or COPEI? Something you'd like to add that hasn't already been mentioned?
what counts as a "good" question grows with one's own experience. In the meantime, you should pre-test your questions with a pilot focus group, whenever possible. You can also share your protocol with colleagues or, even better, friends or contacts living at the research site, to ensure comprehension. And as with all good research, it will be useful to stand on the shoulders of those who have undertaken focus groups previously. Seek out and carefully study the question protocols of those studies you most admire. Ask their authors about any tradeoffs that were undertaken, particularly good choices that they made, or any regrets they might harbor about their question protocol.
Finally, keep in mind that the focus groups themselves will be extremely informative about the quality of the question protocol. With a set of four or five focus groups, you may decide, after carrying out two of them, that a certain question does not work or is not informative. In my case, I quickly discovered that presenting Peruvians with images of former political leaders associated wiihfujimorismo was not particularly illuminating. All participants across the first two focus groups immediately recognized all of them. I promptly decided to leave out that question in order to give more time and space for the questions that were eliciting useful and varied feedback. Certainly, this advice will be less useful when the research design includes only one or two focus groups. Under this circumstance, the objective of the focus group will likely be
Downloaded from https://www.cambridge.org/core. Pontificia Universidad Catolica de Chile, on 03 Aug 2020 at 23:12:53, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/9781316987124.003
Additional Considerations 61
exploratory in nature. Where this is the case, it makes less sense to ask a set of structured questions, and concerns about the quality of the questions should be mitigated.
Additional Considerations Regarding Non-Native Settings, Sensitive Subjects, and Power Differentials between the Researcher and Focus Group Participants
The information covered in this chapter so far is broadly applicable to the wide diversity of settings where focus groups can take place and to the multiple topics that focus groups may cover. In some circumstances, however, extra care may be required as a part of focus group preparation. The researcher may need additional information to create a data collection setting that is as "natural" as possible for the focus group participants. This is the case, for example, when undertaking focus groups in non-native, or international, settings. It is also true when the subject matter of the research is sensitive in nature. This final section considers the extra work involved in preparing for focus groups in non-native settings and when addressing sensitive subjects. It ends with a brief statement on the power asymmetry that operates between the researcher and focus group participants in all focus group settings.
Focus Groups in International Settings
The primary challenge that researchers will confront when carrying out research in international or cross-cultural settings is that they are outsiders -foreign scholars who, explicitly or not, are relying on the knowledge of natives to enrich their own research. There are at least two consequences to this research asymmetry. First, researchers must undertake additional work to ensure that their focus group preparations are informed by the cultural and linguistic practices of the setting in question. I address these concerns in this section and the next section. Second, the power differential that implicitly exists between the researcher and her research subjects is augmented by cultural differences inherent to undertaking focus groups in a non-native setting. I address this very real and important concern in a separate section.
When carrying out focus groups in a non-native setting, the researcher will often be "unfamiliar with the nuances of language, cultural milieu, and traditions of the study participants" (Hennink 2010, 208). Given this, researchers will likely need to rely on local citizens for help with language considerations, recruitment strategies, and focus group logistics. Recall that the focus group should replicate, to the extent possible, a natural (i.e., non-contrived) setting in
Downloaded from https://www.cambridge.org/core. Pontificia Universidad Catolica de Chile, on 03 Aug 2020 at 23:12:53, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/9781316987124.003