MEJUOOS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS Trotter, M. (1992)- life wiling: Exploring the practice of autoethnograpby in anthropology. Unpublished master's diesis, University of Illinois, Urbana-Champaign, True confessions; The age of the literary memoir [Special issue]. (I 996, May 12). New York Times Magazine. Turner, V, & Bruner, E. (Eds.). (1986). The anthropology of experience. Urbana: University of Illinois Press. Tyler, S. (1986). Post-modern ethnography: From document of the occult to occult document. Inj. Clifford and O.E. Marcus (Eitingculture;Tbe poetics and politics of ethnography (pp. 122-140). Berkeley; University of California Press. Van Maanen, J. {1988). Tales of the field: &n writing ethnography. Chicago: University of Chicago Press. Van Maanen, J. (1995). An end to innocence; The ethnography of ethnography. In J. Van Maanen (Ed.), Representation in ethnography (pp. 1-35). Thousand Oaks, CA: Sage. Van Maanen, M. (1990). Researching lived experience: Human science for an action sensitive pedagogy. Albany: State University of New York Press. Wittgenstein, L- (19.53). Philosophical investigations (G. Anscombe, Trans.)H>Jcw York: Macmillan. Zola, I. K. (1982). Missing pieces; A chronicle of living with a disability. Philadelphia: Temple University Press. Zussman, R. (1996). Autobiographical occasions. Contemporary Sociology, 2S, 143-148. Mi 258 '>*- 7 Data Management and Analysis Methods Gery W. Ryon and H. Russell Bernard ♦ Texts Arc Us This chapter is about methods for managing and analyzing qualitative data. By qualitative data we mean text: newspapers, movies, sitcoms, e-mail traffic, folktales, life histories. We also mean narratives—narratives about getting divorced, about being sick, about surviving hand-to-hand combat, about sel ling sex, about trying to quit smoking. In fact, mosrofthearchaeo-logically recoverable information about human thought and human behavior is text, the "good stuff" of social science. Scholars in content analysis began using computers in the 1950s to do statistical analysis of texts (Pool, 1959), but recent advances in technology are changing the economics of the social sciences. Optical scanning today makes light work of convening written texrs to machine-readable form. Within a few years, voice-recognition software will make light work of transcribing open-ended interviews. These technologies are blind to epistemological differences. Interprerivists and posiuvists alike are using these technologies for the analysis of texts, and will do so more and more. Like Tcsch (1990), we distinguish between the linguistic tradition, which treats text as an object of analysis itself, and the sociological tradition, which treats text as a window into human experience (see Figure 7.1). The linguistic tradition includes narrative analysis, conversation (or 259 Dolo Management and Analysis Methods discourse) analysis, performance analysis, and formal linguistic analysis. Methods for analyses in this tradition are covered elsewhere in this Handbook. Wc focus here on methods used in the sociological tradition, which we take to include work across the social sciences. There are two kinds oť written texts in the sociological tradition: (a) words or phrases generated by techniques for systematic elicitation and (b) free-flowing texts, such as narratives, discourse, and responses to open-ended interview questions. In the next section, we describe some methods for collecting and analyzing words or phrases. Techniques for data collection include free lists, pile sorts, frame clicitations, and triad tests. Techniques for the analysis of these kinds of data include componcntial analysis, taxonomies, and mental maps. We then turn to the analysis of free-flowing texts. We look first at methods that use raw text as their input—methods such as key-words-in-context, word counts, semantic network analysis, and cognitive maps. We then describe methods that require the reduction of text to codes. These include grounded theory, schema analysis, classical content analysis, content dictionaries, analytic induction, and ethnographic decision models. Each or these methods of analysis has advantages and disadvantages. Some are appropriate for exploring data, others for making comparisons, and others for building and testing models. Nothing does it all. ♦ Collecting and Analyzing Words or Phrases Techniques for Systematic Elicitation Researchers use techniques for systematic elicitation to identify lists of items that belong in a cultural domain and to assess the relationships among these items (for detailed reviews of these methods, see Bernard, 1994: Borgatti, 1998; Weiler, 1993; Weller &C Romncy, 1988). Cultural domains comprise lists of words in a language that somehow "belong together." Some domains (such as animals, illnesses, things to eat) are very large and inclusive, whereas others (animals you can keep at home, illnesses that children get, brands of beer) are relatively small. Some lists (such ^a the list of terms for members ma family or the names of all the Major League Baseball teams) arc agreed on by all native speakers of a language; others (such as the list of carpenters' tools) represent highly specialized knowledge, and still others (like the list of great left-handed baseball 261 METHODS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS ——i------------------------------------------------------------------------------------------------------------------------------------------- •no-picchcrs of the 20th century) arc matters of heated debate. Below we review some of the most common systematic elicitation techniques and discuss how researchers analyze the data they generate. Free Lists Free lists are particularly useful for identifying the items in a cultural dornen. Toclicit domains, researchers might ask, "What kinds of illnesses do you know?" Some short, open-ended questions on surveys can be considered free lists, as can some responses generated from in-depth ethnographic interviews and focus groups. Investigators interpret the frequency of mention and the order in which items are mentioned in the lists as indicators of items' salience (for measures of salience, see Robbins & Nolan, 1997; Smith, 1993; Smith Ôc Borgatti, 1998). The co-occurrence of items across lists and the proximity with which items appear in lists may be used as measures of similarity among items (Borgatti, 1998; Henley, 1969; fora clear example, see Fleisher & Harrington, 1998). Paired Comparisons, Pih Sorts, Triod Tests Researchers use paired comparisons, pile sorts, and triads rests to explore the relationships among items. Here are two questions we might ask someone in a paired comparison test about a list of fruits: (a) "On a scale of 1 to 5, how similar are lemons and watermelons with regard to sweetness?" (b) "Which is sweeter, watermelons or lemons?" The first question produces a set of fruit-by-fruit matrices, one for each respondent, the entries of which are scale values on the similarity of sweetness among all pairs of fruits. The second question produces, for each respondent, a perfect rank ordering of the set of fruirs. In a pile sort, the researcher asks each respondent to sort a set of cards or objects into piles. Item similarity is the number of times each pair of items is placed in the same pile (for examples, see Boster, 1994; Roos, 1998). Iryi triad test, the researcher presents sets of three items and asks each respondent either to "choose the two most similar items" or to "pick the item that is the most different." The similarity among pairs of items is the number of times people choose to keep pairs of items together (for some good examples, see Albert, 1991; Harman, 1998). 262 Data Monagemenl and Analysis Methods Frome Substitution In the frame substitution task (D'Andrade, 1995; D'Andrade, Quinn, Nerlove, Si Romney, 1972; Frake, 1964; Metzger 6c Williams, 1966), the researcher asks the respondent to link each item in a list of items with a list of attributes. D'Andrade et al. (1972) gave people a list of 30 illness terms and asked them to fill in the blanks in frames such as "You can catch_____ from other people," "You can have_____and never know it," and "Most people- get_____at one rime or other" {p. 12; for other examples of frame substitution, see Furhec &c Benfer, 198.3; Young, 1978). Techniques for Analyzing Data About Cultural Domains Researchers use these kinds of data to build several kinds of models about how people think. Componential analysis produces formal models of the elements in a cultural domain, and taxonomies display hierarchical associations among the elements in a domain. Mental maps are best for displaying fuzzy constructs and dimensions. We treat these in turn. Qomponentiaf Analysis As we have outlined elsewhere, componential analysis (or feature analysis) is a formal, qualitative technique for studying the content of meaning (Bernard, 1994; Bernard & Ryan, 1998). Developed by linguists to identify the features and rules that distinguish one sound from another (Jakobson & Halle, 1956), the technique was elaborated by anthropologists in the 1950s and 1960s (Conklio, 1955; D'Andrade, 1995; Frake, 1962; Goodenough, 1956; Rushforth, 1982; Wallace, 1962). (Fora particularly good description of how to apply che method, seeSpradley, 1979, pp. 173-184.) Componenrial analysis is based on the principle of distinctive features. Any two items (sounds, kinship terms, names of plants, names of animals, and so on) can be distinguished by some minimal set {In) of binary features—that is, features that cither occur or do not occur. It takes two features to distinguish four items {21 » 4, in other words), three features to distinguish eight items (21 « 8), and so on. The trick is to identify the sniiillest set of features that best describes the domain of interest. Table 7.1 shows that just three features arc needed to describe kinds of horses. 263 METHOOS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS MtO' TABLE 7.1 A Component ial Analysis of Six Kinds or Horses Noma Female Neuter Adult Moro + - + Stollion - + Gelding — + + Foal - -f- - Filly* +■ Colt"____________________=_______________=_______________=________ SOURCE: Adapted hrom D'Andrade(1995). Componcntial analysis produces models based on logical relationships among features. The models do noc account for variations in the meanings of terms across individuals. For example, when we tried to do a com-ponential analysis on the terms for cattle [bull, cow, heifer, calf, steer, and ox), we found that native speakers of English in the United States (even farmers) disagreed about the differences between cow and heifer, and between steer and ox. When the relationships among items are less well defined, taxonomies or menta! models may be useful. Nor is there any intimation chat componcntial analyses reflect how "people really think." Toxonomies Folk taxonomies are meant to capture che hierarchical structure in sets or terms and are commonly displayed as branching tree diagrams. Figure 7.1 presents a taxonomy of our own understanding of qualitative analysis techniques. Figure 7.2 depicts a taxonomy wc have adapted from Pamela Erickson's (1997) study of the perceptions among clinicians and adolescents of methods of contraception. Researchers can elicit folk taxonomies directly by using successive pile sorts (Boster, 1994; Perchonock 6c Werner, 1969). This involves asking people to continually subdivide che piles of a,, free pile sort until each item is in its own individual pile. Taxonomie models can also be created with cluster analysis on the similarity data from paired comparisons, pile sores, and triad tcscs. Hierarchical cluster analysis (Johnson, 1967) builds a taxonomie tree where each item appears in only one group. 26-1 »íl 265 METHODS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS i ______________^^^_____— Interinrormanr variation is common in folk taxonomies. That is, different people may use different words to refer to the same category of things, Some of Erickson's (1997) clinician informants referred to the "highly effective" group of methods as "safe," "more reliable," and "sure bets." Category labels need not be simple words, but may be complex phrases; for example, see the category in Figure 7-2 comprising contraceptive methods in which you "have to pay attention to timing." Sometimes, people have no labels at all for particular categories—at least none that 5iey can dredge up easily—and categories, even when named, may be fuzzy and may overlap with other categories. Overlapping cluster analysis (Hartigan, 1975) identifies groups of items where a single item may appear in multiple groups. M&ntal Maps Mental maps are visual displays of the similarities among items, whether or not those items are organized hierarchically. One popular method for making these maps is by collecting data about the cognitive similarity or dissimilarity among a set of objects and then applying multidimensional scaling, or MDS, to the similarities (Kruskal 6c Wish, 1978). Cognitive maps are meant to be directly analogous to physical maps. Consider a table of distances between all pairs of cities on a map. Objects (cities) that are very dissimilar have high mileage between them and are placed far apart on the map; objects that are less dissimilar have low mileage between them and arc placed closer together. Pile sorts, triad tests, and paired comparison tests are measures of cognitive distance. For example, Ryan (1995) asked 11 literate Kom speakers in Cameroon to perform Successive pile sorts on Kom illness terms. Figure 7.3 presents an MDS plot of the collective mental map of these terms. The five major illness categories, circled, were identified by hierarchical cluster analysis of the same matrix used to produce the MDS plot.1 Data from frame substitution tasks can be displayed with correspondence analysis (Weiler &c Romiicy, 1990).1 Correspondence analysis scales both the rows and the columns into the same space. For example, Kirchler (19V2) analyzed S62 obituaries of managers who had died in 1974, 1980, and 1986. He identified ^ 1 descriptive categories from adjectives used in the obituaries and then used correspondence analysis to display how these categories were associated with men and women managers over time. Figure 7.4 shows that male managers who died in 1974 and 1980 were seen 2Ó6 Data Management and Analysis Methods Figure 7.3. Mental Map of Kam Illness Terms by their surviving friends and family as active, intelligent, outstanding, conscientious, and experienced experts. Although the managers who died in 1986 were still respected, they were more likely to be described as entrepreneurs, opinion leaders, and decision makers. Perceptions of female managers also changed, but they did not become more like their male counterparts. In 1974 and 1980, female managers were remembered for being nice people. They were described as kind, likable, and adorable. By 1986, women were remembered for their courage and commitment. Kirchler interpreted these data to mean that gender stereotypes changed in the early 1980s. 15y 1986, both male and female managers were perceived as working for success, but men impressed their colleagues through their knowledge and expertise, whereas women impressed their colleagues with motivation and engagement. 267 METHODS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS iw 1,15 - •71 - .28 m c -.16 n -.59 2 •1,03 ■IM eomredelŕ« coiucicniious active MALE 'MouBUiHling l-intelliBCTl MALE'« experienced expert «W»« ho nesl lespeuied t r cílil ieiil pioneer und likable FEMALE I iilinabk FEMALE 74 faithful sdmir.'bk entrepreneurial irwtaicaw ipiril MALE '86 opinion leader work oriented uaMUlsh decision maket sociable dring amiable foreign led hnjhlf commiKcd coumgcoui FE M ALK -.70 ~l---- •M V S ,3í .60 ÍM .39 1.65 1 91 .17 Dimension 1 Figure 7.4, Correspondence Analysis of rhe Frequencies of 31 Disruptive Obituary Categories by Gender and Year of Publication SOURCE: Erich Kirchler, "Adorable Woman, Expert Man: Changing Gender Images of Women and Men in Management," European Journal of Social Psychologu, 22 (1992), p. 371. Copyright 1992 by John Wiley & Sons Limited. Reproduced by permission of John Wiley & Sons Limited, ♦ Methods for Analyzing Free-Flowing Text Although taxonomies, MDS maps, and the like are useful for analyzing short phrases or words, most qualitative data come in the form of Iree-fiowing texts. There are two major types of analysis. In one, the text is segmented into its most basic meaningful components: words. In the other, meanings are found in large blocks of text. 26« Dolo AHonogemení and Analysis Methods Analyzing Words Techniques for word analysis include kcy-words-in-coutext, word counts, structural analysis, and cognitive maps. We review each below. Key-Woros-in-Corrtexf Researchers create key-words-in-concext (KWIC) lists by finding all the places in a text where a particular word or phrase appears and printing it out in the context of some number of words {say, 30) before and after it. This produces a concordance. Well-known concordances have been done on sacred texts, such as the Old and New Testaments (Dartori, 1976; Hatch oc Redpath, 1954) and the Koran (Kassts, 1983), and on famous works of literature from Euripides (Allen &c Itálie, 1954) to Homer (Prcndcrgast, 1971), to Beowulf (Bcssingcr, 1969), to Dylan Thomas (Farringdon& Farringdon, 1980). (On the use of concordances in modern literary studies, see Burton, 1981a, 1981b, 1982; McKinnon, 1993.) Word Counts Word counts are useful for discovering patterns of ideas in any body of text, trom field notes to responses to open-ended questions. Students of mass media have used use word counts to trace the ebb and flow of support for political figures over time (Danielson Sc Lasorsa, 1997; Pool, 1952). Differences in the use of words common to the writings of James Madison and Alexander Hamilton led Mosteller and Wallace (1964) to conclude that Madison and not Hamilton had written 12 of the Federalist Papers. (For other examples of authorship studies, see Martindale 6c McKenzie, 1995; Yule 1944/1968.) Word analysis (like constant comparison, meinoing, and other techniques) can help researchers to discover themes in texts. Ryan and Weisner (1996) instructed fathers and mothers of adolescents in Los Angeles: "Describe your children. In your own words, just tell us about ihein." Ryan and Weisner identified all the unique words in the answers they got to that grand-tour question and noted die number oi times each word was used by mothers and by fathers. Mothers, for example, were more likely to use words like friends, creative, time, and honest; fathers were more likely to use words like school, good, lack, student, enjoys, independent, and extremely. This suggests that mothers, on first mention, express concern 269 METHODS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS i — »no-over interpersonal issues, whereas fathers appear to prioritize achievement-oriented and individualisiic issues. This kind of analysis considers neither the contexts in which the words occur nor whether the words are used negatively or positively, hut distillations like these can help researchers to identify important constructs and can provide data for systematic comparisons across groups. Structural Analysis and Semantic Networks f .• Network, or structural, analysis examines the properties rhat emerge from relations among things. As early as 1959, Charles Osgood created word co-occurrence matrices and applied (actor analysis and dimensional plotting to describe the relations among words. Today; semantic network analysis is a growing field (Barneti &c Danowski, 1992; Danowski, 1982, 1993). For example, Nolan and Ryan (1999) asked 59 undergraduates (30 women and 29 men) to describe their "most memorable horror film." The researchers identified the 45 most common adjectives, verbs, and nouns used across the descriptions of the films. They produced a_45(word)-by-59(person) matrix, the cells or which indicated whether each student had used each key word in his or her description. Finally, Nolan and Ryan created a 59(person)-by-59(person) similarity matrix of people based on the co-occurrence of the words in their descriptions. Figure 7.5 shows the MDS of Nolan and Ryan's data. Although there is some overlap, it is pretty clear that the men and women in their study used different sets of words to describe horror films. Men were more likely ro use words such as teenager, disturbing, violence, rural, dark, country, and hillbilly, whereas women were more likely to use words such as boy, little, devil, young, horror, father, and evil. Nolan and Ryan interpreted these results to mean that the men had a fear of rural people and places, whereas the women were more afraid of betrayed intimacy and spiritual possession. (For other examples of the use of word-by-word matrices, see Jang 6c Barnett, 1994; Schnegg EC Bernard, 1996.) This example makes abundantly clear the value of turning qualitative data into quantitative data: Doing so can produce information that engenders deeper interpretations of thv„mcanings in the original corpus of qualitative data. Just as in any mass of numbers, it is hard to see patterns in words unless one first docs some kind of data reduction. More about this below. As in word analysis, one appeal of semantic network analysis is that the data processing is done by computer. The only investigator bias intro- 270 Data Management and Analysis Methods GENDER o*Malc 9 Female Figure 7.5. Multidimensional Scaling of Informants Based on Words Used in Descriptions of Horror Films duced in the process is the decision to include words that occur at least 10 times or 5 times or whatever. (For discussion or computer programs that produce word-by-text and word-by-word co-occurrence matrices, see Borgatti, 1992; Docrfcl & Barnett, 1996.) There is, however, no guarantee that the output of any word co-occurrence matrix will be meaningful, and it is notoriously easy to read patterns (and thus meanings) into any set of items. Cognitive Maps Cognitive map analysis combines theyntuirion of human coders with the quantitative methods of network analysis. Carley's work with this technique is instructive. Carley argues that if cognitive models or schemata exist, they arc expressed in the texts of people's speech and can be represented as networks of concepts (see Carley & Palmquist, 1992, ď ď o" ** «r ša a ď * ° " , ?e* 0 g Q 9 271 METHODS OF COLLECTING ANO ANALYZING EMPIRICAL MATERIALS -1(0' p. 602), an approach also suggested by D'Andrade (1991). To the extent that cognitive models are widely shared, Carlcy asserts, even a very small set of texts will contain the information required for describing the models, especially for narrowly defined arenas of life. In one study, Carley (1993) asked students some questions about the work of scientists. Here are two examples she collected: Student A; 1 found that scientists engage in research in order to make discoveries and generate new ideas. Such research by scientists is hard work and .often involves collaboration with oilier scientists which leads to discoveries which make che scientists famous. Such collaboration may be informal, such as when they share new ideas over lunch, or formal, such as when they arc coauthors of a paper. Student B: Ir was hard work to research famous scientists engaged in collaboration and I made many informal discoveries. My research showed that scientists engaged in collaboration with other scientists are coauthors of at least one paper containing their new ideas. Sonic scientists mukc formal discoveries and have new ideas, tp. 89) Carley compared the students' texts by analyzing 11 concepts: /, scientists, research, hard work, collaboration, discoveries, new ideas, formal, informal, coauthors, paper. She coded the concepts for their strength, sign (positive or negative), and direction (whether one concept is logically prior to others), not just for their existence. She found thac although students used the same concepts in their texts, the concepts clearly had different meanings. To display the differences in understandings, Carley advocates the use of maps that show the relations between and among concepts. Figure 7.6 shows Carley's maps of two of the texts. Carley's approach is promising because it combines the automation of word counts with the sensitivity of human intuition and interpretation. As Carley recognizes, however, a lot depends on who does the coding. Different coders will produce different maps by making different coding choices. In the end, native-language competence is one of the fundamental methodological requirements for analysis {see also Carley, 1997; Carley & Käufer, 1993; Carley & Palmquist, 1992; Palmquist, Carley, ÖC Dale, 1997). Key-words-in-context, word counts, structural analysis, and cognitive maps all reduce text to the fundamental meanings of specific words. These reductions make it easy for researchers to identify general patterns and 272 Dato Management and Analysis Methods hard ww diitiivrrlcs idCOllltS hard work »\ research \ discoveries I \ collaboradon* new ideas formal inform» paper ■ ii.'mii ' paper iiijiiiliin'i Shared Concepts ......................................................... 11 Shared Statement* ...................(I bidirectional - X rclallum) S Shared Concepts Riven Shared Relationships...................... 5 Concept» Student A Only .............................................. * Concept* Student II Ooly .............................................. 0 StatemmcsStudent A Only ............................................ 13 Statements Studcot D Only ............................................ 9 positive relationship i negative relationship Figure 7.6. Coded Maps of Two Students' Texts SOURCE: Kathleen Carley, "Coding Chokes for Textual Analysis: AComparison of Content Analysis .ind Map Analysis," in V. Marsden (Ed.), Sociological Methodology {Oxford: Bl.iekwcll,l993),p. 104. Copyright 1993 by the AmericanSociologic.il Association. Reproduced by permission of the American Sociologičtí Association. make comparisons across texts. With tlie exception of KW1C, however, these techniques remove words from the contexts in which they occur. Subtle nuances are likely to be lost—which brings us to the analysis or whole texts. 273 METHODS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS •IK)' Analyzing Chunks of Text: Coding Coding is the heart and soul or whole-text analysis. Coding forces the researcher to make judgments about the meanings uf contiguous blocks of lext. The fundamental tasks associated with coding are sampling, idem Hying themes, building codebooks, marking texts, constructing models (relationships among codes), and testing these models against empirical data, We outline each task below. We then describe some of the major coding traditions: grounded theory, schema analysis, classic content analysis, content dictionaries, analytic induction, and ethnographic decision 11< < v We want to cmphasi/e that no particular tradition, whether hum.mi-.ii. i »r posirivistic, has .1 monopoly on text analysis i Sampling Investigators inn« first identify a corpus of texts, and then select the units of analysis within the texts. Selection can be either random or purposive, but the choice is not a matter of cleaving to one-epistemological tradition or another. Waitzkin and Britt (1993) did a thoroughgoing inter • prctive analysis of encounters between patients and doctors by selecting 50 texts at random from 336 audiotaped encounters. Trost (1986) used classical content analysis to test how the relationships between teenagers and their families might be affected by five different dichotomous variables. He intentionally selected five cases from each of die 32 pOHl blc combinations of ihe five variables and conducted 32 x S ■ 160 interviews, Samples may also be based on extreme or deviant cases, cases that illustrate maximum variety on variables, cases that are somehow typical of a phenomenon, or cases that confirm or disconfirm a hypothesis. (For reviews of nonrandom sampling strategies, sec Patton, 1990, pp. 169-186; Sandelowski, 1995b.) A single case may he sufficient to display something of substantive importance, but Morse (1994) suggests using at least six participants in studies where one is trying u> underhand the esscmr- nt experience, Morse also suggests 30-50 interviews for ethnographies and grounded theory studies. Finding themes and budding theory may require fewer cases than comparing across group» and testing hypotheses or models. Once the researcher has established a sample of texts, the next step is to identify the basic units of analysis. The unils may be entire texts (books, 274 Dato MatxHjvnwnt and Analysis Methods intervieW Iters start with some general themes derived from reading the literature and add more themes and subihemes as they go. Shelley (1992) followed ilns advice in her study of how social networks alle, t people with end stage kidney disease. She used flic Outline of Cultural Materials (Murdoch, 1971) as the basis of her coding scheme and ihen added addition.il themes based on a close leading of the text. Itulin.r (1979) lists 10 different sources of themes, including literature reviews, professional 275 57 9 45 METHODS OF COLLECTING AND ANALYZING LMI'IHICAL MATERIALS i _^__^^^^^^_^^_^^^^^^^^^^^___^^__^____^^_^^_ definitions, local commonscnsc constructs, .md researchers' values and prior experience*. He also notes thai investigators' general theoretical ori-OntatiOflt, die richness oř the existing literature, ami the characteristics of the phenomena being studied influence the themes researchers are likely to find. No matter how the researcher actually does inductive coding, by the time he or she has identified the themes and refined ihetn to the point where t tic v can be applied to an entire corpus of texts, a lot of interpretive «úlVMS h.is .ilready been done. Miles and llubcniuii (1994) say simply, •< oding is analysis''(p. 56). Buiklirtq Codobooks («uli'lmoks arc simply organized lists of codes (nlten in hierarchies). ] |nw i n .c.n, lu'i can develop a codebook iv - ovi rod in detail by Dey (1993, pp. 95-151), Crabtrec and Miller (1992), and Miles and Huhermnn (199**, pp. 55-72). MacQuecn, McLellan, Kay, and Milstcin (1998) suggest that a good codebook mould include a detailed description of each code, inclusion and exclusion criteria, and exemplars of real text for each theme. If a theme is particularly abstract, we suggest that the researcher also provide examples of the theme's boundaries and even some cases that are closely related but not included within the theme. Coding is lUppoacd io be data reduction, not proliferation (Miles, 1979, pp. 593-594). The codes themselves arc nmemona devices used to identify or mark the specific themes in a text. They can be either words «r numbers— whatever the researcher finds easiest to remember and to apply. Qualitativ! researchers working as a team need to ,igree up front on what lo include in their codebook. Morse (1994) suggests beginning the pi i >ccss with a group meeting. MacQueen et al. (1998) suggest that a single team member should be designated "Keeper of the Codebook"—we strongly agree. Good codebortks arc developed and refined as the research goes along. Kurasaki < 1997) interviewed 20 sansei—third-generation Japanese Amer-ii .ms and used a grounded theory approach to do hei analysis of ethnic identity. She started with seven major themes, As the analysis progressed, ihfl -plit the major themes into siiblhcmcs. ľvcutiially, she combined two of the major themes and wound up with six major thnncsand a total of IS sublheines. (KicHards Öc Richards, ľ''»!, disuiss the theoretical principles related to hierarchical coding strmiiires thai emerge out of the data. 276 Dato Monogemonf oner Aj>ntysls Mrtfíiods AraujOi 1995, uses an example from his own rastattt) <>n the traditional British manufacturing industry to describe the process of designing and ' ■ I * 11 n 11; hierarchical codes.) Tha development and refinement of coding categories have long been COntral tasks in classical content analysis (see llerelson, 1952. pp. 147-168; Holsti, 1969, pp. 95-126) and are particularly important in the construction of concept dictionaries (Deese, 1969; Stone, Dunphy, Smith, & Ogilvie, 1966, pp. 134-168). Krippendorf (1980, pp. 71-84) and Carey, Morgan, .md Oxtoby (1996) note that much of codebook refinemenr comes during the training of coders to mark the text and in the act of checking lor intcrcodcr agreement. Disagrectnrm aiming multiple coders shows when the codebook is ambiguous and confusing. The first run also allows die researcher to identify good examples CO include in the codebook. Morhng Texts The act of coding involves the assigning of codes to contiguous units of text. Coding serves two distinct purposes in qualitative analysis. First, codes au is/«*) to mark off text in a corpus In: lain retrieval <>i indexing, lags in- noi i-.., h iated with any fixed unit! "I C*XtJ Hi.-v < .im mark simple phrases oi extend across multiple pages Second, codes act M values assigned to fixed units (sec Bernard, 1991, 1994; Seidel Öc Kelle, 1995). Here, codes are nominal, ordinal, or ratio wale values that are applied to fixed, nonoverlapping units of analysis. The nonoverlapping units can be ■i I -.u. h .is paragraphs, pages, documents), episodes, cases, or persons. Code» as tags arc associated with grounded theory and schema analysis {reviewed below). Codes as values are associated with classic content analysis and content dictionaries. The two types of codes are not mutually exclusive, but the use of one gloss—code—for both concepts can be misleading. Aruily/ifu) Chunks of Te-'s Building Conceptual Models (JnCfl Che researcher identifies a set it things (i hemes, concepts, beliefs, behaviors)) the next step is to identify how these things SM linked to each other in a theoretical model (Miles Öi Huherm.in, 1994, pp. 134-137). Models aie sets of abstract constrmts and the relationships among them 277 9738 METHODS OF COLLECTING ANO ANALYZING EMPIRICAL MATERIALS (Bulmer, 1979). Grounded theory, schema analysis, ethnographic decision modeling, and analytic induction all include model-building phases. Once a model starts to take shape, the researcher looks for negative cases—cases that don't fit the model. Negative cases either disconfirm parts of a model or suggest new connections that need to be made. In either instance, negative cases need to be accommodated. Negative case analysis is discussed in detail by Becker, Geer, Hughes, and Strauss (1961, pp. 37-45), Strauss and Corbin (1990, pp. 10S-109), Lincoln and Guba [1985, pp. 309-313), Dey (1993, pp. 226-233), Miles and Huberman (J994, p. 271), and Becker (1998), and is used by schema analysts {Quinn, 1997), ethnographic decision modelers (Gladwin, 1989), and scholars who use analytic * induction (Bloor, 1976; Cressey, 1953/1971; Lindesmith, 1947/1968). ,_ In ethnographic decision modeling and in classical content analysis, models are built on one set of data and tested on another. In their original formulation, Glaser and Strauss (1967) emphasized that building grounded theory models is a step in the research process and that models need to be validated. Grounded theorists and schema analysts, today are more likely to validate their models by seeking confirmation from expert informants than by analyzing a second set of data. For example, Kearney, Murphy, and Rosenbaum (1994) checked the validity of their model of crack mothers' experiences by presenting it to knowledgeable respondents who were familiär with the research. Regardless of the kind of reliability and validity checks, models are simplifications of reality. They can be made more or less complicated and may capture all or only a portion of the variance in a given set of data. It is up to the investigator and his or her peers to decide how much a particular model is supposed to describe. Below we review some of the most common methods researchers use to analyze blocks or texts. These include grounded theory, schema analysis, classical content analysis, content dictionaries, analytic induction, and ethnographic decision tree analysis. Grounded Theory Of Grounded theorists want to understand people's experiences in as rigorous and detailed a manner as possible. They want to identify categories and concepts that emerge from text and link these concepts into 278 Data Management and Analysis Methods substantive and formal theories. The original formulation of the method (Glaser Sc Strauss, 1967) is still useful, but later works are easier to read and more practical (Charmaz, 1990; Lincoln 6c Guba, 1985; Lonkila, 1995; Strauss, 1987). Strauss and Corbin (1990), Dey (1993), and Becker (1998) provide especially useful guidance. (For some recent examples of grounded theory research, sec Hunt & Ropo, 1995; Irurita, 1996; Kearney et al., 1994; Kearney, Murphy, Irwin, & Rosenbaum, 1995; Sohier, 1993; Strauss & Corbin, 1997: Wilson & Hutchinson, 1996; Wright, 1997.) Grounded theory is an iterative process by which the analyst becomes more and more "grounded" in the data and develops increasingly richer concepts and models of how the phenomenon being studied really works. To do this, the grounded theorist collects verbatim transcripts of interviews and reads through a small sample of text (usually line by line). Sandelowski (1995a) observes that analysis of texts begins with proofreading the material and simply underlining key phrases "because (hey make some as yet inchoate sense" (p. 373). In a process called "open coding," the investigator identifies potential themes by pulling together real examples from the text (Agar, 1996; Bernard, 1994; Bogdan Sc Biklen, 1992; Lincoln 8c Guba, 1985; Lofland & Lofland, 1995; Strauss & Corbin, 1990; Taylor & Bogdan, 1984). Identifying the categories and terms used by informants themselves is called "in vivo coding" (Strauss 8c Corbin, 1990). As grounded theorists develop their concepts and categories, they often decide they need to gather more data from informants. As coding categories emerge, the investigator links them together in theoretical models. One technique is to compare and contrast themes and concepts. When, why, and under what conditions do these themes occur in the text? Glazer and Strauss (1967, pp. 101-116) refer to this as the "constant comparison method," and it is similar to the contrast questions Spradley (1979, pp. 160-172} suggests researchers ask informants. (For other good descriptions of the comparison method, see Glaser, 1978, pp. 56-72; Strauss Sc Corbin, 1990, pp. 84-95.) Another useful tool for building theoretical models is the conditional matrix described by Strauss and Corbin (1990, pp. 158-175). The conditional matrix is a set of concentric circles, each level corresponding to a different unit of influence. At the center are actions and interactions; the outer rings represent international and national concerns, and the inner rings represent individual and small group influences on action. The 279 METHODS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS •no' matrix is designed to help investigators to be more sensitive to conditions, actions/inreractions, and consequences of a phenomenon and to order these conditions and consequences into theories. Mcmoing is one of the principal techniques for recording relationships among themes. Strauss and Corbin (1990, pp. 18, 73-74,109-129,197-219) discuss three kinds of memos: code notes, theory notes, and operational notes. Code notes describe the concepts that are being discovered in "the discovery of grounded theory." In theory notes, the researcher tries to summarize his or her ideas about what is going on in the text. Operational notes are about practical matters. Once a model starts to take shape, the researcher uses negative case analysis to identify problems and make appropriate revisions. The end results of grounded theory are often displayed through the presentation of segments of text—verbatim quotes from informants—as exemplars of concepts and theories. These illustrations may be prototypical examples of central tendencies or they may represent exceptions to the norm. Grounded theory researchers also display their theoretical results in maps or the major categories and the relationships among themXKearney et ah, 1995; Miles &c Huberman, 1994, pp. 134-137). These "concept maps" are simitar to the personal semantic networks described by Leinhardt (1987, 1989), Strauss (1992), and D'Andrade (1991) (see below). Schemo Analysis Schema analysis combines elements of the linguistic and sociological traditions. It is based on the idea that people must use cognitive simplifications to help make sense of the complex information to which they are constantly exposed (Casson, 1983, p. 430). Schänk and Abelson (1977) postulate that schemata—or scripts, as they call them—enable culturally skifled people to fill in details of a story or event. It is, says Wodak (1992, p. 525), our schemata that lead us to interpret Mona Lisa's smile as evidence of her perplexity or her desperation. From a methodological view, schema analysis is similar to grounded theory. Both begin with a careful reading of verbatim texts and seek to discover and link themes into theoretical models. In a series of articles, Quinn {1982,1987, 1992,1996, 1997) has analyzed hundreds of hours of interviews to discover concepts underlying American marriage and to show how these concepts arc tied together. Quinn's {1997) method is CO 280 Data Management and Analysis Methods "exploit clues in ordinary discourse for what they tell us about shared cognition—to glean what people must have in mind in order to say the things they do" {p. 140). She begins by looking at patterns of speech and the repetition of key words and phrases, paying particular attention to informants' use of metaphors and the commonalities in their reasoning about marriage. Quinn found that the hundreds of metaphors in her corpus of texts fit into just eight linked classes, which she calls lastingness, sharedness, compatibility, mutual benefit, difficulty, effort, success {or failure), and risk of failure. Metaphors and proverbs are not the only linguistic features used to infer meaning from text. D'Andrade (1991) notes that "perhaps the simplest and most direct indication of schematic organization in naturalistic discourse is the repetition of associative linkages" (p. 294). He observes that "indeed, anyone who has listened to long stretches of talk—whether generated by a friend, spouse, workmate, informant, or patient—knows how frequently people circle through the same network of ideas" (p. 287). In a study of blue-collar workers in Rhode Island, Claudia Strauss {1992) refers to these ideas as "personal semantic networks." She describes such a network from one of her informants. On rereading her intensive interviews with one of the workers, Strauss found that her informant repeatedly referred to ideas associated with greed, money, businessmen, siblings, and "being different." She displays the relationships among these ideas by writing the concepts on a page of paper and connecting them with lines and explanations. Price (1987) observes that when people tell stories, they assume that their listeners share with them many assumptions about how the world works, and so they leave out information that "everyone knows." Thus she looks for what is not said in order to identify underlying cultural assumptions (p. 314). For more examples of the search for cultural schemata in texts, see Holland's {1985) study of the reasoning that Americans apply to interpersonal problems, Kempton's (1987) study of ordinary Americans' theories of home heat control, Claudia Straitss's (1997) study or what chemical plant workers and their neighbors rhink about the free enterprise system, and Agar and Hobbs's {198J) analysis of lrow an informant became a burglar. We next turn to the two other methods used across the social sciences for analyzing text: classical content analysis and content dictionaries. 281 METHODS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS Displaying Concepts and Models Visual displays are an important pare of qualitative analysis- Selecting key quotes as exemplars, building matrices or forms, and laying theories out in the ŕorm of flowcharts or maps are all potent ways to communicate ideas visually to others. Models arc typically displayed using boxes and arrows, with the boxes containing themes and the arrows representing the relationships among them. Lines can be unidirectional or bidirectional. 'For example, taxonomies are models in which the lines represent the .super- and subordinate relationships among items. Relationships can include causality, association, choices, and time, to name a few. A widely used method for describing themes is the presentation of direct quotes from respondents—quotes that lead the reader to understand quickly what it may have taken the researcher months or years to figure out. The researcher chooses segments of text—verbatim quotes from respondents—as exemplars of concepts, of theories, and of negative cases-Ryan (1999) has used multiple coders to identify typical quotes. He asks 10 coders to mark the same corpus of text for three themes. Ryan argues that the text marked by all the coders represents the central tendency or typical examples of the abstract constructs, whereas text marked by only some of the coders represents less typical examples and is more typical of the "edges" of the construct. Tables can be used to organize and display raw text or can be used to summarize qualitative data along multiple dimensions (rows and columns). The cells can be filled with verbatim quotes (Bernard Sc Ashton-Voyoucalos, 1976; LeinhardtČVc Smith, 1985, p. 254; Miles ScHubermaii, 1994, p. 130), summary statements (Yoder, 1995), or symbols (Fjellman&c Gladwin, 1985; Van Maanen, Miller, AC Johnson, 1982). (For a range of presentation formats, see Bernard, 1994; Miles & Huberman, 1994; Werner & Schoepflc, 1987.) Classical Content Analysis Whereas grounded theory is concerned with the discovery of data-induced hypotheses, classical content analysis comprises techniques for reducing texts to a unit-by-variable matrix and analyzing that matrix quantitatively to test hypotheses. The researcher can produce a matrix by applying a set of codes to a set of qualitative data (including written texts as well as audio and video media). Unlike grounded theory or schema 282 Data Management one Analysis Methods analysis, content analysis assumes that the codes of interest have already-been discovered and described. Once the researcher has selected a sample of texts, the next step in classical content analysis is to code each unit for each of the themes or variables in the codebook. This produces a unit-by-variable matrix that can be analyzed using a variety of statistical techniques. For example, Cowan and O'Brien (1990) tested whether males or females are more likely to be survivors in slasher films. Conventional wisdom about such films suggests that victims are mostly women and slashers are mostly men. Cowan and O'Brien selected a corpus of 56 slasher films and identified 474 victims. They coded each victim for gender and survival. They found that slashers are mostly men, but it turned out that victims are equally likely tö be male or female. Women who survive are less likely to be shown engaging in sexual behavior and arc less likely to be physically attractive than their nonsurviving counterparts. Male victims are cynical, egotistical, and dictatorial- Cowan and O'Brien conclude that, in slasher films, sexually pure women survive and "unmitigated masculinity" leads to death (p- 195). The coding of texts is usually assigned to multiple coders so that the researcher can see whether the constructs being investigated are shared and whether multiple coders can reliably apply the same codes. Typically, investigators first calculate the percentage of agreement among coders for each variable or theme. They then apply a correction formula to take account of the fact that some fraction of agreement will always occur by chance. The amount of that fraction depends on the number of coders and the precision of measurement for each code. If two people code a theme present or absent, they could agree, ceteris paribus, on any answer 25% of the time by chance. If a theme, such as wealth, is measured ordinally (low, medium, high), then the likelihood of chance agreement changes accordingly. Cohen's (1960) k;ippa, or K, is a popular measure for taking these chances into account. When K is zero, agreement is whar might be expected by chance. When K is negative, the observed level of agreement is less than one would expect by chance. How much intercoder agreement is enough? The standards are still ad hoc, but Krippendorf (1980, pp. 147-148) advocates agreement of at least .70 and notes that some scholars (e.g., Brouwer, Clark, Gerbner, oc Kri$pendorf, 1969) use a cutoff of. 80. Flciss (1971) and Light (1971) expand kappa to handle multiple coders. For other measures of intercoder agreement, sec Krippendorf (1980, pp. 147-154) and Craig a case is found that doesn't fit, then, under the rules of analytic induction, the alternatives are to change the explanation (so that you can include the new case) or redefine the phenomenon (so that you exclude the nuisance case). Ideally, the process continues until a universal explanation for all known cases or a phenomenon is attained. Explaining cases by declaring them all unique is a tempting but illegitimate option. Classic examples of analytic induction include Lindcsmith's (1947/1968) study of drug addicts, Cressey's (1953/1971) study of embezzlers, and McQeary's (1978) study of how parole officers decide when one oftheir charges is in violation of parole. For a particularly cleať example of the technique, see Bloor's (1976, 1978) analysis of how doctors decide whether or not to remove children's tonsils. Ragin (1987, 1994) formalized the logic of analytic induction, using a Boolean approach, and Roníme (1995) applies the approach to textual data. Boolean algebra involves just two states (true and false, present and absent), but even with such simple inputs, things can get very complicated, very quickly. With just three dichotomous causal conditions (A and not A, B and not B, and C and not C) and one outcome variable (D and not D), there are 16 possible cases: A, B, C, D; A, not B, C, D; A, B, not C, D; and so on. Boolean analysis involves setting up what is known as a truth table, or a matrix of the actual versus the possible outcomes. (For more on truth tables and how they are related to negative case analysis, see Becker, 1998, pp. 146-214.) Schweizer (1991, 1996) applied this method in his analysis of conflict and social status among residents of Chen Village, China. (For a discussion of^Schweizer's data collection and analysis methods, sec Bernard 6c Ryan, 1998.) All the data about the actors in this political drama were extracted from a historical narrative about Chen Village. Like classic content analysis and cognitive mapping, analytic induction requires that human coders 286 Data Management and Analysis Methods read and code text and then protluče an cvcnt-by-variablc matrix. The object or the analysis, however, is not to show the relationships among all codes, but to find the minimal set or logical relationships among the concepts that accounts for a single dependent variable. With more than three variables» the analysis becomes much more difficult. Computer programs such as QCA (Drass, 1980) and ANTHROPAC (Borgatti, 1992) test all possible multivariate hypotheses and find the optimal solution. (QCA is reviewed in Weinman 6c Miles, 1995.) Ethnographic Decision Models Ethnographic decision models (EDMs) are qualitative, causal analyses that predict behavioral choices under specific circumstances. An EDM, often referred to as a decision tree or flowchart, comprises a series of nested if'tben statements that link criteria (and combinations of criteria) to the behavior of interest (Figure 7.7). EDMs have been used to explain how fishermen decide where to fish (Gatewood, 1983), what prices people decide to place on their products (Gladwin, 1971; Quinn, 1978), and which treatments people choose for an illness (Mathews 6c Hill, 1990; Ryan & Martinez, 1996; Young, 1980). EDMs combine many of the techniques employed in grounded theory and classic content analysis. Gladwin (1989) lays out the fundamental steps for building an ethnographic decision tree model. (For other clear descriptions of the steps, sec Hill, 1998; Ryan tic Martinez, 1996.) EDMs require exploratory data collection, preliminary model building, and model testing. First, researchers identify the decisions they want to explore and the alternatives that are available. Typically, EDMs are done on simple yes/no types of behaviors. They can be used, however, to predict multiple behaviors (Mathews & Hill, 1990; Young, 1980) as well as rhe order of multiple behaviors (Ryan 6c Martinez, 1996). Next, the researchers conduct open-ended interviews to discover rhe criteria people use to select among alternatives. The researchers first ask people to recall the most recent example or an actual—not a hypothetical—behavior and to recall why they did or did nor do rhe behavior. Here is an example from ajtudy we've done recently: "Think about the last time you had a can ofsomething to drink in your hand— soda, juice, water, beer, whatever. Did you recycle rhe can? Why [Why not]?" This kind of question generates a list or decision criteria. To 287 1 3 .r. C "H r:. 3 ■' 3 !; S 11 ' > j H '.j Ä o *-* ■" O i ■•--. i- 5 S ( n •2 á - 1 3 1 r*. h r» « Ul p '-> ŕl b o t/l ZSö laoogemení and Analysis Methods understand how these criteria might he linked, EDM researchers ask people to compare the latest decision with other similar decisions made in the past, Some researchers have used vignettes to elicit the relationships among criteria (e.g., Weiler, Ruebush, & Klein, 1997; Young, 1980). With a list of decision criteria in hand, the researchers' next step is to systematically collect data, preferably from a new group of people, about how each criterion applies or does not apply to a recent example of the behavior. "Was a recycling bin handy?" and "Do you normally recycle cans at home?" are 2 of the 30 questions we've asked people in our study of recycling behavior. The data from this stage are used to build a preliminary model of the decision process for the behavior under scrutiny. Cases that do not fit the model are examined closely and the model is modified. Researchers tweak, or tune, the model until they achieve a satisfactory level of postdictive accuracy—understood to be at least 80% among EDM researchers. Parsimonious models arc favored over more complicated ones. (For automated ways of building and pruning decision trees, see Mingers, 1989a, 1989b.) The process doesn't end there—the same data are used in building a preliminary model and in testing its postdictive accuracy. When EDM researchers feel confident in their model, they test it on an independent sample to see if it predicts as well as it postdicts. Typically, EDMs predict more than 80% of whatever behavior is being modeled, far above what wc expect by chance. (For more detailed arguments on how to calculate accuracy in EDMs, sec Ryan ŮC Martinez, 1996; Weiler et al.> 1997.) Because of the intensive labor involved, F.DMs have been necessarily restricted to relatively simple decisions in relatively small and homogeneous populations. Recently, however, wc found wc could effectively test, on a nationally representative sample, our ethnographically derived decision models for whether or not to recycle cans and whether or not to ask for paper or plastic bags at the grocery store {Bernard, Ryan, 6c Borgatti, 1999). EDMs can be displayed as decision trees (e.g., Gladwin, 1989), as decision rabies (Mathews & Hill, 1990; Young, 1980), or as sers of rules in the form of if-then statements (Ryan &c Martinez, 1996). Like componential analysis, folk taxonomies, and schcma/nalysis, EDMs represent an aggregate decision process and do not necessarily represent what is going on inside people's heads (Garro, 1998). 289 METHODS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS ♦ Breaking Down the Boundaries Tex i analysis as a research strategy perineales die so» 1.1I sciences, .nul the range of methods for conducting text analysis ll Inspiring« Investigators examine words, sentences, paragraphs, pnges, documents, ideas, meanings, puralinguistic features, and even what is missing Iron» the text. They interpret, mark, retrieve, and count. By turns, they apply interpretive analysis and numerical analysis. They UM text analysis for exploratory and ■• confirmatory purposes. Researchers identify themes, describe them, and . compare them across cases and groups. Finally, they combine themes into conceptual models and theories to explain and predict social phenomena. ľigurc 7,1 depicrs a broad range of analysis techniques found across the suii.il sciences. To conform our presentation with the literature on qualitative methods, we have organized these techniques according to the goals of the tnVMtigfUOrt and the kinds of texts to which the techniques arc typically applied. In this chapter, wc focus on the sociological tradition 1h.11 uses text as a "window into experience™ rather than du- linguistic tradition that dc-.. 1 IbeS how lexis arc developed ami structured, lexis such as conversation',, performances, and narratives ate iiualyml by investigators from boili 1 he sociological and linguistic traditions. All hough the agendas of the investigators may differ, we see no reason why many nt the sociological techniques we describe could not be useful in the linguistic tradition and vice versa. We also distinguish between those analyses associated with systematically elicited data and those associated with free-flowing texts. We argue, however, that these data-analytic pairings are ones of convention rather than necessity. Investigators want to (a) identify the range and salience of key items and concepts, (b) discover the relationships among these items iml tomepts, .md (c) build and test models linking thetc concepts together. They use free-listing tasks, KWIC, word counts, and the exploratory phases .»I grounded theory, schema analysis, ami I-DM to discover potentially useful themes and concepts. ReeotrcherS use pile sorts, paired comparisons, triads lesis, frame substitution tasks, semantic networks, cognitive maps, content analysis and content dictionaries, and che modeling phases ol grounded theory, schema analysis, ami IÍDM to discover how abstract concepts are related to each other. They display the relationships as models or frameworks. These 290 Doto Monogement and Anofyin Atof/ior/s frameworks include form.il models that rely tin Boolean logic (componcn-iial analysis and analytic induction), hierarchical models (taxonomies and ethnographic decision models), probabilistic models (.lassie content analysis and content dictionaries), and more abstract models such as those produced by grounded theory and schema analysis, below we describe two important examples of studies in which researchers combined methods to understand their data more fully. Jehn and Doucct (1996, 1997) used word counts, classical content analysis, and mental mapping to examine conflicts among Chinese and U.S. business associates. They asked 76 U.S. managers who had worked in Sino-Amcrican joint ventures to describe recent interpersonal conflicts With business partners. Each person described a situation with a same-culture manager and a different-cultural manager. The researchers made sure that each manager interviewed included information about his or her relationship to the other person,who was Involved, what the conflict was about, what caused the conflict, and how the conflict was resolved. After collecting the narratives, Jehn and Doucct asked their informants to help identify the cmic themes in the narratives, lirsi, they generated Separate lists ol words from the intcrculimal mu\ miracultural conflict nnrratlveSi They asked three expatriate managers to QCl RS judges and to identify all the words that were related to conflict ľhey settled on .1 list of 542 conflict words from the intcrcultural lisi ami 2-12 conflict words from the intracultural list. Jehn and Doucct then asked the three judges to sort the words into piles or categories. The experts identified 1S subcategories for the intcrcultural data (things like conflict, expectations, rules, power, and volatile) and 15 categories for the mtraciiltur.il data (things like coh-flict, needs, standards, power, contentious, and lose). Taking into consideration the rotal number of words in each corpus, conflict words were used mote m intracultural interviews and resolution terms were more likely to be used in intcrcultural interviews. Jehn and Doucct also used traditional content analysis on their data. 1 he had two coders read the 152 conflict scenarios (76 intracultural and '() iiiiiiuilntr.il) and evaluate (on a 5-point scale) each on 27 different memos they had identified from the literature« This produced two 76 x 27 s.eiiario-by-thcme profile matriccs-ronc for the inltaailtural conflicts and one for the interciilmral conflicts. The first three factors from the intcrcultural matrix reflect (a) interpersonal animosity and hostility, (b) aggravation, and (c) the volatile nature of the conflict. The first two 291 417� MErHOOS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS i_______________________________^^—^^^^^^^ »no1 (aCtOM Írom the intracultural matrix reflect (-0 hatred .mil animosity with a volanie nuuitc :\nó (b) conflicts conducted calmly with little verbal tntciiMtv Finally. Jchn and Doucci identified the iO miraculuir.il and the 10 mtcrcultural scenarios that (hey felt werť the clearest and pithiest. They recruited 50 more expatriate managers to assess the similarities (on a 5-point scale) of 60-120 randomly selected pairs of scenarios. When combined across informants, the manager«' judgments produced two aggrc-ifttc, sccnario-by-scenario, similarity matrices—one for the innaculmral conflicts and one for the inrercultural conflicts. Multidimensional scaling of the inrercultural similarity data identified four dimensions: (a) open versus resistant to change, (b) situational causes versus individual traits, (c) high- versus low-resolution potential based on trust, and {d) high-vcrsus low-rcsoluiioii potential based mi patience, Scaling ol ihe intra Cultural similarity data identified lour different dimensions: (a) high versus low cooperation, (b) high versus low confrontation, (c) problem solving versus accepting, and (d) resolved versus ongoing. I In- work of Jehu mu\ I >omet is impressive because the analysis of the data from these tasks produced different Htt of themes. All three emicully induced theme sets have some iniuilivc appeal, and all three yield analviu results that are useful. The researchers could have also used the techniques of grounded theory or schema analysis to discover even more themes. Jchn and Doucet arc not the only researchers ever to combine different analytic techniques. In a series of articles on young adult "occasional" drug users, Agar (1979, 1980, 1983) used grounded theory methods to build models of behavior. He then used clinical content analysis to test his hypotheses. Agar conducted and transcribed three interviews with each of his three informants. In his 1979 article, Agar describes his initial, intuitive analysis. He pulled all the statements that pertained to informants' interactions or assessments of other people. I-lc then looked at the statements and sorted them into piles based on their content He named each pile as a theme and assessed how the themes interacted. He found that he had three piles. The first contained KamMMI In which the informant was express mg negative feelings toward a person m a dominant social position. I he second was made up of staiements emphasizing the other's knowledge or awareness. The statements in the third IBUÜI cluster emphasized thfl Im-portance of change or openness to new experiences. 293 Data Management and Anorys.* Method* From this Intuitive analysis, Agar felt that Ins informants were telling him that those m authority were only interested in displaying their authority unless they had knowledge or awareness; that knowledge or awareness comes through openness to new experience; and that mmt in authority are closed to new experience or change. To test his intuitive understanding of the data, Agar (1983) used all the statements from a single informant and coded the statements for their role type (kin, friend/acquaintance, educational, occupational, or other), power (dominant, symmetrical, subordinate, or undetermined), and affect (positive, negative, ambivalent, or absent), Agar was particularly interested in whether negative sentiments were expressed toward those in dominant social roles. For one informant. Agar found that out of 40 statements coiled as dominant, 32 were coded negative ami 8 were coded positive. For the id statements coded as symmetrical, 10 were coded positive and 16 negative, lending support to his original theory. Next, Agar looked closely at the deviant cases—thr S suteuienis where the informant expressed positive affect toward a person m a dominant role. These counterexamples suggested that tlie positive allect was expressed inward a dominant um ial Oth« when thi IO< lal othei possessed, or was communicating to the informant, knowledge that the Informant valued. Finally, Agar (I 980) developed a more systematic questionnaire to test his hypothesis further. He selected 12 statements,-I Ironi each of the control, knowledge, and change themes identified earlier. He matched these statements with eight roles from the informant's transcript (father, mother, employer, teacher, friend, wife, coworker, and teammate). Agar then returned to his informant and asked if the resulting statements were true, false, or irrelevant. (In no case did the informant report "irrelevant.") Agar then compared the informant's responses to his original hypotheses. He found that on balance his hypotheses were correct, hut discrepancies between his expectations and his results suggested areas for further research. These examples show that investigators can apply one «■< Imiquc to different kinds ol J.u.i ind they can apply multiple techniques to the nunc data set. Texi analysis is used by avowed positivism and mterprctivists alike. As we have argued elsewhere (Bernard, 1991; Bernard fk Ryan, 1998), methnď. are simply tools that belong tu everyone. 293 11913243 90 29 METHODS OF COLLECTING AND ANALYZING EMPIRICAL MATERIALS ■ ♦ What's Next? We do not want to minimize the profound intellectual differences in the epistemological positions of positivists and intcrpretivists. We think, however, that when researchers can move easily and cheaply between qualitative and quantitative data collection and analysis, the distinctions between the two epistemological positions will become of less practical importance. That is, as researchers recognize the full array of tools at their disposal, and as^these tools become easier to use, the pragmatics of research will lessen the distinction between qualitative and quantitative data and analysis. The process is under way—and is moving rast—with the development of increasingly useful software tools for qualitative data analysis. Useful tools create markets, and market needs create increasingly useful tools.; Qualitative data analysis packages (ATLAS/ti, NUD "1ST, Codc-A-'Icxt, the Ethnograph, AnSWR, and others) have improved dramatically over the past few years (Fischer, 1994; Kelle, 1995; Weitzman & Miles, 1995). These products, and others, make it easier and easier for researchers to identify themes, build codebooks, mark text, create memorand develop theoretical models. Based loosely on a grounded theory type of approach to qualitative analysis, many program suites have recently folded in techniques from classical content analysis. Several programs, for example, allow researchers to export data to matrices that they can then analyze using other programs. Investigators, however, remain constrained by program-defined units of analysis—usually marked blocks of text or informants. Researchers need the flexibility to create matrices on demand, whether they be word-by-themc or word-by-informanl matrices for word analysis and sentence-by-codc or paragraph-by-code matrices for content analysis. A series of word analysis functions would greatly enhance the automated coding features found in programs that are geared to the interests of scholars in the grounded theory school. Investigators should be able to code a section of text using grounded theory, then identify the key words associated with each t heme. They should be able to use key words to search for additional occurrences of the theme in large corpuses of text. When programs make it easy ro use multiple coders and tô identify intercoder agreements and disagreements systematically, researchers will be better able to describe themes and to train assistants. Adding a variety of measures for calculating intercoder agreement, which only some pro- 294 Data Management and Analysis Methods grams do, would also be helpful. Some programs offer researchers the option of recording the marking behavior of multiple coders, yet offer no direct way to measure intercoder agreement. The evolution of text analysis software is just beginning. Some 15 years ago, spell checkers, thesauruses, and scalable fonts were all sold separately. Today, these functions arc integrated into all full-featured word-processing packages. Just 10 years ago, graphics programs were sold separately from programs that do statistical analysis. Today, graphics functions are integrated into all full-featured packages for statistical analysis. As programmers of text analysis software compete for market share, packages will become more inclusive, incorporating methods from both sides of the epistemological divide. It can't happen too soon. ♦ Notes 1. MDS displays arc highly evocative. They beg lo be interpreted. In face, they must be interpreted. Why ate some illnesses at ihe rop of Higurc 7.3 and some at the borcom; We think (lie illnesses at the top are more of (lie eh tonic variet)', whereas those at the botcom arc more acute. We also think that the iUnesseson the left ate less serious than those o n the right. We can test ideas like these by asking key informants to help us understand the arrangement of the illnesses in the MDS plot. {For more examples of mental maps, see Albert, 1991; D'Andradcctal., 1972; Rrickson. I997.J (There is a formal method, called property fining analysis, or PROFIT, fur testing ideas about the distribution of items m .in .MDS map. This method is based on linear regression. See Kruskal & Wish, 1978.) 2. Alternatively, profile matrices (the usual ching-by-varinblc attribute matrix ubiquitous in the social sciences) can be converted to similarity matrices (thing-by-thing matrices in which the cells contain measures of similarity among paits of things) and ihcn analyzed with MDS (for stcp-by-stcp instructions, sec Bvrgattu 1999). ♦ References Agar, M. {1979). Themes revisited; Some problems in cognitive anthropology. Discourse Processes, 2, 11-31. Agar, M. (1980). Getting better quality stuff: Methodological competition in an interdisciplinary niche. Urban Life, '), 34-50. Agar, M. (1983). Microcomputers ns field Tools. Computers and the Humanities, 17, 19-26. Agar, M. (1996). Speaking of ethnography (2nd ed.). Thousand Oaks, CA: Sage. 295