Generalization in and from Qualitative Analysis
Joseph A. Maxwell and Margaret Chmiel
Generalization, in research, refers to extending research results, conclusions, or other accounts that are based on a study of particular individuals, settings, times, or institutions, to other individuals, settings, times, or institutions than those directly studied (Polit and Beck, 2010). A widely accepted view, among both quantitative and qualitative researchers, is that there are two main types of, or strategies for, generalization in social research, typically (although not necessarily) associated with quantitative and qualitative research, respectively (Gobo, 2008: 195-6). Yin (2003: 32-3), addressing generalization in case study research, called these two strategies statistical generalization and analytic generalization; other terms for these (or similar) types include enumerative induction and analytic induction (Znaniecki, 1934) and empirical generalization and theoretical generalization (Hammersley, 2008: 36) or theoretical inference (Hammersley, 1992: 86ff).
Yin described statistical generalization as occurring when 'an inference is made about a population (or universe) on the basis of
empirical data collected about a sample' (2003: 32). This definition does not require that statistical methods be used, and the term is therefore somewhat misleading. For this reason, and because qualitative research rarely uses probability sampling (see Rapley, Chapter 4, this volume), we will use the term empirical generalization in the rest of this chapter; this seems to us to better capture what is most central to such generalization in qualitative research - that it relies on the descriptive representativeness of the sample (or set of participants or settings on which data are actually collected), in terms of the distribution of properties of individuals or groups, for the larger population to which the researcher wants to generalize.
Analytic generalization, in contrast, is that in which 'a previously developed theory is used as a template with which to compare the empirical results of the case study. If two or more cases are shown to support the same theory, replication may be claimed' (Yin, 2003: 32-3); Yin elsewhere described this strategy as 'generalizing to theory' (1984: 39;
GENERALIZATION IN AND FROM QUALITATIVE ANALYSIS
541
see also Kelle, Chapter 38, this volume). Yin argued that this is the appropriate form of generalization for case study research, and that it is similar to the logic by which experiments are generalized.1 This term has been adopted by some qualitative researchers to describe a sort of generalization that qualitative research can support (e.g., Schwandt, 1997: 2-3, 58), although dropping Yin's restriction to prior theory, since theories in qualitative research are often inductively developed (see Reichertz, Chapter 9; Thornberg and Charmaz, Chapter 11; and Kelle Chapter 38, this volume).
However, other qualitative researchers have proposed a third approach to generalization, in which the emphasis is not on the generality of the findings or interpretations so much as on their case-to-case transferability; this has become the usual term for this approach (Guba and Lincoln, 1989: 241-2; Jensen, 2008; Lincoln and Guba, 1985: 114— 15; Schwandt, 1997: 57-60). Transferability does not require the discovery of the general conditions under which a finding or theory is valid; instead, it involves a transfer of knowledge from a study to a specific new situation. This shifts the responsibility for making generalizations from the researcher to the reader or potential user of the findings, and Misco (2007, cited by Polit and Beck, 2010) has called this 'reader generalizability.' Lincoln and Guba (1985; 1986; Guba and Lincoln, 1989) and Schofield (1990) identified some of the properties that a qualitative study must possess in order for such transferability to be possible, and Donmoyer (1990; 2008) developed a model for how transferability operates. Schwandt stated that Lincoln and Guba 'urge the investigator to provide sufficient details ... so that readers can engage in reasonable but modest speculation about whether the findings are applicable to other cases' (1997: 58).
These three strategies for generalization in qualitative research - empirical generalization, analytic generalization, and case-to-case transfer - have been widely recognized (Firestone, 1993; Polit and Beck, 2010).
However, these terms have often been interpreted in different ways. Polit and Beck argue that all of these models of generalization are idealized goals that are often not adequately supported by research publications, and that 'both quantitative and qualitative researchers uphold certain myths about adherence to the three models of generalization, and these myths hinder the likelihood that real opportunities for generalization will be pursued' (2010: 1451).
In what follows, we want to address an additional distinction among the types of generalization that are appropriate for qualitative research. Maxwell (1992) described this distinction as between internal generalizability and external generalizability. Internal generalizability refers to generalizing within the setting, institution, or case studied, to persons, events, times, and settings that were not directly observed (see Marvasti, Chapter 24, this volume), interviewed (see Roulston, Chapter 20, this volume), or otherwise represented in the data collected. For interview studies, this can also be seen as generalization to other aspects of the experiences, perspectives, actions, or relationships of the individuals interviewed than those that were addressed in the interview, that is, treating the individual as the 'case.' External generalizability, in contrast, refers to generalization beyond the case or cases specifically studied, to other persons or settings.
Brown-Saracino et al. (2008) made a similar distinction, between lower-order generalizability and higher-order generalizability. Lower-order generalizability is the generalizability of findings within the unit of analysis; higher-order generalizability is generalizability of findings across units of analysis of the same type (e.g., across similar organizations or neighborhoods). Mabry (2008), discussing generalization in case study research, likewise distinguished Ericksons'spetite generalizations (Erickson, 1986), generalizations within a case, from Firestone's case-to-population generalizations (Firestone, 1993). We believe that this
542
USING AND ASSESSING QUALITATIVE DATA ANALYSIS
distinction has important implications for generalizing from qualitative data and for the ways in which qualitative data analysis can support such generalizations.
These two types of generalization overlap Hammersley's and Yin's distinctions, but only partly, because while the first type, internal generalization, often involves empirical generalization (is the sample or selection actually observed or interviewed representative of the case, setting, or group?), the second, making inferences to, or across, cases or populations other than those studied or sampled, also often follows an 'empirical' logic, of deciding to what extent the features of some target population match those of the participants or settings of the study - what Donald Campbell called 'proximal similarity' (Polit and Beck, 2010: 1453). Analytic or theoretical generalization is fundamentally different from this, since the 'generalization' is initially to theory, rather than to a population or a universe of cases (Polit and Beck, 2010: 1452).
The distinction between internal and external generalization is not an absolute or clear-cut distinction, and intermediate or ambiguous examples are common. Someone doing research on school principals in a particular school district, for example, is rarely able to observe every school or interview every principal, and whether generalizations beyond the schools or principals actually observed or interviewed are seen as internal or external depends on how the researcher defines the units or cases studied. However, it is important to be aware of how the times and places actually observed may differ from those that were not observed, or the ways in which interviewed individuals' beliefs and perspectives, as expressed in the interviews, may be atypical, or contextually variable in ways that the interview does not capture. The concept of internal generalizability is intended to highlight this potential variability, and to help researchers become aware of the risks of unrefiectively extending their results and interpretations beyond what is justified by their data.
We see generalization in, and from, qualitative data analysis as central to generalization in qualitative research as a whole; the discussion of generalization (including transferability) in qualitative research has usually assumed that this pertains to generalization of the results or conclusions drawn from the analysis of the data. We will discuss internal and external generalization separately, identifying the implications of different approaches to generalization for each.
INTERNAL GENERALIZATION
Internal generalization is a key issue for qualitative data analysis. The validity of the results of such analyses, for the case, setting, or the collection of participants studied, depends on their internal generalizability to this case, setting, or collection of participants as a whole; for this reason, internal generalizability overlaps substantially with what are generally seen as validity issues2 (see Barbour, Chapter 34, this volume). Sampling (selection, see Rapley, Chapter 4, this volume) is particularly relevant for internal generalizability. Miles and Huberman (1984) asked, 'Knowing, then, that one cannot study everyone everywhere doing everything, even within a single case, how does one limit the parameters of a study?' (1984: 36). They argued:
Just thinking in sampling-frame terms is healthy methodological medicine. If you are talking with one kind of informant, you need to consider why this kind of informant is important, and, from there, which other people should be interviewed. This is a good, bias-controlling exercise.
Remember that you are not only sampling people, but also settings, events, and processes. It is important to line up these parameters with the research questions as well, and to consider whether your choices are doing a representative, time-efficient job of answering them. The settings, events, or processes that come rapidly to mind at the start of the study may not be the most pertinent or data-rich ones. A systematic review can sharpen early and later choices. (1984: 41)
GENERALIZATION IN AND FROM QUALITATIVE ANALYSIS
543
For example, if you are studying the patterns of interaction between a teacher and students in a single classroom, your account of that classroom as a whole is jeopardized if you have selectively focused on particular times, activities, or students and ignored others.3
This issue of representativeness is not only relevant to selection strategies, however; it also has important implications for data analysis. Representativeness, as a validity concern, pertains not simply (or even primarily) to the data sources used, or even to the data themselves, but to the conclusions, interpretations, or theories about the setting or participants that are drawn from the data (Brinberg and McGrath, 1985; Hammersley, 1992: 43-57; Maxwell, 2011; Shadish et al., 2002: 34). How these conclusions or interpretations are drawn is thus a key issue for internal generalizability.
An important aspect of internal generalizability is adequately understanding and representing the diversity in the phenomena of interest in the setting or group of people studied. Diversity is often underestimated in both quantitative and qualitative research (Maxwell, 1996; 2011). This can be the result of theoretical biases that emphasize similarities or common features at the expense of differences, or of methodological biases that obscure or neglect actual variation. We will address these two sources of bias in turn.
Theoretical biases toward uniformity are a threat to the internal generalizability of any analysis that assumes or is grounded in such theories. Postmodernism has been particularly critical of the imposition of such 'totalizing metanarratives' (Ayres, 2008; Olsson, 2008), and postmodernists have argued for more attention to diversity, seeing this as fundamental rather than superficial. For example, Rosenau identified as key characteristics of postmodernism its search for 'diversity rather than unity, difference rather than synthesis, complexity rather than simplification' (1992: 8). Wolf likewise saw diversity as central: 'The postmodernist goal is, I take it, to encourage the author to present
a less tidy picture with more contradictory voices' (1992: 53).
In particular, the traditional definition of 'culture,' as those beliefs, values, and practices that are shared by members of a community, inherently marginalizes and obscures the actual diversity within cultures and communities. Many anthropologists have challenged this definition, and advocated instead for what has been called a 'distributive' concept of culture, one that recognizes the prevalence and importance of intracul-tural diversity (e.g., Atran and Medin, 2008; Hannerz, 1992; Wallace, 1970; for a more detailed review of this issue, see Maxwell, 2011). Hannerz warned that even postmodernism has tendencies toward ignoring or suppressing diversity:
It is a problem of postmodernist thought that as it has emphasized diversity and been assertively doubtful toward master narratives, it has frequently been on the verge of becoming another all-encompassing formula for a macroanthropol-ogy of the replication of uniformity. (1992: 35)
In addition to such theoretical biases, both quantitative and qualitative methods contain methodological biases that tend to conceal the existence of diversity and make it more difficult to understand its nature and influence. These biases can undercut the value of an approach that has the theoretical potential to illuminate the extent and consequences of diversity. Thus, Strauss (1992) argued that Bourdieu's analysis of socialization in terms of 'habitus' (see Bohnsack, Chapter 15, this volume), the mental structures unconsciously created by individuals from the practices of everyday life, has exactly this problem:
In Outline [of a Theory of Practice] he never analyzes the habitus of any particular individuals, but instead, like all too many social researchers, makes assumptions about the contents of the habitus of his Kabyle informants on the basis of social facts such as the organization of their households or the rhythms of their agricultural calendar. This leads him to ignore the potential for intracultural variation and change that is built into his theory of habitus formation and to stress instead the reproduction of hegemonic relations, at least for
544
USING AND ASSESSING QUALITATIVE DATA ANALYSIS
'traditional' societies. In other words, although Bourdieu's theory takes us away from what I call 'fax' models of socialization, his own practice falls back into them. (Strauss, 1992: 9)
Such methodological biases include a lack of attention to selection issues, and emphasizing common features or themes and ignoring less prevalent ones (Maxwell, 2011: 64-5). For qualitative data analysis specifically, it is important to take account, not just of the intended selection strategy, but also of the actual selection of persons, settings, and times that the data represent, and the implications of this for the analysis, results, and interpretations. In addition, there is a strong and often unconscious tendency for researchers to notice supporting instances and ignore ones that do not fit their prior conclusions (Miles and Huberman, 1994: 263; Shweder, 1980).
A second issue for internal generalizability is understanding, and adequately theorizing (see Thornberg and Charmaz, Chapter 11, and Kelle, Chapter 38, this volume), the social and cultural processes that are operating in these settings or influencing these individuals. Arguments that an understanding of causal processes is a valid goal of qualitative research are increasingly common (Anderson and Scott, 2012; Donmoyer, 2012; Erickson, 1986; 2012; Hammersley, 2008: 80^; Maxwell, 2004; 2011; 2012a), and this emphasis on processes, rather than variables, in understanding causality is supported by much recent work in philosophy (e.g., Little, 2010; Putnam, 1999; Salmon, 1998; see Maxwell, 2011, for a more extensive discussion) and in social science more generally (e.g., Lawson, 2003; Mohr, 1996; Pawson, 2006; Sayer, 2000: 114-18). The acceptance of causal processes, rather than causal laws, as fundamental to causal explanation entails that causal inference is legitimate in single cases (Miles and Huberman, 1994; Scriven, 2008), and thus that such inferences can be a matter of internal as well as external generalizability.
Interviewing (see Roulston, Chapter 20, this volume) poses some special problems
for internal generalizability, because the researcher usually is in the presence of the person interviewed for only a brief period, and must necessarily draw inferences from what happened during that brief period to the rest of the informant's life, actions, and perspective. An account based on interviews may be valid as an account of the person's perspectives as expressed in that interview, but may miss other aspects of the person's perspectives that were not expressed in the interview, and can easily lead to false inferences about his or her perspectives or actions outside of that interview context.
In psychology, this lack of attention to context has been termed the fundamental attribution error (Nisbett, 2004) or correspondence bias (Gilbert and Mai one, 1995). The term refers to the assumption that a person's behavior in some situation is a result of fundamental properties of the person, rather than features of the situation, and can thus be generalized to the person's behavior in other situations. This assumption has often been challenged in psychology (e.g., Nisbett, 2004), and has been central to the 'traits vs. states' debate in psychology. It has also been challenged by postmodern approaches to identity (Holstein and Gubrium, 1999; Rosenau, 1992) that deny the existence of a coherent, integrated self that is constant across situations, leading some researchers to completely reject the idea that interviewing is a way of accessing participants' understandings, and to treat it strictly as an interactional event (Olsson, 2008).
While we would not go this far in rejecting the generalizability of the results of interviews, it is undeniable that the interview is itself a social situation, and inherently involves a relationship between the interviewer and the informant. Understanding the nature of that situation and relationship, how it affects what takes place in the interview, and how the informant's expressed views could differ in other situations, is crucial to the analysis of accounts based on interviews (Briggs, 1986; Maxwell, 2012b; Mishler, 1986; Weiss, 1994).
GENERALIZATION IN AND FROM QUALITATIVE ANALYSIS
545
STRATEGIES FOR INTERNAL GENERALIZATION IN QUALITATIVE DATA ANALYSIS
Qualitative researchers have many ways of assessing and improving internal generaliza-bility, including systematic sampling (see Rapley, Chapter 4, this volume) or selection decisions (Miles and Huberman, 1994) - for example, stratified or maximum variation sampling (Morgan, 2008) - and asking participants themselves about the typicality of their views or of the situations you observed. For data analysis, a key strategy is paying specific attention to the diversity of views, behaviors, or contexts represented in the data you have collected, deliberately searching for variability and for data that do not fit prior expectations (Maxwell, 2011; 2012b). These data must be analysed in ways that retain these differences and attempt to understand their significance. Thorne and Darbyshire (2005: 1108, cited by Polit and Beck, 2010: 1453), discussing qualitative health research, identified a number of practices that interfere with valid generalization, including premature closure of analysis and imposing an artificial coherence on the data.
An additional strategy for supporting internal generalization, although one that is underutilized in qualitative research, is analysing the data to provide numerical results about the frequency and distribution of observations, or the number of instances of a particular type of event or statement (Maxwell, 2010; Seale, 1999). Becker called this strategy quasi-statistics, arguing that 'One of the greatest faults in most observational case studies has been their failure to make explicit the quasi-statistical basis of their conclusions' (1970: 81-2). This can be used to identify and accurately characterize the diversity in the case or setting you are studying.
Using numbers in this way is not 'statistical,' and does not make a study 'quantitative' in the usual meaning of this term (Maxwell, 2010); it simply makes explicit, and more precise, the implicitly numerical nature of
claims such as that a particular activity, theme, or pattern is common, rare, or prevalent in the setting or interviews included in the study. The appropriate use of numbers not only allows you to test and support such claims, but also enables you to assess the amount of evidence in your data that bears on a particular conclusion or threat, such as how many discrepant instances exist and from how many different sources they were obtained. This strategy was used effectively in a classic participant-observation (see Marvasti, Chapter 24, this volume) study of medical students by Becker et al. (1961), which presented more than 50 tables and graphs of the amount and distribution of observational and interview data supporting their conclusions. In addition, numbers are important for identifying and communicating the diversity of actions and perspectives in the settings and populations you study (Heider, 1972; Sankoff, 1971; Zentella, 1990).
In addition, a goal of understanding specific, local, context-dependent processes has major implications for qualitative data analysis. The elucidation of such processes requires a different sort of analysis from the traditional coding and aggregation of data by coding categories (see Thornberg and Charmaz, Chapter 11, and Schreier, Chapter 12, this volume) that have been predominant in qualitative research. Abbott (1992) and Becker (1992; 2008) argued that narrative approaches to analysis (see Esin et al., Chapter 14, this volume) are far more useful for understanding processes than the traditional quantitative analysis of variables, and the same argument can be made for narrative analysis vs. coding. Coding inherently strips away both context and the sequencing of events, things that are intrinsic to process, leaving only the possibility of an aggregate understanding of the things coded. A discussion of how narrative and other sorts of connecting strategies for qualitative data analysis can be used for an understanding of process is beyond the scope of this chapter (see Maxwell and Miller, 2008, and Maxwell and Chmiel, Chapter 2, this volume, for such a discussion).
546
USING AND ASSESSING QUALITATIVE DATA ANALYSIS
To summarize, internal generalizability is mainly concerned with the representativeness of the data and conclusions for the case, settings, or individuals studied, and relies primarily on empirical generalization, rather than analytic generalization or transfer. It thus depends significantly on sampling/ selection issues; inadequate or unrepresentative selection can lead to flawed inferences about the case, setting, or individuals studied. These are problems that data analysis cannot by itself fully correct, although it can help you to identify such problems and provide ways to address them. However, other threats to internal generalizability, such as researcher bias or uniformist theoretical assumptions, can be addressed by data analysis strategies, including a deliberate search for data that are inconsistent with the emerging interpretation (see Willig, Chapter 10, this volume), the use of numbers to evaluate the actual variability and distribution of your data, and analyses appropriate for connecting data in ways that elucidate causal processes.
EXTERNAL GENERALIZATION
As noted above, external generalization in qualitative research is often assimilated, by quantitative researchers, to empirical generalization, and the imposition of quantitative views of generalization on qualitative research has frequently been criticized (e.g., Donmoyer, 1990; Guba and Lincoln, 1989). Bryman argued that:
There are grounds for thinking that the 'problem' of case study generalization entails a misunderstanding of the aims of such research. In particular, the misconception arises from a tendency to approach a case study as if it were a sample of one drawn from a wider universe of such cases. (1988: 90)
However, the external generalizability of qualitative studies is normally either to theories (see Kelle, Chapter 38, this volume) (analytic generalization), or through the transferability
of particular results or understandings to other cases, rather than to populations or universes (statistical generalization). For this reason, Bryman (1988: 50-4) and Yin (2003: 47-51) saw generalization in case study research as following a replication logic rather than a sampling logic, seeking to test the theory in other cases (similarly to multiple experiments) rather than to assess its representativeness for some larger population. Similarly, Donmoyer argued that 'good case studies employ theoretical constructs the way the historian of a particular revolution uses the construct of 'revolution' - to show not just its similarities to, but also its differences from, other revolutions' (1990: 196). Eisner described generalization as coming about in qualitative research via qualitative research's ability to 'bring about a set of observations or images' that can then facilitate 'the search and discovery processes when examining other situations' (1977: 270). That is, observations from qualitative studies are generalized in that they are used to create heuristics for other studies. He argued that in both quantitative and qualitative research, findings are used analogically or heuristically, but that the boundary conditions for developing an appropriate theory are seldom drawn.
For these reasons, the external generalizability of a qualitative study may depend on its lack of empirical or statistical generalizability, in the sense of being representative of a larger population. It may provide an account of a setting or population that is illuminating as an extreme case or 'ideal type,' one that highlights processes that are found in less visible form in many other cases. For example, Freidson's study of an innovative medical group practice (1975) made an important contribution to theory and policy precisely because this was a group for whom social controls on practice should have been most likely to be effective. The failure of such controls in this case not only highlights a social process that is likely to exist in other groups, but also provides a more persuasive
GENERALIZATION IN AND FROM QUALITATIVE ANALYSIS
547
argument for the unworkability of such controls than would a study of a 'representative' group. Similarly, Ruddin (2006) argued that studying an ideal case is a good way to falsify an existing generalization or theory. In such situations, the specific elements of the case produce knowledge that is itself general, namely, that the existing theory or knowledge is not a viable way to explain cases like this. In this case the general phenomena in question are best understood by seeking out a non-representative situation where we expect to find a particular effect. Here, the understanding of the general is attained by explicitly looking at an extreme.
Wievorka (1992) provided several instances of studies in which unrepresentative cases
were particularly valuable in supporting or disconfirming general theories about some social phenomena. For example, in one study (Goldthorpe et al., 1968-1969), the researchers, in order to test the view that the working class were being assimilated into middle-class society, selected a case that would be highly favorable to this position: workers who were extremely affluent. The finding that these workers still retained a clear working-class identity provided more convincing refutation of this theory than a study of 'typical' workers would.
A detailed example of how generalization is possible despite a lack of empirical representativeness is given in Example 1.
Example 1
Becker (1990) provided an example of how a theory of the process by which prisoners' deprivations create a distinctive prison culture can be generalized from men's to women's prisons, despite the fact that the actual prison culture is quite different in the two cases. Studies of men's prisons found that prisoners created an elaborate culture separate from the formal administration of the prison, including a black market for cigarettes, drugs, and various services, a convict government that kept order, and a strict code of conduct that prevented prisoners from providing information about other prisoners to guards or officials. The researchers theorized that this culture was a response to the deprivations of prison life; deprived of autonomy, many goods and services, and sexual relations with women, they organized ways to provide a degree of autonomy, some of these goods, and prison-specific homosexual relations that did not threaten their male identities.
Other researchers then tried to apply this theory to women's prisons. However, they did not find the sort of culture that existed in men's prisons. There was no convict government, and not much of an underground market for anything; prisoners incessantly informed on other prisoners; and instead of the sorts of sexual relations found in the men's prisons, the women developed pseudo-families, in which some women acted as husbands and fathers of a group of wives and daughters. Becker argued that these differences did not invalidate the attempt to generalize the theory; it simply meant 'that the generalizations are not about how all prisons are the same, but about a process ... in which variations in conditions create variations in results' (1990: 240).
Deprivations shaped the prison culture in both cases, but the deprivations experienced by women were different than those experienced by men. Men primarily felt deprived of autonomy, while women, who by their own account had never had much autonomy to begin with, primarily felt deprived of protection. While the cultures of the two populations were different, those differences were explained by the general phenomenon of deprivation and the processes through which it influenced prison culture, in the context of the different deprivations felt by men and women. (Becker, 1990: 240-1)
548
USING AND ASSESSING QUALITATIVE DATA ANALYSIS
Becker's focus on processes is consistent with the discussion of causal processes above, under internal generalization, and makes generalization from a single case an acceptable strategy, although by no means a straightforward and unproblematic one. Erickson argued that 'while certain causal processes may be at work in a local setting, the specific causal mechanisms in operation there may manifest differently in another setting, depending on the local social and cultural ecology of each' (2012: 687).
The theorizing of a causal process (see Kelle, Chapter 38, and Maxwell and Chmiel, Chapter 2, this volume), one that also elucidates the contextual influences on this process, can be the basis for some fairly specific ideas about the ways in which this process might apply to other situations and individuals. It can thus reconcile analytic or theoretical generalization and transferability, as two sides of the same coin: that 'transfer' can be, and often is, based on a theoretical understanding of the processes (including contextual influences) involved in a particular situation or outcome, an understanding that can then be applied to other situations. Eisner, in his presentation of an arts-based approach to qualitative inquiry, drew on learning theory to argue that 'I conflate generalization and transfer because transfer always requires more than the mechanical application of a set of skills, images, or ideas from one situation to another. ... Some features of the situations always differ. Hence transfer is a process that has generalizing features' (1998: 198).
STRATEGIES FOR EXTERNAL GENERALIZATION IN QUALITATIVE DATA ANALYSIS
There are a number of ways in which researchers can increase the credibility of the external generalizations they make from qualitative studies. First, qualitative studies often have what the statistician
Judith Singer (personal communication) called face generalizability: there is no obvious reason not to believe that the results apply more generally to similar cases or settings. Hammersley (1992: 189— 91) and Weiss (1994: 26-9) listed a number of features that lend plausibility to generalizations from case studies or non-random samples, including participants' own assessments of generalizability, the similarity of dynamics and constraints to other situations, the presumed depth or universality of the phenomenon studied, and corroboration from other studies. The logic of such generalization is empirical rather than analytic or transfer based, but it can nonetheless be a useful starting point for developing generalizations.
In addition, external generalization in qualitative research usually involves the development of a theory of how the results of a study came about, which goes beyond simple description. As Thomas (2010) argued, it may be better to think of how researchers develop knowledge not as induction (moving from individual incidents to an understanding of general phenomena), but as what Peirce described as abduction (see Reichertz, Chapter 9, and Thornberg and Charmaz, Chapter 11, this volume), the inference to the best explanation or interpretation (Shank, 2008). The answers to such 'how' questions involve an understanding of processes, which is not well served by coding/categorizing analysis strategies (see Maxwell and Chmiel, Chapter 2, this volume). This goal is better addressed by analysis methods that can elucidate the specific connections between events and among events and contexts, and allow the development of theories that can be applied to other settings and individuals that may or may not be superficially similar to those studied.
Thus, external generalization in qualitative research overlaps substantially with the development of theory, a topic that has been extensively discussed since Glaser and Strauss's creation of the concept of
GENERALIZATION IN AND FROM QUALITATIVE ANALYSIS
549
'grounded theory' (1967; Anyon, 2009; Charmaz, 2006; Dressman, 2008; Glaser, 1978; Maxwell and Mittapalli, 2008 - see also Thornberg and Charmaz, Chapter 11, this volume). One particular strategy that is commonly invoked in qualitative research is 'theoretical saturation,'which Sandelowski described as when 'the properties and dimensions of the concepts and conceptual relations selected to render the target event are fully described and ... have captured its complexity and variation' (2008: 875). However, Sandelowski stated that theoretical saturation is often confused with data saturation, and Charmaz and Bryant argued that 'researchers often erroneously believe that they have achieved theoretical saturation when their data become repetitive. ... Most researchers assert saturation rather than provide evidence for it' (2008: 375).
Many of the analysis strategies described for internal generalization are also relevant for external generalization through theory development, including narrative and connecting approaches to analysis and the deliberate testing of theory by searching for discrepant data. In addition, it is important to develop and test alternative explanations for the results. One particular strategy that is rarely used explicitly in qualitative research, but is potentially useful for this, is what Scriven (1974) called the 'modus operandi method': developing alternative theories and then deliberately searching for 'clues' that could indicate which theory (or some combination of these) best explained the data. Maxwell (2011, Chapter 10) provides an ethnographic example of this strategy. Such a strategy would also be helpful in seeing whether the theory applied to other cases or situations than those studied. Similarly, Inglis argued for the value of sociological forensics, in which researchers think of studies of individual situations and cases as places 'to see clues that reveal the connection between macro level processes and structures and the micro level of action, meaning and emotion' (2010).
SUMMARY
In summary, there are analysis strategies that can be valuable in generalizing in and from qualitative data, but these differ somewhat between internal and external generalization. In addition, generalization cannot be guaranteed by mechanically applying a particular strategy as an algorithm or procedure. The effectiveness of any strategy depends on the specific theory and context involved, and, for external generalization, the particular cases or populations that are the targets of transfer or theoretical generalization.
NOTES
1. Statistical inference in experiments is almost always used to assess whether the observed effect was likely to be due to chance variation in assignment, not to make claims about the generalizability of the conclusions. Experimental studies rarely select participants or settings by using probability sampling, because participants are almost always volunteers, and statistical methods for generalizing the results to a larger population are therefore inappropriate (Bloom, 2008: 1 16).
2. Internal generalizability is analogous to what, in experimental research, is called statistical conclusion validity: the validity of inferences about a population (normally, inferences involving covariation of variables) that are based on sample data (Cook and Campbell, 1979: 37). Shadish et al. (2002) describe various threats to statistical conclusion validity, and strategies for dealing with these, some of which are relevant for qualitative research; the most important of these are discussed in this chapter.
3. The issues of sampling or selection in qualitative research, and its implications for generalization, are too complex to address in detail in a handbook on data analysis. Some qualitative researchers prefer the term selection, rather than sampling (e.g., LeCompte and Preissle, 1993: 69; Maxwell, 2012b; Stake, 1995), because, in quantitative research, samples are intended to be representative of a larger population (Morgan, 2008). The sort of sampling/ selection done in qualitative research is usually what is called purposeful (Patton, 2001) or theoretical (Strauss, 1987) sampling (or selection), rather than random sampling or some other method of attaining statistical representativeness. Such selection can have a variety of purposes other than representativeness, including understanding the heterogeneity in the setting studied, or among the participants; selecting participants or settings that are critical for developing or testing the theory
550
USING AND ASSESSING QUALITATIVE DATA ANALYSIS
employed; and selecting participants who are the most knowledgeable about, and/or most willing to discuss, the phenomena studied (Maxwell, 2012b: 97-9). For a more detailed discussion of these issues, see Gobo (2004; 2008) and Gomm et al. (2000).
FURTHER READING
Becker, Howard S. (1990) 'Generalizing from case studies,' in Elliot Eisner and Alan Peshkin (eds.), Qualitative Inquiry in Education: The Continuing Debate. New York: Teachers College Press, pp. 233-42.
Gobo, Giampietro (2008) 'Re-conceptualizing generalization: Old issues in a new frame,' in Pertti Alasuutari et al. (eds.), The SAGE Handbook of Social Research Methods. Thousand Oaks, CA: Sage. pp. 193-213.
Polit, Denise F. and Beck, Cheryl T. (2010) 'Generalization in quantitative and qualitative research: Myths and strategies,' International Journal of Nursing Studies, 47: 1451-58.
REFERENCES
Abbott, Andrew (1992) 'What do cases do? Some notes on activity in sociological analysis,' in Charles C. Ragin and Howard S. Becker (eds.), What Is a Case? Exploring the Foundations of Social Inquiry. Cambridge: Cambridge University Press, pp. 53-82.
Anderson, Gary L. and Scott, Janelle (2012) 'Toward an intersectional understanding of causality and social context,' Qualitative Inquiry, 18 (8): 674-85.
Anyon, Jean (2009) Theory and Educational Research: Toward Critical Social Explanation. New York: Routledge.
Atran, Scott and Medin, Douglas (2008) The Native Mind and the Cultural Construction of Nature. Cambridge, MA: MIT Press.
Ayres, Lioness (2008) 'Meta-narrative,' in Lisa M. Given (ed.), The SAGE Encyclopedia of Qualitative Research Methods. Thousand Oaks, CA: Sage. pp. 508-9.
Becker, Howard S. (1970) Sociological Work: Method and Substance. New Brunswick, NJ: Transaction.
Becker, Howard S. (1990) 'Generalizing from case studies,' in Elliot Eisner and Alan Peshkin (eds.), Qualitative Inquiry in Education: The Continuing Debate. New York: Teachers College Press, pp. 233-42.
Becker, Howard S. (1992) 'Cases, causes, conjunctures, stories, and imagery,' in Charles Ragin and Howard S. Becker (eds.), What Is a Case? Cambridge: Cambridge University Press, pp. 205-16.
Becker, Howard S. (2008) 'How to find out how to do qualitative research.' available at: http://home.earth-link.net/~hsbecker/articles/NSF.html (accessed February 8, 2011).
Becker, Howard S., Geer, Blanche, Hughes, Everett C, and Strauss, Anselm L. (1961) Boys in White: Student Culture in Medical School. Chicago: University of Chicago Press. Reprinted by Transaction Books, 1977.
Bloom, Howard S. (2008) 'The core analytics for randomized experiments in social research,' in Pertti Alasuutari et al. (eds.), The SAGE Handbook of Social Research Methods. Thousand Oaks, CA: Sage. pp. 115-33.
Briggs, C harles (1986) Learning How to Ask. Cambridge:
Cambridge University Press. Brinberg, David and McGrath, Joseph E. (1985)
Validity and the Research Process. Newbury Park,
CA: Sage.
Brown-Saracino, Japonica, Thürk, Jessica, and Fine, Gary Alan (2008) 'Beyond groups: Seven pillars of peopled ethnography in organizations and communities,' Qualitative Research, 8 (5): 547-67.
Bryman, Alan (1988) Quantity and Quality in Social Research. London: Unwin Hyman.
Charmaz, Kathy (2006) Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. Thousand Oaks, CA: Sage.
Charmaz, Kathy and Bryant, Antony (2008) 'Grounded theory,' in Lisa M. Given (ed.), The SAGE Encyclopedia of Qualitative Research Methods. Thousand Oaks, CA: Sage. pp. 374-7.
Cook, Thomas D. and Campbell, Donald T. (1979) Quasi-experimentation: Design and Analysis Issues for Field Settings. Boston, MA: Houghton-Mifflin.
Donmoyer, Robert (1990) 'Generalizability and the single-case study,' in Elliot W. Eisner and Alan Peshkin (eds.), Qualitative Inquiry in Education: The Continuing Debate. New York: Teachers College Press, pp. 175-200.
Donmoyer, Robert (2008) 'Generalizability,' in Lisa Given (ed.), The SAGE Encyclopedia of Qualitative Research Methods. Thousand Oaks, CA: Sage. pp. 371-2.
Donmoyer, Robert (2012) 'Can qualitative researchers answer policymakers' what-works question?', Qualitative Inquiry, 18 (8): 662-73.
GENERALIZATION IN AND FROM QUALITATIVE ANALYSIS
551
Dressman, Mark (2008) Using Social Theory in Educational Research: A Practical Guide. London: Routledge.
Eisner, Elliot (1997) 'The new frontier in qualitative research methodology,' Qualitative Inquiry, 3 (3): 259-73.
Eisner, Elliot (1998) The Enlightened Eye: Qualitative Inquiry and the Enhancement of Educational Practice. Upper Saddle River, NJ: Prentice Hall.
Erickson, Fred (1986) 'Qualitative methods,' in Merlin C. Wittrock (ed.), Handbook of Research on Teaching. New York: Macmillan.
Erickson, Fred (2012) 'Comments on causality in qualitative inquiry,' Qualitative Inquiry, 18 (8): 686-8.
Firestone, William A. (1993) 'Alternative arguments for generalizing from data as applied to qualitative research,' Educational Researcher, 22: 16-23.
Freidson, Eliot (1975) Doctoring Together: A Study of Professional Social Control. Chicago: University of Chicago Press.
Gilbert, Daniel T. and Malone, Patrick S. (1995) 'The correspondence bias,' Psychological Bulletin, 117: 21-38.
Glaser, Barney G. (1978) Theoretical Sensitivity. Mill Valley, CA: Sociology Press.
Glaser, Barney G. and Strauss, Anselm (1967) The Discovery of Grounded Theory. Strategies for Qualitative Research. Chicago: Aldine.
Gobo, Giampietro (2004) 'Sampling, representativeness, and generalizability,' in Clive Seale etal. (eds.), Qualitative Research Practice. London: Sage. pp. 435-56.
Gobo, Giampietro (2008) 'Re-conceptualizing generalization: Old issues in a new frame,' in Pertti Alasuutari et al. (eds.), The SAGE Handbook of Social Research Methods. Thousand Oaks, CA: Sage. pp. 193-213.
Goldthorpe, John H., Lockwood, David, Beckhofer, Frank, and Piatt, Jennifer (1968-1969) The Affluent Worker, 3 vols. Cambridge: Cambridge University Press.
Gomm, Roger, Hammersley, Martyn, and Foster, Peter (eds.) (2000) Case Study Method. London: Sage.
Guba, Egon G. and Lincoln, Yvonna S. (1989) Fourth Generation Evaluation. Thousand Oaks, CA: Sage.
Hammersley, Martyn (1992) What's Wrong with Ethnography? Methodological Explorations. London: Routledge.
Hammersley, Martyn (2008) Questioning Qualitative Inquiry Critical Essays. London: Sage.
Hannerz, Ulf (1992) Cultural Complexity: Studies in the
Social Organization of Meaning. New York: Columbia
University Press. Heider, Eleanor R. (1972) 'Probability, sampling, and
ethnographic method: The case of Dani colour
names,' Man, 7: 448-66. Holstein, Jaber F. and Gubrium, James A. (1999) The
Self We Live By: Narrative Identity in a Postmodern
World. New York: Oxford University Press. Inglis, Tom (2010) 'Sociological forensics: Illuminating
the whole from the particular,' Sociology, 44 (3):
507-22.
Jensen, Devon (2008) 'Transferability,' in Lisa Given (ed.), The SAGE Encyclopedia of Qualitative Research Methods. Thousand Oaks, CA: Sage. p. 886.
Lawson, Tony (2003) Reorienting Economics. London: Routledge.
LeCompte, Margaret and Preissle, Judith (1993) Ethnography and Qualitative Design in Educational Research, 2nd edition. San Diego, CA: Academic Press.
Lincoln, Yvonna S. and Guba, Egon G. (1985) Naturalistic Inquiry. Thousand Oaks, CA: Sage.
Lincoln, Yvonna S. and Guba, Egon G. (1986) 'But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation,' in David D. Williams (ed.), Naturalistic Evaluation. San Francisco: Jossey-Bass. pp. 73-84.
Little, Daniel (2010) New Contributions to the Philosophy of History. New York: Springer.
Mabry, Linda (2008) 'Case study in social research,' in Pertti Alasuutari et al. (eds.), The SAGE Handbook of Social Research Methods. Thousand Oaks, CA: Sage, pp. 214-27.
Maxwell, Joseph A. (1992) 'Understanding and validity in qualitative research,' Harvard Educational Review, 62: 279-300.
Maxwell, Joseph A. (1996) 'Diversity and methodology in a changing world,' Pedagogia, 30: 32-40.
Maxwell, Joseph A. (2004) 'Causal explanation, qualitative research, and scientific inquiry in education,' Educational Researcher, 33 (2): 3-11.
Maxwell, Joseph A. (2010) 'Using numbers in qualitative research,' Qualitative Inquiry, 16 (6): 475-82.
Maxwell, Joseph A. (2011) A Realist Approach for Qualitative Research. Thousand Oaks, CA: Sage.
Maxwell, Joseph A. (2012a) 'The importance of qualitative research for causal explanation in education,' Qualitative Inquiry, 18 (8): 649-55.
Maxwell, Joseph. A. (2012b) Qualitative Research Design: An Interactive Approach, 3rd edition. Thousand Oaks, CA: Sage.
552
USING AND ASSESSING QUALITATIVE DATA ANALYSIS
Maxwell, Joseph A. and Miller, Barbara (2008) 'Categorizing and connecting strategies in qualitative data analysis,' in Patricia Leavy and Sharlene Hesse-Biber (eds.), Handbook of Emergent Methods. New York: Guilford Press, pp. 461-77.
Maxwell, Joseph A. and Mittapalli, Kavita (2008) 'Theory,' in Lisa Given (ed.), The SAGE Encyclopedia of Qualitative Research Methods. Thousand Oaks, CA: Sage. pp. 876-80.
Miles, Matthew B. and Huberman, A. Michael (1984) Qualitative Data Analysis: A Sourcebook of New Methods. Thousand Oaks, CA: Sage.
Miles, Matthew B. and Huberman, A. Michael (1994) Qualitative Data Analysis: An Expanded Sourcebook. Thousand Oaks, CA: Sage.
Misco, Thomas (2007) 'The frustrations of reader gen-eralizability and grounded theory: Alternative considerations for transferability' Journal of Research Practice, 3 (1): 1-11.
Mishler, Eliot G. (1986) Research Interviewing: Context and Narrative. Cambridge, MA: Harvard University Press.
Möhr, Lawrence B. (1996) The Causes of Human Behavior: Implications for Theory and Method in the Social Sciences. Ann Arbor: University of Michigan Press.
Morgan, David L. (2008) 'Sample,' in Lisa Given (ed.), The SAGE Encyclopedia of Qualitative Research Methods. Thousand Oaks, CA: Sage. pp. 797-8.
Nisbett, Robert (2004) The Geography of Thought: How Westerners and Asians Think Differently... and Why. New York: Free Press.
Olsson, Michael R. (2008) 'Postmodernism,' in Lisa Given (ed.), The SAGE Encyclopedia of Qualitative Research Methods. Thousand Oaks, CA: Sage. pp. 655-9.
Patton, Michael Q. (2001) Qualitative Research and Evaluation Methods, 3rd edition. Thousand Oaks, CA: Sage.
Pawson, Ray (2006) Evidence-based Policy: A Realist
Perspective. London: Sage. Polit, Denise Rand Beck, CherylT.(2010) 'Generalization
in quantitative and qualitative research: Myths and
strategies,' International Journal of Nursing Studies,
47: 1451-8.
Putnam, Hilary (1999) The Threefold Cord: Mind, Body and World. New York: Columbia University Press.
Rosenau, Pauline M. (1992) Post-Modernism and the Social Sciences. Princeton, NJ: Princeton University Press.
Ruddin, Lee P. (2006) 'You can generalize stupid! Social scientists, Bent Flyvbjerg, and case study methodology,' Qualitative Inquiry, 12 (4): 797-812.
Salmon, Wesley C. (1998) Causality and Explanation.
New York: Oxford University Press. Sandelowski, Margarete (2008) 'Theoretical saturation,'
in Lisa Given (ed.), The SAGE Encyclopedia of
Qualitative Research Methods. Thousand Oaks, CA:
Sage. pp. 875-6. Sankoff, Gillian (1971) 'Quantitative aspects of sharing
and variability in a cognitive model,' Ethnology, 10:
389-408.
Sayer, Andrew (2000) Realism and Social Science. London: Sage.
Schofield, Janet W. (1990) 'Increasing the generalizabil-ity of qualitative research,' in Elliot W. Eisner and Alan Peshkin (eds.), Qualitative Inquiry in Education: The Continuing Debate. New York: Teachers College Press, pp. 201-32.
Schwandt, Thomas A. (1997) The SAGE Dictionary of Qualitative Inquiry. Thousand Oaks, CA: Sage.
Scriven, Michael (1974) 'Maximizing the power of causal investigations: The modus operandi method,' in W. James Popham (ed.), Evaluation in Education -Current Perspectives. Richmond, CA: McCutchan. Reprinted (1976) in G.V. Glass (ed.), Evaluation Studies Review Annual 1. Beverly Hills, CA: Sage, pp. 108-18.
Scriven, Michael (2008) 'A summative evaluation of RCT methodology: And an alternative approach to causal research,' Journal of MultiDisciplinary Evaluation, 5(9): 11-24.
Seale, Clive (1999) The Quality of Qualitative Research. London: Sage.
Shadish, William R., Cook, Thomas D., and Campbell, Donald T. (2002) Experimental and Quasi-experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin.
Shank, Gary (2008) 'Abduction,' in Lisa Given (ed.), The SAGE Encyclopedia of Qualitative Research Methods. Thousand Oaks, CA: Sage. pp. 1-2.
Shweder, Richard A. (ed.) (1980) Fallible Judgment in Behavioral Research. San Francisco: Jossey-Bass.
Stake, Robert (1995) The Art of Case Study Research. Thousand Oaks, CA: Sage.
Strauss, Anselm (1987) Qualitative Analysis for Social Scientists. Cambridge: Cambridge University Press.
Strauss, Claudia (1992) 'Models and motives,' in Roy G. D'Andrade and Claudia Strauss (eds.), Human Motives and Cultural Models. Cambridge, England: Cambridge University Press, pp. 1-20.
GENERALIZATION IN AND FROM QUALITATIVE ANALYSIS
553
Thomas, Gary (2010) 'Doing case study: Abduction not
induction, phronesis not theory,' Qualitative Inquiry,
16 (7): 575-82. Thome, Sally and Darbyshire, Philip (2005) land mines
in the field: A modest proposal for improving the
craft of qualitative health research,' Qualitative
Health Research, 15: 1105-13. Wallace, Anthony F. C. (1970) Culture and Personality,
2nd edition. New York: Random House. Weiss, Robert S. (1994) Learning From Strangers: The
Art and Method of Qualitative Interviewing. New
York: Free Press. Wievorka, Michael (1992) 'Case studies: History or
sociology?,' in Charles C. Ragin and Howard S.
Becker (eds.), What Is a Case? Cambridge: Cambridge
University Press, pp. 159-72.
Wolf, Margery (1992) A Thrice-Told Tale: Feminism, Postmodernism, and Ethnographic Responsibility Stanford, CA: Stanford University Press.
Yin, Robert K. (1984) Case Study Research: Design and Methods. Thousand Oaks, CA: Sage.
Yin, Robert K. (2003) Case Study Research: Design and Methods, 3rd edition. Thousand Oaks, CA: Sage.
Zentella, Ana Celia (1990) 'Integrating quantitative and qualitative methods in the study of bilingual code switching,' in Edward H. Bendix (ed.), The Uses of Linguistics. Annals of the New York Academy of Sciences, vol. 583. pp. 75-92.
Znaniecki, Florian (1934) 'Analytic induction,' in Florian Znaniecki, The Method of Sociology. New York: Farrar and Rinehart.