9 Methods for Answering Research Questions: Data Gathering, Generation, Reduction and Analysis 9.1 Chapter Summary • The timing of data collection is a fundamental choice in designing social research leading to cross-sectional, longitudinal or historical studies. • The labels 'quantitative' and 'qualitative' should be applied only to types of data, methods of data collection and analysis and, perhaps, styles of research; they are inappropriate for classifying research paradigms and social research in general. • The distinction between quantitative and qualitative is closely associated with the degree of involvement researchers have with research participants and sites; usually limited in the former and extensive in the latter. • The most commonly used quantitative methods of data collection are self-administered questionnaires and structured interviews. • These two methods need to be distinguished as both the format of the instruments and the methods of administration are different. • The most commonly used qualitative methods include focused and in-depth interviews, and various types of observation (semi-structured, unstructured and participant). • Data produced by most methods require some manipulation or reduction to make them suitable for analysis. • Depending on the type of data, methods of data reduction include: coding of various kinds; index and scale construction; and other ways of producing categories, factors, clusters and types. • When quantitative methods of data collection are used it is necessary to be fully aware of the level(s) of measurement and their consequences for choice of data analysis and type of conclusions that can be drawn. • Quantitative methods of data analysis are of four main types: univariate descriptive, bivariate descriptive, explanatory and inferential. " Qualitative methods of data analysis include various kinds of description as Well as techniques for theory generation. 196 Designing Social Research • Almost all data used in social research begins in the qualitative form and has to be transposed to become quantitative. • Arguments about the relative merits of quantitative and qualitative research are pointless; both types have their uses and limitations. • The issue of how findings from qualitative research can be generalized to other sites and populations has received considerable attention in recent years. • While it is possible to generalize quantitative findings from a probability sample to the population from which the sample was drawn, to generalize beyond that population, across time and space, requires the use of similar procedures to those used to generalize qualitative findings, that is, procedures based on evidence of similarities and differences. • After years of neglect and controversy, the use of mixed methods in social research is gaining popularity. • Mixed methods research has been classified in various ways, such as: triangula-tion (concurrent use of both quantitative and qualitative methods); embedded (one type of method is supplementary to the other); explanatory (sequential use with quantitative preceding qualitative); and exploratory (sequential use in the reverse order). • Care needs to be taken when methods used with incompatible ontological assumptions are to be combined concurrently rather than sequentially. • Mixed methods research should be seen as a normal and, perhaps, necessary part of knowledge generation, rather than as a special type. 9.2 Introduction With research questions in place, one or more logics of inquiry selected, ontological assumptions made explicit, data sources and method of selection established, and, if appropriate, hypotheses specified, the next step in the development of a research design is to decide how to obtain and analyse data to answer the research questions. The kind of data that are considered to be appropriate will depend on a variety of factors. On the one hand, there are methodological considerations that are associated with logics of inquiry and, on the other hand, there is a range of pragmatic factors that will need to be taken into account. These include the context of the research, the nature of the research topic, the kind of research questions being investigated and their associated research purposes, the expertise and personality of the researcher(s), time and budget considerations, the availability of equipment (including computer hardware and software), and the expectations of funding bodies, clients, colleagues and/or the consumers of the research findings. This chapter covers six main topics. • The role of timing of data collection in determining the character of a research design. • Major qualitative and quantitative methods of data collection, reduction and analysis. • Differences between qualitative and quantitative methods. T Methods for Answering Research Questions • Problems in generalizing from qualitative research. • The use of mixed methods. • The relationships between logics of inquiry and research methods. From simple beginnings about seventy years ago, there is now a vast and complex literature on data collection and/or generation and analysis. In some textbooks on research methods there is a tendency to simplify this complexity into a dichotomy of 'quantitative' and 'qualitative' research designs (e.g. Punch 2014; Neuman 2006, 2014). However, we believe it is necessary to confine the use of these two concepts to methods of data collection and/or generation and analysis. This is the approach taken in this chapter. 9.3 Timing Data Collection All social research adopts a position with regard to the timing of data collection (see Figure 9.1). In fact, some writers regard time as the critical defining characteristic of all research designs. This appears to stem from classic experimental designs in which variables in experimental and control groups are measured at different points in time, the experimental group having been subjected to some kind of treatment in between. ■•gui-e 9 Historical m 7 Quantitative Mixed Method, Mixed Methods ^ Mixed Methods SweVed"** ™ h°» research questions are to 197 198 Designing Socio! Research There are three basic choices for the social researcher with regard to time. A study may: • be confined to the present rime - cross-sectional; • extend over a period of time - longitudinal; • be confined to the past - historical. Cross-sectionol studies Cross-sectional studies capture a picture of aspects of social life, including: population (demographic) characteristics; individual attitudes, values, beliefs and behaviour; social interaction; and aspects of social groups, organizations, institutions and structures. However, this type of study is not well suited to research on social processes and social change, as these require the collection of data over time. It also places severe restrictions on achieving the purpose of explanation. Longitudinal studies Time Dimension Cross-sectional In the present Longitudinal Before and after Impact of an intervention Change over time Prospective: looking forward Time series Panel study Cohort study Retrospective: looking back Historical In the past Some longitudinal studies involve only two points in time and are referred to as before-and-after designs. A common example is the study of social change, which may involve nothing more than two cross-sectional studies at different points in time. Other longitudinal studies involve a number of points in time, or a series of cross-sectional studies on the same or a similar population or group. There are three variations of longitudinal research all of which can be regarded as prospective studies; they begin in the present and plan further stages in the future. In time series research the situation remains the same but some or all of the participants may change; for example, a study of people who happen to live in a particular street. A panel study involves contact with the same people, group or organization over a period of time. 1 A variation of the panel study is cohort analysis, in which the same categories of people, rather than the same individuals, are studied over time. A cohort is defined in terms of specific criteria that identify people who have had similar life experiences due to having had a critical life event in common. Examples of cohorts include people who were born in the same year (birth cohorts), who Methods for Answering Research Questions 199 are in the same year in educational institutions, or who joined or left an organization at around the same time. Each of these three variations has its advantages and disadvantages. Panel studies may be the ideal but are expensive. There is also the difficulty of keeping track of the members and dealing with the fact that people will drop out for various reasons. Cohort designs are easier to conduct, but their results are less powerful. Time series designs are different from the other two. Panel studies track the changes in a sample or group of people over time, while a time series study may be more concerned with the changing effects over time on people who have some circumstances in common. Therefore, cohort and time series studies are compromises on panel studies. Retrospective studies take the present as a base and seek information about recent history. A common form of this research, oral and life histories, gets people who lived through a particular period or particular events to recall their experiences, or uses the traces that such people have left behind, such as diaries. There are clearly some limitations to this kind of research as memory is fallible and its use always involves a possible reconstruction of the past under the influence of subsequent experiences. Historical studies By definition, historical studies deal with social events or phenomena in the past. Such studies are very different in character to those that can collect data about and from people in the present. While retrospective studies may be seen as historical, data normally come from written records of some kind, or from other traces of past social activity. However, depending on the nature of the research topic and research questions, research in the present may need to be linked to the past. Combinations Research can be made up of a combination of these approaches to the time dimension. For example, a cross-sectional study conducted today can become a longitudinal study if it is replicated, say, in five years. A retrospective or a prospective study can be used in combination with a cross-sectional study to produce a longitudinal study and a historical study can be either cross-sectional or longitudinal. It follows that research design choices related to the time dimension will be determined largely by the purposes of the study and the type of research questions to be answered. Some purposes and research questions can be pursued successfully using a cross-sectional study, whereas others, ideally, require longitudinal research. While exploratory and descriptive studies are likely to be concerned with the present, longitudinal description is certainly 4 legitimate research activity. Similarly, while the purposes of explanation and understanding may be more easily achieved in longitudinal research, the Potential costs involved may lead to the common compromise of trying to do 200 Designing Social Research this using cross-sectional research. The applied purposes of change and evaluation, and their associated research questions, might also be better achieved in a longitudinal study. As longitudinal studies are usually more costly than cross-sectional studies, many if not most research designs have to make compromises between achieving the ideal way to answer the research questions and the costs involved in doing so. A dilemma A typical dilemma for researchers is how to answer research questions that deal with the passage of time using only a cross-sectional study. For example, we may be interested in the effect on students of exposure to a particular three-year educational programme. One method would be to follow a cohort of students over the three years, making measurements of their position on relevant variables at the beginning and end of the programme and, possibly, at intervals in between. However, if the programme had been in operation for at least three years, it would be to measure these variables at one time for each year cohort of students, and see if they differed. This is a compromise as it lacks random assignment to each year cohort. It also lacks controls for the fact that each cohort enters the programme, and passes through the three years, during a different time period, thus making it possible that its members have been subjected to the influence of different social conditions. Nevertheless, versions of this compromise are commonly used, and the passage of time is dealt with statistically. 9.4 Quantitative and Qualitative Methods It has become common practice to divide research methods into two broad types, quantitative and qualitative. However, the concepts have been used to contrast five different aspects of the social research enterprise: • methods - techniques of data collection and analysis; • data - produced by particular types of methods; • research - using particular types of methods; • researchers - who use particular methods; and • research paradigms - using different ontologkal and epistemological assumptions. The first four aspects are closely related; a method produces a particular form of data, and research and researchers are related to the use of particular methods. The association of the concepts of qualitative and quantitative with research paradigms is distinctive and different. Unfortunately, it elevates a distinction between types of data to the level of a research paradigm; this is both unhelpful and inappropriate (see also Guba and Lincoln 2005). Quantitative methods are generally concerned with counting and measuring aspects of social life, while qualitative methods are more concerned with producing discursive descriptions and exploring social actors' meanings an Methods for Answering Research Q 'uestions 201 interpretations. While it is convenient to classify research methods as qualitative or quantitative, and we do this in the first part of this chapter, there is a growing body of literature that has questioned the legitimacy of this dichotomy. These arguments are reviewed later in the chapter. Data-Collection Methods As there are countless books that describe the nature and use of social research methods, no attempt will be made to provide details here. The list in the box below is only intended to be indicative of the commonly used quantitative and qualitative methods. Many refinements have been made to this classification. Quantitative data-collection methods The most commonly used quantitative data-gathering methods in the social sciences are undoubtedly the self-administered questionnaire and the structured interview, both of which keep the researcher at a distance from actual social situations and processes. There is a great deal of confusion, particularly in the popular literature, about the way these two methods are identified. The commonest practice is not to distinguish them and, therefore, to assume that they are identical. For example, 'survey' is used to refer to questionnaires, and questionnaires are seen to be used in structured interviews. De Vaus (2002) used 'questionnaire' as the generic term and then distinguished between face-to-face, telephone, postal, self-administered and Internet as different methods of administration. On the other hand, Oppenheim (1992) made the distinction between 'standardized interview' and 'questionnaire' very clear. As the processes by which data are ji—, j ■•<•<• ■ • collected differ in important ways, it is necessary to distinguish between these two methods. The formats of the instruments are different and they have their own particular advantages and disadvantages. Questionnaires have to be prepared in such a way that respondents can complete them without any assistance other than built-in and/or separate written instructions. An interview schedule, on the other hand, will usually contain instructions to the interviewer, and the interviewer will provide other instructions to the respondent. Samples of an interview schedule and a questionnaire format can be found in Smith (1981: appendices C and D). Hi ere are a Data Collection and Generation Methods Quantitative Questionnaire (self-administered) Structured interview Observation: structured Content analysis of documents Qualitative Participant observation Observation: semi-structured and unstructured Focused interview In-depth interview Oral/Life histories Focus groups/Group interviews Content analysis of documents 202 Designing Social Research number of discussions in the literature on the strengths and weaknesses of both methods (see, for example, Groves and Kahn 1979; Oppenheim 1992; deVaus 2002). The use of structured observation is much less common and is confined largely to experiments and observational studies in artificial settings. Examples of the former would be observing the response of individuals to periods of social isolation, or their reaction to an authority figure acting in an aggressive manner. ■—i of children at play, or appli- cases, fthe Sp.:; o e^™!d b^bserv^ group of child™ at play or a, cantTfo a leadership position undertaking a problem-solving task In both c video recoVdTng may be used, but all this means is that the numerical coding o behavou"can be delayed and the 'original' data rev.s.ted. Structured observa on maybe combmed with other methods of data collection, such as the structured may interview. Qualitative data *« "ample, Ecke 205 h™ever, techniques of Quantitative data analysis methods Quantitative methods of analysis fall into four main categories: univariate descriptive, bivariate descriptive, explanatory and inferential (see Blaikie 2003). Univariate descriptive methods focus on single variables and are used to report the distributions of a sample or population (using all four levels of measurement), and to produce summary measures of the characteristics of such distributions. These summary measures include: frequency counts (which can also be represented graphically); measures of central tendency (such as mode, median and mean, depending on the level of measurement); and measures of the dispersion of the distribution (such as the inter-quartile range and standard deviation). Bivariate descriptive methods are used to establish the degree to which two variables co-vary, that is, whether positions on one variable are likely to be consistently associated with positions on another variable. An example is the extent to which people with high levels of education also have high-status occupations. I he way in which association is established depends on the level or levels of measurement involved and whether there are two or more variables being analysed. Data Analysis Methods Quantitative Univariate Descriptive Distribution: numerical and graphical Central tendency and dispersion Bivariate Descriptive Correlation: simple, partial and multiple Analysis of variance and covariance Regression: simple, partial and multiple Explanatory Factor analysis Path analysis Regression: simple, partial and multiple Inferential Sample statistic to population parameter Sample differences to population differences Qualitative Descriptive Theory generation Analytic induction Grounded theory: open and axial coding Categorizing and connecting From everyday typifications to typologies 206 Designing Social Research In order to answer 'why' questions with cross-sectional quantitative data, it is necessary to establish the influence of one or more independent variables on a dependent variable. There are three popular methods for doing this: factor analysis, path analysis and regression. The aim in this kind of analysis is to deal with a set or network of relationships in the hope that causation can be demonstrated. Inferential analysis is used for two purposes: to make estimates of population characteristics (parameters) from sample characteristics (statistics); and to establish whether differences or relationships within a sample (such as an association between education and occupation) can be expected to exist, other than by chance, in the population from which the sample was drawn. It is for these purposes that tests of significance are used (see the discussion in chapter 8). Qualitative data analysis methods As a result of the growing popularity of qualitative research over recent decades, the literature on methods of qualitative data analysis has expanded considerably (see, for example, Glaser and Strauss 1967; Lofland 1971; Turner 1981, 1994; Martin and Turner 1986; Richards and Richards 1987, 1991, 1994; Dey 1993; Miles and Huberman 1994; Bryman and Burgess 1994; Kelle 1995, 2004; Coffey and Atkinson 1996; Ryan and Bernard 2000; Silverman 2001; Morse and Richards 2002; Northcutt and McCoy 2004; Corbin and Strauss 2015). While there is no one dominant method, various versions of grounded theory have become popular, particularly as software, such as Ethnograph, NVivo, ATLAS, ti and MAXQDA, are now available to aid the process (see, for example, Seidel and Clark 1984; Richards and Richards 1987, 1991, 1994; Pfaffenberger 1988; Tesch 1990; Fielding and Lee 1991, 1998; Miles and Huberman 1994; Coffey and Atkinson 1996; Weitzman 2000; Fielding 2001; Seale 2002, 2005; Kelle 2004; Lewins and Silver 2007). While there is no industry leader, NVivo is the best-known package (Bryman 2016: 602). We noted in chapter 8 that analytic induction has been used in association with case studies as a logic for theory development. While data from such case studies can be either qualitative or quantitative, they are commonly the former. The advocates of grounded theory have argued for the use of the constant comparative method as being superior to analytic induction (Glaser and Strauss 1967: 101-13). Analytic induction refines an initial hypothesis, while in grounded theory and its variants an ongoing process of data collection and data analysis is used to construct new theories from data. For an overview of how grounded theory is conducted see Turner (1981) and the summary by Bryman (1988: 83-4). Dey (1993) has formulated the method of coding in grounded theory' as a circular or spiral process involving three activities: describing, classifying and connecting. The first step is to produce 'thick' or 'thorough' descriptions or| the phenomenon being studied (Geertz 1973; Denzin 1978). 'Thin' description merely states 'facts', while 'thick' description includes the context of the action, the intentions of the social actors, and the processes through which social action and interaction are sustained and/or changed. The next part of the process, clas- Methods for Answering Research Questions sifying, refers to the open and axial coding of grounded theory. In the same way, Dey has argued that classifying data is an integral part of the analysis and without this there is no way of knowing what is being analysed. Classification is achieved by creating categories, assigning categories to the data, and splitting and splicing categories. Classification is a conceptual process. When wc classify, we do two things. We don't just break the data into bits, we also assign these bits to categories or classes which bring these bits together again, if in a novel way. Thus all the bits that 'belong' to a particular category are brought together, and in the process, we begin to discriminate more clearly between the criteria for allocating data to one category or another. Then some categories may be subdivided, and others subsumed under more abstract categories. The boundaries between these categories may be defined more precisely. Logic may require the addition of new categories, not present in the data, to produce a comprehensive classification. Thus the process of classifying the data is already creating a conceptual framework through which the bits of data can be brought together again in an analytically useful way. (Dey 1993: 44-5) Classification is not a neutral process; the researcher will have a purpose in mind that will provide direction and boundaries. The third part of the process is making connections between categories. The aim is to discover regularities, variations and singularities in the data and thus to begin to construct theories. Following Sayer (1992), Dey has distinguished between formal and substantive relations. Formal relations are concerned with similarities and differences while substantive relations are concerned with how things interact (1993: 152). Methods of qualitative analysis differ in the extent to which they attempt to 'tetain the integrity of the phenomenon'. That is, the extent to which researchers remain close to the language, the concepts and meanings of the social actors rather than imposing their own concepts and categories on lay accounts. There is a choice between a high stance, in which the researcher imposes concepts and meanings, and a low stance, in which the researcher derives concepts and meanings from lay language. In its purest form, the use of Abductive logic involves a low stance because it develops technical concepts and theoretical propositions from accounts provided in lay language (see chapter 6). Technical concepts generated in this way are designed to be more abstract and generalizable than is possible with lay concepts. Mainstream grounded theory, on the other hand, is much more a process of a researcher 'inventing' and imposing concepts on the data; it adopts a high stance. The various forms of coding are a search for technical concepts that will organic and make sense of the data. While these concepts can be either those that are already in use, or those developed by a researcher for a particular purpose, there appears to be little attempt to derive them from lay concepts, to make use of lay meanings associated with the concepts, or to tie them to lay concepts. For this reason, grounded theory uses Inductive rather than Abductive logic. Another major variation in qualitative analysis is whether the researcher is sated NV''h description, or whether the goal is the development of theory. There 207 208 Designing Social Research is, however, no clear divide between these two activities. Some would argue that description is all that a researcher can legitimately do; others would argue that description, particularly 'thick' description, already provides understanding and possibly explanation, and that nothing more is needed; while others seek to develop 'bottom-up' theories consisting of testable propositions (Hammersley 1985). For grounded theorists, the testing of these theoretical propositions is tied intimately to the process of their generation. Others may be willing to subject their theories to independent post-testing, although this may consist of examining the relevance of the theory in other contexts, still using 'bottom-up' techniques rather than the formal testing that is advocated by the use of Deductive logic. Researchers interested in combining logics of inquiry, and qualitative and quantitative methods, may develop their theory by some kind of grounded method and then test it deductively using quantitative methods. Clearly there is a range of possibilities here, and if qualitative methods are used, a decision has to be made as to what will constitute understanding or explanation, and how these will be developed and tested. An alternative to grounded theory can be found in Blaikie's elaboration of the use of Abductive logic, as advocated by Schutz and Giddens. He began working on this in the 1970s (Blaikie 1974) and elaborated the ideas about ten years later (Blaikie and Stacy 1982, 1984). Since then, a number of postgraduate students have successfully used versions of it in a variety of studies (Kelsen 1981; Stacy 1983; Drysdale 1985; Balnaves 1990; Smith 1995; Priest 1997; Ong 2005), and other students and colleagues have worked with it {e.g. Drysdale 1996; Priest 2000). Elaborations and illustrations of how Abductive logic can be used to generate theory from data can be found in Blaikie and Priest (2017). There the emphasis is on the generation of types and typologies rather than concepts and categories. See also Blaikie (Forthcoming) on the origins of Abductive logic. 9.5 Differences Between Quantitative and Qualitative Methods For decades there has been considerable debate about the relative merits of quantitative and qualitative methods. Quantitative researchers have adopted the methodological high ground and qualitative researchers have been seen by them as a troublesome but largely irrelevant sect. Qualitative researchers have tended to struggle under the shadow of this dominant orthodoxy and, as a result, have had to be content with much lower levels of funding for their research.2 These debates have been conducted in terms of various contrasts in purpose and style, and have produced a slate of pejorative terms used to defend preferred methods and to abuse other methods. Fortunately, in recent years these prejudices have receded to disciplines such as economics and mainstream psychology, and no longer dominate sociology. Social researchers have become more eclectic in their choice of methods.3 As noted earlier (chapter 8, 'Forms of Data'), data used by social researchers begin in a qualitative form. It is only after work has been done to transpose words into numbers that quantitative data come into being. 'We can regard all Methods for Answering Research Questions of the information which we acquire about the world as qualitative, and then see that under some circumstances we can use this information to create a particular kind of data, quantitative data, to which the properties of number can be applied' (Turner 1994: 195). Halfpenny (1996) argued that in spite of the surface differences between words and numbers, quantitative and qualitative data are not fundamentally different. Quantitative data is usually produced by coding some other data, which is reduced to a number by stripping off the context and removing content from it. Later, after manipulating the numbers, they are interpreted, that is, expanded by adding content and context which enable one to see through the numerical tokens back to the social world. (Halfpenny 1996: 5) It is the order and power offered by numerical analysis that have made quantified information so attractive and led qualitative information to be treated with suspicion. However, for quantitative data to achieve these apparent advantages, it is necessary to assume that the properties of a number system correspond to some features of the original data. A moment's reflection will reveal to us that, even where the use of numerical analysis seems most self-evident, to use it we need to make certain working assumptions. We may regularly count apples or sheep or pounds sterling. But every apple, every animal, is unique. When we count, we merely agree, tacitly, that for this everyday purpose, we are willing to apply rules which disregard the differences between individual apples or individual sheep, and which stress their similarities for numbering purposes. (Turner 1994: 195) When quantitative methods are used, a researcher is likely to have very limited or possibly no contact with the people being studied. The use of some quantitative methods, such as mailed questionnaires, structured observation and unobtrusive methods that involve the use of secondary data, require no face-to-face or verbal contact at all. When there is contact, such as in structured interviewing and experiments, it is formal and of limited duration. However, even in these cases, the researcher may have no contact if assistants are employed to carry out these tasks. This maintenance of distance from the people being studied, and the fanatical resistance to any form of personal disclosure or emotional involvement by the researcher, is largely practised in the belief that it will ensure that objectivity is achieved. What constitutes objectivity, and why it must be achieved, is, of course, related to the epistemological assumptions (probably from either Positivism or Critical Rationalism) that the researcher has explicitly or implicitly adopted. On the other hand, the use of qualitative methods usually requires an extended and/or intensive period of involvement in some social world. The most extreme form is participant observation in which a researcher can become fully immersed 111 the social actors' world with all the levels of personal involvement that this entails. Such qualitative methods allow a researcher to become an 'insider' and to discover the social actors' culture and worldviews. 209 210 Designing Social Research The contact and involvement in in-depth, unstructured interviewing lies somewhere between participant observation and structured interviewing, and will involve varying degrees of personal involvement and disclosure on the part of the researcher. When a series of in-depth interviews is conducted with rhe same person, the level of involvement is likely to be higher. Rather than attempt to adopt the position of a detached 'scientific' observer, qualitative researchers may deliberately choose to 'go native'; to allow themselves to become part of the world of the researched and be seduced by the social actors' constructed reality. Some would argue that, without a period of immersion in a social world, no adequate understanding of it can be achieved. While it is probably not possible for a researcher to go completely native, a test of whether the social actors' meanings have been 'discovered' is dependent on the researcher being able to interact with rhe research participants successfully. To achieve this, it is necessary to become as subjective as possible rather than to try ro adopt some kind of objective stance, at least at the data-generation stage. An important feature of the use of quantitative methods is their highly structured nature. They are usually located within a research design that includes a set of predetermined stages, and the data gathering will be accomplished by the use of predetermined procedures and pre-tested instruments. In using such methods, a researcher aims for maximum control over the data gathering and the achievement of uniformity in the application of the techniques. Usually, it is only after data are collected that analysis will begin. The main justification for uniformity, control and rigid stages is to achieve some notion of objectivity and replicability. Perhaps quantitative methods are selected by researchers who prefer order, predictability and security, and who have a low level of tolerance for uncertainty and ambiguity. This contrasts with the use of qualitative methods in which the procedures are much more open and flexible. Frequently, when qualitative methods are used, researchers have a very limited idea of where they should start, how they should proceed, and where they expect to end. They have to accept opportunities when they open up and they will want to follow leads as they occur. They see research as a learning process and themselves as the measuring (data-absorbing) instrument. They will want to allow concepts, ideas and theories to evolve and they will resist imposing both preconceived ideas on everyday reality and closure on the emerging understanding. Qualitative data gathering is messy and unpredictable and seems to require researchers who can tolerate ambiguity, complexity, uncertainty and some lack of control. While studies are frequently classified as being either quantitative or qualitative, some use both kinds of data. Quantitative studies may collect some data in words (e.g. open-ended questions in a survey, or text on which content analysis is to be undertaken). By means of some coding process, these data are transformed into a numerical form. Similarly, some qualitative studies may produce simple tables of frequencies and percentages to summarize some features of non-numerical data. Such counting in qualitative research can provide support for the representativeness of certain features within a social group or category. Researchers who are wedded to quantitative methods may also use qualitative methods. However, qualitative methods and data are commonly viewed as being supplementary to quantitative methods and data. Qualitative methods may he Methods for Answering Research Questions 21 I used by quantitative researchers in an exploratory stage to suggest hypotheses or to facilitate the development of research instruments. The range of qualitative methods used is likely to be limited to those that come closest to quantitative methods, for example semi-structured interviews or observation. Some quantitative researchers seem to regard open-ended questions in a mainly structured questionnaire as constituting qualitative data gathering. The fact that the questions are in the researcher's language, and that the method used to code the responses is devised by the researcher, makes it a very different kind of data collection to the more traditional use of qualitative methods. Quantitative and qualitative researchers have different methods for achieving rigour in their research. Traditionally, the former have focused on establishing the 'validity' and 'reliability' of their measurements; that their instruments measure what they claim to measure and that they do so consistently. An examination of the methods by which validity and reliability are established reveals that they involve corroboration and replication. However, there are no ultimate standards against which a measuring instrument can be compared; there are only well-used instruments about which communities of researchers have a high degree of confidence. Objectivity is always relative. The character of qualitative data makes corroboration and replication more difficult, some would say impossible. The qualitative researcher is usually the measuring instrument and no two instruments are the same. If you were to conduct in-depth interviews with twenty people about their working life, you may discover three different types of work orientations. Clearly, it would be difficult for others to then interview these same people. It might be possible to interview samples of people from the same population but even if two researchers have agreed ahead of time to discuss the same topics, the conversations of one of them with their sample may be different from what the other has with their sample, and similarly for other interviewers. They may end up with a variety of typologies and, if they happened to agree on the types, there may be differences in their descriptions. However, this is not to suggest that one of them has produced the correct account and the other is wrong. Hopefully, they accurately reported what ach learnt from their sample of people. Qualitative researchers are rather divided on the need to establish the authenticity of their findings. Some would argue that researchers produce their unique accounts and that corroboration or replication is impossible. They would claim that if they have acted professionally, and have explained how they went about their research, their accounts should be trusted. Other qualitative researchers have argued that the social actors concerned must corroborate any account that a researcher gives of social life. In other words, the researcher's account must correspond closely to social actors' accounts. (See Blaikie 2007 for a discussion of this issue.) 9.6 Generalizing with Qualitative Methods and Data Many of the issues raised in the previous chapter about generalizing and theorizing from case studies are echoed in discussions about the possibility of 212 Designing Social Research generalizing from studies that use qualitative methods and data. As indicated earlier, case studies can use any kind of method for data collection and analysis; they are not restricted to qualitative methods. However, the use of qualitative methods in any kind of research poses a similar set of problems; combine case studies and qualitative methods and the problems are exacerbated. For a start, the kind of generalizing that is available when quantitative methods are used is not available with qualitative methods. Methods of statistical inference can only be used with probability samples and numerical data. In addition, many researchers with qualitative preferences give generalization low priority, and some have explicitly rejected any form of context-free generalization as a goal (e.g. Guba and Lincoln 1981, 1982; Denzin 1983). Also, the idea of replicating previous studies, a goal of much experimental and survey research, is usually regarded as being inappropriate in qualitative research. The reflexive character of qualitative research means that individual researchers inevitably inject something of themselves into the research process and, hence, into the outcomes. In addition, social situations are never sufficiently similar, across space and time, to make replication possible. Studies conducted by different researchers, in different locations, and at different times, will be unique because of the particular characteristics of the researcher and the researched, their effects on each other, and the hermeneutic processes involved in the production of the researcher's account (see Blaikie 2007). The goal is not to produce a standardized set of results that any other careful researcher in the same situation or studying the same issues would have produced. Rather it is to produce a coherent and illuminating description of and perspective on a situation that is based on and consistent with detailed study of that situation. (Schofield 1993: 202) In contrast, quantitative researchers generally believe that their methods, and the application of them, can control for any possible researcher influence; they also ignore or try to eliminate the hermeneutic processes; and they tend to assume that time and space do not pose insurmountable limitations. In recent years, there has been a marked increase in interest in the issue of gen-eralizability among qualitative researchers, particularly in educational research where qualitative methods are now used extensively. Agencies that fund this type of research are usually interested in its relevance beyond the site in which it was conducted, and ethnographic researchers in education are also likely to want their research to have wider relevance. How can generalization be achieved when qualitative methods are used? Answers to this question are mainly confined to generalizing from one research site to some other site or population. As we have seen in the previous discussion of generalizing from case studies, a number of answers were offered that are relevant here (see, for example. Stake 1978; Bassey 1981; Guba and Lincoln 1981, 1982; Goetz and Le Compte 1984; Gomm et al. 2000). Some of these suggestions require detailed descriptions of both the site in which a study is conducted and the sites about which generalizations are to be made. Similarities and differences can then be taken into account in any judgement about the relevance of findings Methods for Answering Research Questions obtained from one site for some other sites (Schofield 1993: 207). The aim in such comparisons is to establish whether the research site is typical of other sites. Attempts are made to generalize across time as well as across research sites. Hence, a researcher may wish to make a judgement about whether the findings from a particular site will also hold in the future at the same site as well as at the same or other rimes at other sites (Hammersley 1992: 87). Now let us turn this discussion on its head and tie it back to sampling issues discussed in the previous chapter. It is a mistaken belief that problems of generalizing are confined to case study research and the use of qualitative methods. Rather, statistical generalization is only possible when probability sampling is used, and then it only applies to the population from which the sample was drawn. While some populations may be large, for example in national studies, they can also be limited in size, for example a school classroom. If a probability sample is used, results can be statistically generalized to the population. However, the problem still remains as to whether the findings about this population can be generalized to other populations or to other social contexts, such as other schools. Therefore, if a researcher wishes to generalize beyond a population, whether or not sampling is used, issues concerning the selection of the population are similar to those in the selection of case studies and of research sites in ethnographic research. There are only limited advantages in using samples and quantitative methods when it comes to wider generalization, as the scope of statistical generalization is limited to the population selected. Beyond that population, the problems and their solutions are the same, regardless of whether quantitative or qualitative methods are used. 9.7 Mixed Methods Using a combination of methods of data collection and analysis and, hence, different types of data, has been a controversial issue since the very early days of social research. Over the past fifty years or more many concepts have been used to refer to this practice, including: 'multitrait/multimethod research' (Campbell and Fiske 1959), although this referred to the use of several quantitative methods; 'triangulation' (Denzin 1970); 'mixed strategies' (Douglas 1976); 'combining methods' (Cook and Reichardt 1979); 'linking data' (Fielding and Fielding 1986); 'combining research' (Bryman 1988); 'multimethod research' (Brewer and Hunter 1989, 2006); 'combined research' (Creswell 1994); and 'mixed methodology' (Tashakkon and Teddlie 1998). While there has been no general agreement on nomenclature, there seems to be a move towards using 'mixed methods' as the identifying concept. According to its advocates, the use of mixed methods involves the collection, analysis and mixing of both quantitative and qualitative data in a single study or a series of studies (Creswell and Piano Clark 2017). Some writers have gone so far as to suggest that mixed methods constitute a third methodological movement, along with quantitative and qualitative research (e.g. Tashakkori and Teddlie 2003b; Teddlie and Tashakkori 2003; Johnson and Onwuegbuzie 2004). The term 'triangulation' was used initially to refer to the use of combinations °f methods. In some fields - for example, in nursing research - it is still the 213 214 Designing Social Research more commonly used term (see Twinn 2003). Tashakkori and Teddlie (2003b) have argued for the use of terms that distinguish various types of combinations: 'mulrimethod research' for studies that use more than one method but of the same type - that is, combinations of just qualitative or qualitative; 'mixed methods' for studies that combine quantitative and qualitative methods, either in parallel or sequence; and 'mixed model' for studies that combine methods that are used within different ontological and epistemological assumptions. Rather than focusing on combinations of methods that produce different types of data, Blaikie and Priest (2017) have focused on the use of 'multiple paradigms' to answer a set of research questions, using different logics of inquiry and ontological and epistemological assumptions, either in parallel or sequence. On the assumption that quantitative and qualitative methods are being combined, the following arguments have been put forward for the use of mixed methods. • Strengths of one method offset weaknesses in other methods. • Provide more comprehensive evidence. • Help answer research questions that cannot be answered by one method alone. • Encourage researchers with different views (and skills) to collaborate. • Encourage the use of multiple paradigms.^ • Is practical as the researcher is free to use all possible methods (Creswell 2007: 9-10). Unfortunately, some of these points are nothing more than platitudes. For example, the first point was made about triangulation but, as we shall see, combining methods with similar weaknesses may simply magnify rather than neutralize them. Early Developments The idea of using a combination of methods of data collection goes the earlv years of social research in America and Europe. A variety of were used in the pioneering study that was commenced tn London in_1886 by {The Life and Labour of the People of London, Charles Booth and his associates < published in seventeen volumes between 1891 and 1903). '- research showed us for the first time nov His collaborator, seventeen Beatrice Webb, claimed that Booth's rnc«u, ,-------~ — ^ best to combine the quantitative and qualitative examination osoaa (Webb 1948: 210-2). Interviews, participant observat.on and statistical o» from secondary sources were used. Later, community studies m America su 'Middletown' (Lynd and Lynd 1937, 1956) and 'Yankee City g ________i.. „r '\A„,:a„th^\' in Ansrna (Lazarsieiu « as Middletown (LyiiudiHi.)™ - - ,, .„rjj et Lunt 1941), and the community study of 'Manenthal' in Austria Of While Questions have Deen i«» back to methods Lunt and UlC iuiuuiujiu; amv., .------------ al 1933) all used a combination of methods. While questio as to whether the way these researchers combined methods sausuo...... fi p -ceo, or notn or compound their deficiencies. If they are divergent ^^^l^C^^^^l?)^^ Lges; " °r aCt'°n "7 POSSibk= «™ of *em; trlat all thelnd- A« r^f.<-<■;..« —----- that these pioneer soci of various kinds mgs with ca„ti„n ■---------j liLW or mem; treat all the find- werpretation; or V1ew the discrepancy as something that requires further 216 Designing Social Research investigation. Just which alternative a researcher chooses will depend on such things as background knowledge of the research topic and the social context, relative confidence in the methods themselves, and the aims of the research. The so-called 'combining' of methods is not a mechanical process; it requires a great deal of judgement based on knowledge and experience. It is the comparison of data produced in different ways that is of greatest value. Rossman and Wilson (1985) addressed the issue of convergence and recognized that data produced by different methods will invariably not converge. They used combinations of quantitative and qualitative data for three purposes: corroboration, elaboration and initiation. Corroboration is the classic use of triangulation to establish validity; elaboration occurs when a variety of data expands understanding of the phenomenon, perhaps by providing different perspectives; and initiation refers to the use of non-convergent data in a provocative way to produce new interpretations and conclusions, to suggest further areas of research, or to reformulate research questions. They illustrated these uses in their own research. At about the same time as Denzin was advocating the use of triangulation, Sieber (1973) went beyond concerns about validity checking to seeing data-collection methods as building upon one another in a staged process. His ideas on how survey research and field work can be usefully combined anticipated many current ideas and are still relevant. Obstacles to Combining Methods A number of writers have recognized that there are obstacles to using a combination of methods: it can be expensive; it takes more time; researchers may not have sufficient training in both qualitative and quantitative methods; and fads in the preferences of funding agencies for one type of method (Reichardt and Cook 1979). However, we need to ask why social researchers have tended to use only one method of data collection. It is a common experience in the social sciences that individual researchers work on individual research projects with a single method of data collection and analysis. Types of Mixed Methods Research Over the past twenty years, various attempts have been made to identify ways in which mixed methods have been used (see, for example, Greene et al. 1989; Parton 1990; Morse 1991; Brannen 1992; Steckler et al. 1992; Hammersley 1996; Greene and Caracelli 1997; Morgan 1998; Tashakkori and Teddlie 1998; Creswell 1999; Sandelowski 2000; Creswell etal. 2003; Tashakkori and Teddlie 2003b; Creswell et al. 2004; Bryman 2006b, 2016; Creswell and Piano Clark 2007,2017; Bergman 2008; Creswell and Poth 2017; Creswell and Creswell 2018). Creswell and Piano Clark (2007) provided an elaboration of Hammersley s classification into four major types: triangulation, embedded, explanatory and exploratory. Triangulating quantitative and qualitative data can be d ways: Methods for Answering Research Questions 217 one in a variety of • comparing quantitative and qualitative data of equal weight within the same time frame (convergence model); • transforming one type of data into the other form and mixing them during the analysis stage (data transformation model); • within a survey instrument, including a few open-ended questions to provide interesting quotes (validating quantitative data model); and • using different methods at different levels within, for example, an organization (multilevel model). In the embedded procedure, one type of data plays a supplementary role in helping to design the study, elaborate the procedures and/or interpret the results. Usually it is the qualitative component that is supplementary, although the reverse is in principle possible. Embedding can occur in both experimental and non-experimental research. The explanatory procedure is the most straightforward kind of mixed methods research and comes in two forms: a quantitative phase produces results that need to be elaborated or explained and this is done by a follow-up qualitative phase; or a preliminary quantitative phase is used ro provide the basis for selecting participants for the major qualitative phase. The exploratory procedure usually reverses the order of the phases. An initial qualitative phase can be used to develop new quantitative measures or instruments. Alternatively, this phase can be used to identify unknown variables, develop a classification system or develop propositions to be tested, perhaps from an emergent theory. A more elaborate classification can be found in Tashakkori and Teddlie (2003b). Bryman (2006b) undertook a study of 232 articles in which mixed methods were used by researchers in five fields: sociology; social psychology; human, social and cultural geography; management and organization behaviour; and media and cultural studies. He concluded that sixteen distinct types of mixed methods research have been practised. Most of these can be reduced to Creswell and Piano Clark's (2007) four types. A Neglected Issue In the current explosion of writing on the use of mixed methods, the issue of ontological assumptions is largely absent. The focus has been on methods of data collection and, sometimes, data analysis. We wish to argue that, while qualitative and quantitative methods need not be associated with particular ontological assumptions, although this may frequently occur in practice, both types °f methods are used in the context of some kind of ontological assumptions, whether or not researchers are conscious of the fact. Some researchers appear to work with one type of assumption consistently, regardless of what method they ate using. 218 Designing Social Research If a researcher does adopt different ontological assumptions with a mixture of methods, the question arises as to how this should be handled. Is it legitimate to compare or combine results that have been produced with different assumptions? Is it possible to move between different assumptions as different methods are used? And, if so, how can this be done? The answers to these questions will differ depending on whether methods are being used concurrently or sequentially. And they can become relevant when one type of data is transformed into another type. What is assumed about the nature of reality when each method is being used can become significant when they are 'mixed' in some way. As we have seen, methods can be combined both concurrently and in sequence. The first alternative allows for the use of qualitative and quantitative methods together, provided both types are used with the same ontological assumptions. It is only under these circumstances that most of what has been written about triangulation is relevant. Different methods can be used to explore aspects of the same (assumed) 'objective' reality, regardless of whether they use words or numbers. They could also be used to explore (assumed) single or multiple socially constructed realities, although some methods (qualitative) might be more suitable for this than others. Data from different sources can be translated from one form to another (although usually from qualitative to quantitative) with impunity. However, what cannot be done is to combine data that are produced by different methods if each deals with different (assumed) realities. For example, it is not possible to use data related to a single 'absolute' reality to test the validity of data related to multiple 'constructed' realities, regardless of what methods are used in each case. If different types of methods are used with the same ontological assumptions, the implications for achieving convergence or divergence of results can be handled, either in terms of providing reciprocal support in the case of convergence, or as an explanatory challenge in the case of divergence. When two or more methods are used with different assumptions, convergence and divergence take on different meanings; the results may relate to different realities. In this case, while converging results cannot be used as any kind of test of validity, or for mutual support, comparison can be used to stimulate theory development or further research. The second alternative is to use different methods, possibly with different assumptions, at different stages of a research project; that is, sequentially. It is not necessary for all stages to adopt a consistent set of ontological and episte-mological assumptions. To answer research questions relevant to each stage, the researcher can move back and forth between logics of inquiry, and ontological assumptions, during the course of the research. Data collected at one stage can be interpreted in the light of data gathered at another stage. For example, statistical results from a survey could be interpreted, that is, be better understood, in terms of social actors' meanings and motives that have produced the statistical patterns, using some in-depth interviews with a carefully selected sub-sample of respondents. See Blaikie and Priest (2017) for illustrations of how multiple paradigms can be used in practice, both in parallel and in sequence. Switching between ontological assumptions requires considerable awareness of the various sets of assumptions that are being used and the capacity to keep the various (assumed) realities separate. For example, a researcher may use a Methods for Answering Research Questions 219 questionnaire with Deductive logic, and then in-depth interviews with Abductive logic. However, ro then attempt to combine the data produced by these two methods is to fail to recognize the differences in ontological assumptions that can be associated with the use of each logic. What is more common is for a researcher to use a combination of qualitative and quantitative methods with one logic of inquiry, say Inductive or Deductive. The ontological problems of combining different methods, particularly quantitative and qualitative methods, are not a critical issue in these two logics of inquiry as all methods can be interpreted within a consistent ontology. In this case, qualitative data are likely to be viewed as uncoded quantitative data that the researcher has to translate into variables, rather than, say, as evidence of the social actors' meanings and interpretations. Where meanings are recognized, they will be conceived differently than they would when Abductive logic is used. (See Halfpenny 1979: 815-16.) Conclusion It should be clear from this discussion that, while the use of combinations of methods within research projects can be traced back more than a hundred years, and arguments for it go back about fifty years, there has been a quite recent upsurge in the interest in mixed methods in a number of branches of social research. In our view, the use of multiple methods should be seen as normal and usually necessary in many types of research, and particularly when more than one research question is being investigated. It is the naive 'mixing' of them that is at issue. The four sample research designs in chapter 12 illustrate the use of multiple methods. They represent a series of stages in a research programme that began with quantitative methods and then moved to the use of qualitative methods as knowledge in the area increased and the nature of the research questions changed. Social research should be seen as a process of moving back and forth between stages that may require the use of different logics of inquiry and different methods for different purposes at each stage. Incidentally, this process has nothing to do with conventional views of triangulation and its accompanying concern with the improvement of validity. Using a combination of methods should not be seen as a special category of research; perhaps the use of single methods should be treated this way. The use of single methods is usually associated with narrow and perhaps one-off research topics. Such research provides limited opportunities for advancing knowledge. Hence, the use of a variety of methods, however this is organized, should be seen as the norm. 9.8 Logics of Inquiry and Research Methods There is no necessary connection between logics of inquiry and methods of data collection and analysis. While some methods may be more commonly associated *Kn a particular logic of inquiry, this is largely a matter of convention rather 220 Designing Social Research than a methodological requirement. However, this does not mean that methods can be used without ontological and epistemological assumptions. Rather, if a particular method (e.g. unstructured interviewing) is used in association with Inductive logic, in that context it will be serving whatever assumptions have been adopted. Similarly, if the same method is associated with the use of Abductive logic, the data that it produces will need to be interpreted within the assumptions adopted. In other words, methods can serve a number of masters, but they need to change their 'colours' to do so, and the data they produce will need to be interpreted within the particular ontological and epistemological assumptions that are adopted. When Inductive logic is used, any kind of data is appropriate; generalizations can be produced from data in either words or numbers. While numbers may allow for more precise propositions, the relevance or the possibility of achieving such precision will be a matter of judgement or circumstances; sometimes using words alone may be all that is possible or necessary. In any case, propositions in words are still necessary in quantitative studies. The issue is how such propositions are arrived at, and that is a matter of choice or opportunity. Deductive logic is more commonly used in association with quantitative methods and data, but this need not be so. Qualitative researchers may also construct theories in the deductive form and test them using qualitative data. The more important issue is whether researchers are engaged in theory construction or theory testing, rather than the type of methods or data they use. It is evident from research inspired by Harre's constructionist version of realism, and the application of Retroductive logic, that a combination of methods can be used. Pawson (1995) and Pawson and Tilley (1997) have certainly advocated the use of both quantitative and qualitative methods, very much in a sequence, in terms of establishing and exploring observed patterns, contexts and mechanisms. Both these uses of Retroductive logic rely on cognitive mechanisms for their explanations and they incorporate constructionist elements. The question of the ontological status of cognitive mechanisms (in contrast to the structures and mechanisms in Bhaskar's version of Critical Realism), that are dependent on the socially constructed reality social actors inhabit, needs further consideration. Because of the nature of the subject matter (social actors' meanings, motives and interpretations), Abductive logic is commonly associated with qualitative methods. The patterns, however, for which an understanding or explanation is sought, may have been established using quantitative data. It should now be clear that the choice of logic of inquiry does not determine the method or methods that should be used in social research. However, careful consideration needs to be given to what a method is supposed to do when associated with a particular logic of inquiry, and with the ontological and epistemological assumptions that lie behind its use at a particular time and in a particular place. Many researchers appear to use particular methods with little or no awareness of the ontological and epistemological assumptions that they have adopted. This seems to be common among quantitative researchers who have been socialized in, or have confined themselves to, a narrow research tradition; their assumptions are taken for granted and are unlikely to be seen to involve choices. On the other hand, some researchers have the capacity, and find the need, to work with more Methods for Answering Research Questions than one logic of inquiry and their associated ontological and epistemological assumptions, either for different research projects, or even within a single project. While we strongly recommend this approach to social research, we recognize that it requires a sophisticated awareness of both the assumptions and the logics of inquiry that are being used at any time. 9.9 Conclusion This chapter has brought us to the 'nuts and bolts' of social research, of obtaining and processing data. But we are not done with making choices yet.1 A critical decision has to be made about the timing of data collection or generation, and this will determine an important feature of a research design: whether it will be regarded as cross-sectional, longitudinal or historical. Social research that involves the passage of time can be faced with compromises on the ideal of, say, following a sample or population over time. Throughout the book reference has been made to data collection and data generation. The former usually occurs when traditional quantitative methods arc used; for example, when people are asked to answer a predetermined set of questions, frequently with pre-coded response categories. The researcher is in control of what will be asked and how responses are to be made. Qualitative methods are more likely to involve the generation of data. What this means is that the process is much more flexible, thus allowing participants some scope to respond on their terms, not simply the researchers'. Perhaps the information-rich form of data generation occurs when the process requires participants and researchers to be joint producers of the 'data'. An important step in moving from data collection and/or generation to data analysis is the use of methods of data reduction. Just what is involved here will depend on whether the data are quantitative or qualitative in from. The aim at this stage is to transform the data into a suitable form for the analysis. If quantita-tive, this may be just the recoding, reordering or combining of pre-coded categories, or it can involve sophisticated techniques to extract 'latent' dimensions or clusters in the data. Coding and recoding are also used with textual (qualitative) data, and more complex methods are also available to 'abstract' themes or types from the data. However, with some qualitative methods, the boundaries between data generation, reduction and analysis are very blurred; these activities may occur in an iterative and cyclical manner. When quantitative data analysis is undertaken, it is important to take into account the level of measurement that was used in its collection, and to distinguish between methods that are appropriate for analysing single variables, two variables, and many variables, as well as between descriptive and explanatory methods. If probability samples have been used, then some form of inferential analysis will be required to provide estimates of population characteristics from those found in a sample. It is important to understand the principles here, as the research literature is replete with the misuse of inferential analysis. Qualitative data analysis may be confined to descriptive methods, and some researchers regard this as being all that is necessary. However, theory generation 221 222 Designing Socio/ Research is an important, perhaps the ultimate and certainly the most difficult, form of qualitative analysis. This is where the use of 'mixed methods' can come into play; where research is viewed as occurring in stages, perhaps using different methods at each stage, and involving both theory generation and testing. T Methods for Answering Research Questions 223 9.10 Further Reading Blaikie, N. (2003). Analyzing Quantitative Data. A non-technical introduction to the theory and practice of quantitative analysis. Bryman, A. (2016). Social Research Methods. A readable and comprehensive introduction to both quantitative and qualitative methods, Bryman, A. (ed.) (2006a). Mixed Methods (4 vols). A comprehensive review of the development and variety of mixed methods research. Corbin, J. M. and Strauss, A. (2015). Basics of Qualitative Research. A long-standing and well-used discussion of grounded theory. Crcswell, D. and Piano Clark, V. L. (2017). Designing and Conducting Mixed Methods Research. One of a growing number of texts devoted to mixed methods. Denz.in, N. K. and Lincoln, Y. S. (eds) (2005). Handbook of Qualitative Research. This and previous editions provide a number of relevant chapters, de Vaus, D. (2014). Surveys in Social Research. The standard text on survey research. Flick, U. (2014). An Introduction to Qualitative Research. A useful introduction with a European flavour. Hammersley, M. and Atkinson, P. (2007). Ethnography. The standard text on ethnography. Hardy, M. and Bryman, A. (eds) (2004). Handbook of Data Analysis. A broad, middle-range collection on quantitative and qualitative analysis. Kaplan, D. (ed.) (2004), The Sage Handbook of Quantitative Methodology for the Social Sciences. A narrow and advanced collection on data analysis. Lewis-Beck, M. S., Bryman, A. and Liao, T. F. (eds) (2004). The Sage Encyclopedia of Social Research Methods (3 vols). A comprehensive coverage of everything a social researcher needs to know, and more. Mason, J. (2017). Qualitative Researching. A practical and methodologically sophisticated introduction. Minichiello, V., Aroni, R. and Hays, T. (2008). ln-Depth interviewing. A very useful coverage of this method. Neuman, W. L. (2014). Social Research Methods. A standard and comprehensive discussion of both quantitative and qualitative methods. Piano Clark, V. L. and Creswell, J. W. (eds) (2008). The Mixed Methods Reader. A useful collection of previously published papers. Punch, K. F. (20,4, lntToduction t0 Soaa! R^ An extensive set of especially written paper, W hams, M (2003). Mak.ng Sense of Social Research ds in Social and e relationship between theory and See also the Sage four-volume collections on various aspects of social research.