1
A Comparative Analysis of Qualitative and Quantitative Research Methods and a
Justification for Adopting Mixed Methods in Social Research
Abstract
The aim of this review is to create awareness about uses of available social research methods
and to provide a guideline in adopting appropriate methods specifically in qualitative and
mixed methods research genre. Based on the review of contemporary social research methods
I believe that mixed methods research produces more accurate results than relying on either
qualitative or quantitative methods alone in explaining complex social issues. This paper
contributes to the methodological literature in two areas. First, create awareness among social
researchers and students about the available research methods in order to help them to adopt
suitable research designs in addressing their particular research questions. Second, encourage
scholars from all disciplines to theorize further, especially in the field of mixed methods, and
engage in a dialogue in order to improve methodological appropriateness for future research
in social sciences.
Keywords: qualitative research methods; quantitative research methods, mixed methods
research; social research paradigms
Correspondence: Muhibul Haq, Bradford University School of Management, Emm Lane,
Bradford, West Yorkshire BD9 4JL. Email: m.haq@bradford.ac.uk, Ph: 07975544094
Author Biography
Muhibul Haq is a Social Researcher and is affiliated with the School of Management,
University of Bradford. His research interests are Strategic Human Capita, Ethnic Minority
Enterprises and Social Research Methodologies.
Cite this article as:
Haq, M (2014). A comparative analysis of qualitative and quantitative research methods and a
justification for use of mixed methods in social research. Annual PhD Conference, University
of Bradford Business School of Management. June 2014.
Introduction
Social research has traditionally depended on two paradigms, the functional or scientific
paradigm and the interpretive or constructivist paradigm (Shah and Corley, 2006; Scotland,
2012). A paradigm is a basic set of belief systems we hold and it is this belief system which
guides our disciplined actions (Guba, 1990). The type of these paradigms (e.g., positivism,
interpretivism etc.) can be distinguished based on their ontological (nature of knowledge or
reality) and epistemological (relationship between the knower and the knowable) positions
and methodological approaches (how to access the knowledge) as well as ethical standards
(moral principles guiding the enquiry) (Guba, 1990; Bazeley, 2002). Those who follow the
scientific paradigm tend to adopt quantitative research methods, whereas the constructivist
paradigm leads to embracing qualitative tools and techniques of data collection and analysis.
The divide between the two fundamental paradigms has increased over time due to continued
differences in their philosophical underpinnings. For example, the scientific paradigm is
based on the ontological assumption of objectivity, i.e., a single reality exists about a social
phenomenon which can be understood using quantifiable facts obtained by adopting
quantitative data collection and analysis methods (Shah and Corley, 2006). On the other
hand, the constructivist paradigm implies that an objective reality does not exist since all
realities are socially constructed (Maxwell, 2012; Wikgren, 2005).
2
The divide between the two approaches paved the way for a third approach called mixed
methods. The justification of mixed methods use in social research is based on the pragmatic
philosophical position and explicates that social realities can be better understood by using
both qualitative and quantitative data collection and analysis methods in the same research
(Creswell, 2002; Dellinger and Leech, 2007; Caruth, 2013).
In favor of this argument, researchers argue that it is difficult to capture social reality by
following only either a positivist or a relativist paradigm and their associated methods
because of their selective nature (Mays and Pope, 1995). Scholars also believe that a
combination of the methods will help researchers to understand the nature of a social reality
more coherently (Driscoll et al., 2007). Mixed methods research, therefore, has gained
popularity gradually and researchers started using it as an alternative approach form a variety
of disciplines including psychology, education, health, sociology, business and management,
ecology and environment (Bryman, 2006; Caruth, 2013; Driscoll et al., 2007; Jogulu and
Pansiri, 2011). It is the pragmatic nature of mixed methods that some researchers argue
mixed methods help achieve multiple goals, such as; explanation, confirmation and
triangulation (Jick, 1979; Caruth, 2013; Gioia and Thomas, 1996; Creswell, 2002; Tashakkori
and Teddlie, 2010; Teddlie and Tashakkori, 2012; Feilzer, 2010) in explaining complex
social constructs. For example, it was found in the United Kingdom in the last decade that
the rise in volume of quantitative social data required alternative methods (qualitative
followed by mixed methods ) in order to better investigate social policy related questions
(Brannen and Moss, 2012).
Subsequently, a number of scholarly resources were made available for researchers and
students to guide them in how and when to choose mixed methods. These include, but not
limited to, The Handbook of Mixed Method Research (Tashakkori and Teddlie, 2010) as well
as Journal of Mixed Methods and International Journal of Multiple Research Approaches
(Brannen and Moss, 2012; Denzin, 2010).
The first motivation to write this review came from the realization that in spite of availability
of the above guidance and the fact that mixed methods offer clear benefits over the other two
methods, an interdisciplinary dialogue seems to be absent in the methodological literature.
Although this call is not new as others (Denzin, 2010; Creswell and Garrett, 2008) have
already asked for a debate in this area, their call has not gone far enough in making clearer
the emerging nature of mixed methods and acquainting novice researchers and students with
methodological trends and complexities.
The second motivation in writing this paper came from the concept of the so called paradigm
wars. As outlined by Denzin (2010) researchers in social sciences witnessed at least three
paradigm wars since 1980s. Historically, the result of the first war was the decreased
popularity of positivism and the increased use of qualitative methods, while the second
paradigm war resulted in mixed methods being increasingly used. The third war emerged in
the 1990s and still is going on. As a resulted pragmatic approaches became trendy in
addressing social research questions, i.e., researchers found compatibility in combining both
qualitative and quantitative methods in the same research project. The third war also gave
birth to other similar, but not the same as mixed methods, moments (such as emergentmethods
and evidence-based discourses (Denzin, 2010)). Elaboration of these moments is not
in the scope of this paper.
In the light of the above two motivation factors I posit, based on the methodological literature
that the lack of coordination and cooperation across the social research spectrum is leading to
3
isolated furtherance of methodological knowledge and practice by each discipline in
confinement. One may argue that the prevailing divergence or lack of coordination among
disciplines and scholars might be an opportunity for further research as research
methodologies keep evolving, specifically mixed methods which “has not yet reached the
consensus stage” (Creswell and Garrett, 2008). Nevertheless, the lack of cooperation among
disciplines and scholars results in lack of understanding of methodological adoptability and
appropriate training in methodological issues. This in turn causes negative consequences in
social enquiry (Denzin, 2010) when it comes to mixing methods according to specific social
research questions in context, such as cultural and religious sensitivities as well as space and
time.
In other words, mixed methods research boundaries are still blurred and perhaps this is why
some researchers (e.g., Thornberg, 2010) tend to believe that they have used mixed methods
in their research when in fact they have only described their small qualitative samples in
numbers. More specifically, while quantifying qualitative data is not rare, merely describing
purposefully selected small samples in numerical form and calling it mixed methods is
questionable if we consider the widely used definitions of mixed methods (Driscoll et al.,
2007; Jogulu and Pansiri, 2011; Creswell and Plano Clark, 2007; Creswell and Tashakkori,
2007).
In summary, I argue in this paper, that the social research community from various
paradigmatic standpoints needs to communicate to each other so that issues surrounding
methodological adaptation, particularly in mixed methods, could be demystified and social
researchers could have a clearer understanding about the methods appropriate for them to use.
To achieve the above objective, I compare and contrast qualitative and quantitative methods
at each stage of the research process by providing brief examples of most widely used
tools/techniques. This comparison highlights the unique weaknesses and strengths of
qualitative and quantitative methods and will help researchers, especially novices and
students, in choosing and tailoring research methods in the context of their research
questions and according to their epistemological and methodological positions. I then discuss
mixed methods, in relatively greater detail, as a way of a tailored research design. This is
followed by a critical evaluation of complementary benefits and non-overlapping
disadvantages when qualitative and quantitative methods are mixed. By doing so, I argue that
researchers can achieve more reliable research out comes when mixed methods are adopted.
Table 1 summarises advantages and disadvantages of using qualitative, quantitative and
mixed methods social research. Furthermore, this paper advocates for the adoption of a
pragmatic research approach and encourages further theorization and empirically
investigation to illuminate the usefulness and limitations of social research methods,
particularly mixed methods. This will guide social research community in generating
knowledge leading to better understanding of complex social constructs within specific
social, economical, religious, cultural and political settings.
Comparing and Contrasting Qualitative and Quantitative Research Methods
Qualitative Social Research
Qualitative research became popular as social research methodology since 1960s when social
scientists realized that they needed to understand complex social issues much further than
before when they relied on quantitative methods (Alasuutari, 2010). In other words,
qualitative research is typically concerned with inductive analysis of a social reality with a
descriptive and exploratory orientation (Guest et al., 2011).
4
Table 1: Comparison of Qualitative, Quantitative and Mixed Methods in Social Research
Note: list of the above issues is not exhaustive and the order does not represent priority.
Unlike quantitative researchers who generally impose a framework of their own on the
researched, qualitative researchers see the world of social reality from the perspective of their
participants (Holloway and Wheeler, 1996; Guest et al., 2011; Curry et al., 2009). This is
called the emic perspective, which means how local people think about their world instead of
etic perspective (quantitative research) which is concerned how researchers think what is
important in a local context (Holloway and Wheeler, 1996; Harris, 1976; Morris et al., 1999).
Etic approach generally depends on brief and structured observations of a large group(s) of
Issue Qualitative methods Quantitative methods Mixed methods
1. Overall aim Understanding and
explanation of social
phenomena
Generalization and
conformation
Aims both explanation and
generalization
2. Sample size small Large Both small and large
3. Amount of data Large amounts of
textual raw data
Relatively small
amount of numerical
data
Both large and small amounts
of data
4. Relationship
with respondents
Close one-to-one
relationship
Almost no direct
relationship
Close one-to-one relationships
with some but not with all
respondents
5. Frequently used
data collection
techniques
Semi-structured
interviews, easy but
costly and time
consuming
Large scale surveys,
low response rates, less
costly and less time
consuming
Combines methods based on
objectives, more costly and
time consuming than the other
two methods
6. Frequently used
data analysis
techniques
Thematic content
analysis, tedious and
time consuming
Statistical analysis
using computer-aided
programmes, relatively
simple and quick
Combines methods from
qualitative and quantitative
approaches, takes longer time
and costs more
7. Flexibility and
standardization
Flexible Less flexible than
qualitative analysis
More flexible than both
8. Research
process and data
quality
Meticulous record
keeping adds value to
quality of process and
data
Compromises quality
of data for
standardization
Quality of process and data is
considered better than the other
two methods
9. Interpretation of
results
Lot of interpretation is
required
Interpretation is
concise due to use of
statistics
Interpretation is harder and
longer because of the use of
both qualitative and
quantitative methods
10.Generalizability In general,
generalizability is not
an objective
Highly generalizable in
general
Generalizability is stronger
than in any of the other two
methods
11.Triangulation In general, no
triangulation is done
In general, no
triangulation is done
Triangulation is done
12.Overall
usefulness,
assuming cost,
time and expertise
are not issues
More useful than
quantitative methods
in understanding
social phenomena
More useful than
qualitative methods in
replication
More useful than both
qualitative and quantitative
methods in all aspects
13.Term(s) used
for quality of
research
trustworthiness rigour Both trustworthiness and rigour
5
individuals and generalizes results from sample to population. Emic approach, however,
focuses on a wide range of observations of a small group(s) and strives to paint an inside
picture of the way social reality is created (Orb et al., 2001).
Therefore, the overall aim of qualitative research is to build theory and define new variables,
which quantitative researchers test (Shah and Corley, 2006; Press, 2005), using rich and indepth
information from participants’ standpoint (Krefting, 1991). Qualitative researchers
achieve this aim by understanding lived experiences of people and then represent these
experiences as abstracts (Elliott et al., 1999).
In short, qualitative research typically leads to explanation rather than generalization (Payne
and Williams, 2005) as it untangles the meanings attached to a given social construct.
However, there are three areas qualitative research is criticized. First, small samples lead to
no or poor generalization or replication; second, researcher bias guiding to interpretation of
raw data based on the researchers’ own predispositions (Stenbacka, 2001; Morse et al., 2008;
Mays and Pope, 1995); and third, in extreme cases qualitative research can lead to
idiosyncratic theory building, i.e., one theory relates to one issue or one case or one
individual only (Eisenhardt, 1989). Although qualitative methods contributed to the overall
systemization of social research, they did not necessarily replace any existing methods of
empirical as well as theoretical research.
Quantitative Social Research
Quantitative social research is about collecting numerical data and analyzing it using
statistical methods to explain a phenomenon. Unlike qualitative research which argues that
there is no pre-existing reality, quantitative research in its extreme form assumes that there
exists only one single reality about a social phenomenon which is not influenced by
researchers in any way (Muijs, 2010).
Scholars at the other extreme of the epistemological spectrum believe that reality is socially
constructed and researchers play a vital part in its interpretation based on personal and or
societal factors such as their own mental models, cultures and politics. However, both Press
(2005) and Muijs (2010) believe that the above extreme views are not actually practiced in
social research due to two reasons (Eisenhardt, 1989). First, laws of natural sciences, in
which all objects behave in the same way to any changes in their environment, are not
generally applicable to human beings as different individuals would react differently to the
same change in their situations. Second, it will be worthless to produce a piece of research
work which is idiosyncratic to one person and is not generalizable or transferable to other
people in the same or similar situations.
Unlike qualitative research which is used when little is known or if there is uncertainty about
a phenomenon (Cronholm and Hjalmarsson, 2011), quantitative research is used to find cause
and effect or the relationships between variables mostly to verify/nullify theory or hypothesis
(Creswell, 2002; Teddlie and Tashakkori, 2012; Feilzer, 2010).
While quantitative methods are primarily concerned with numerical data, most social
phenomena (such as education, health and organizational performance) do not naturally
generate numerical data (Muijs, 2010). Social researchers mitigate these weaknesses by using
a number of techniques, instruments or indirect variables such as scales in questionnaires that
ask to rate a phenomenon like strongly agreed to strongly disagreed (Muijs, 2010). The use of
proxy indicators is another way to generate indirect numerical data to measure a social
6
phenomenon, such as number of years of schooling and number of trainings attended to
measure the level of human capital within an individual (Crook et al., 2011; Hatch and Dyer,
2004). Critics argue that although these measures may provide solutions to social issues to
some extent, quantitative researchers over rely on procedures (Jogulu and Pansiri, 2011) and
sacrifice quality of information for standardization (Stenbacka, 2001). Moreover, while
quantitative research is considered more rigorous because it is relatively easier in quantitative
research than in qualitative research to check validity, reliability and generalizability of
results (Stenbacka, 2001; Jick, 1979), quantitative researchers tend to miss more by looking
at the phenomenon as outsiders than qualitative researchers do as they become part of the
research process (Mays and Pope, 1995; Harris, 1976).
Sampling in Qualitative Research
Unlike quantitative research in which large sample sizes are considered important for rigour,
qualitative research tends to focus on smaller size samples where the researcher acts as part
of the instrument and records information in a natural context to uncover its meaning in a
variety of qualitative ways such as description, explanation and exploration (Suter, 2011).
Quantitative sampling and data collection techniques, such as random sampling and surveys,
are not practical in the qualitative research for a number of reasons including the nonfeasibility
of researching large samples because of time and cost constraints (Marshall, 1996).
The most commonly used sampling methods in qualitative research are snowball (Suter,
2011) and purposive (Suter, 2011; Jogulu and Pansiri, 2011; Tongco, 2007) sampling, or a
combination of them. Snowballing is adopted if fewer potential informants are available and
if it is expected that these respondents will lead to more or more-relevant informants (LewisBeck
et al., 2004). In purposive sampling researchers select informants who are considered
knowledge experts about the phenomenon in consideration (Tongco, 2007; Basu, 1998; Suter,
2011).
Other sampling techniques, which are used less commonly in qualitative social research, are
incidental quota sampling, convenience sampling, intensity sampling, deviant case sampling,
maximum variation sampling, criteria sampling and open-ended questionnaire surveys
(Bricki and Green, 2007; Bryman and Bell, 2007; Marshall, 1996). While the selection of
fewer samples in qualitative research leads to weak generalization of results from samples to
population (though this is not the aim of qualitative researchers), they provide the opportunity
of analyzing a specific social reality in detail yielding unique insights from rich data (Suter,
2011) which is not possible in quantitative research.
Sampling in Quantitative Research
The most frequently used sampling technique in quantitative research is random sampling
which has a variety of variations such as systematic random sampling, stratified random
sampling and quota random sampling. The most popular one is systematic random sampling
in which each sample is selected from a list of potential respondents by following a random
selection method. This type of sampling is adopted when the population is large enough. In
case of a small population the whole population can be selected in order to draw reasonably
acceptable statistical conclusions. Smaller samples are not useful in quantitative research due
to the fact that: response rate is generally low (Newby et al., 2003) and responses less than
100 cannot be used to draw statistically significant results (Shook et al., 2004).
Data Collection in Qualitative Research
7
The data collection methods available in qualitative research include (but not limited to)
ethnography, focus group discussions, open ended questionnaires, unstructured interviews,
analysis of videos and photographs, and archival documents analysis. The most commonly
used technique in qualitative research is face-to-face semi-structured interviews (Eisenhardt
and Graebner, 2007; Press, 2005). Compare to other methods, this method records more and
better-accurate data, depending on the research question and objectives, by capturing feelings
and body gestures of respondents in addition to what they verbally express (Gilbert, 2008). In
this method, a number of open ended questions are asked which give freedom to both the
interviewer and interviewee to ask relevant and varied questions and to express feelings and
experiences in a relaxed environment (Bricki and Green, 2007; Hollway and Jefferson, 2008;
Bryman and Bell, 2007; Guest et al., 2011; Denscombe, 2010; May, 2011; Orb et al., 2001).
This method gives freedom to respondents to tell their story in terms of their own lived
experiences in their own words, which they may be unwilling or unable to share in an indirect
situation such as surveys.
In addition, the one-to-one semi-structured interview process is easy to control as it involves
only two people, the interviewer and the respondent, and only one data source at a time (the
respondent) (May, 2011; Denscombe, 2010). Although this methods is generally more
expensive and time consuming than other methods (except unstructured interviews (Zhang
and Wildemuth, 2006)), its benefits are greater as it provides the opportunity for the
researcher to collect in-depth information during the interview by asking both probing and
prompting questions (Denscombe, 2010) which is not possible in most other data-gathering
methods. Moreover, without being so close to their respondents and without observing or
noting each bodily gestures as well as each word they (respondents) utter, Goffman (1989)
believes social researchers will not be able to do a serious piece of research work.
Data Collection in Quantitative Research
Various instruments are used to collect quantitative data including telephone interviews, webbased
surveys, postal surveys and structured questionnaires. Each instrument has its own
strengths and limitations in terms of time, cost and quality of data. Since most quantitative
researchers in social sciences use structured questionnaires for data collection (Newby et al.,
2003; Bryman, 2006; Driscoll et al., 2007; Edwards et al., 2002; Jogulu and Pansiri, 2011),
they do not have any control or influence over the respondents that leads to low response
rates. Researchers use various techniques to increase response rates including, but not limited
to, use of pre-notification letters to respondents, post card follow-ups, first class outgoing
postage and cash/monetary incentives (Fox et al., 1988; Newby et al., 2003; Edwards et al.,
2002). Similar to sampling, data collection methods in quantitative research are also
relatively simple (Newby et al., 2003). Therefore, this topic is not further discussed in this
review.
Data Analysis in Qualitative Research
In general, data analysis is more complicated in qualitative than in quantitative research
because data are buried in the text in most qualitative research (Bricki and Green, 2007).
Since most qualitative researchers deal with issues where no or little prior knowledge is
available, they tend to adopt inductive analysis (Elo and Kyngäs, 2007) to make sense of the
data. The most commonly used qualitative data analysis techniques are: thematic analysis,
discourse analysis, descriptive approaches, interpretative phenomenological analysis,
grounded theory and narrative analysis (Bricki and Green, 2007; Braun and Clarke, 2006;
Graneheim and Lundman, 2004; Arregle et al., 2007).
8
The specific data analysis technique to be adopted depends on the research objectives and the
type of data collected. For example, in an exploratory or explanatory qualitative research in
which data are collected using a semi-structured interview technique, researchers may find
thematic analysis as a most suitable technique as it is most widely used for identifying,
analyzing, and reporting patterns or themes from the data (Braun and Clarke, 2006;
Graneheim and Lundman, 2004; Bricki and Green, 2007).
The process of identifying themes by breaking down the textual data into manageable chunks
and them coding each of these chunks is called code and retrieve process (Hollway and
Jefferson, 2008). Although the process of coding and pattern-building is a tedious job, it
helps researchers in identifying rich sources of codes and themes providing basis for
interpretation of the data and drawing meaningful conclusions.
Data Analysis in Quantitative Research
Quantitative researchers use a number of statistical data analysis tools to analyze and make
sense of quantitative data. These tools include, but not limited to, SEM, LISREL, SPSS,
MATLAB, MINITAB, STATA, STATISTICA, SPSS etc. The decision to use a data analysis
tool depends on the research objectives and type of data available.
Discussing each of these tools is not in the scope of this paper, however, Structural Equation
Modeling (SEM) is discussed here as an exemplar due to the fact that it is widely used in
various disciplines including psychology and social sciences (Anderson and Gerbing, 1988),
Management Information Systems (Chin, 1998) and Strategic Management (Shook et al.,
2004). In other words, the use of SEM is viewed an appropriate strategy as it is a
comprehensive analysis tool for statistical analysis of quantitative data (Anderson and
Gerbing, 1988; Suhr, 2006). Therefore, SEM is briefly discussed here as an exemplar of
quantitative data analysis.
Unlike many first generation statistical analysis methods; such as regression, correlation and
analysis of variance; use of SEM, as a second generation method offers a number of benefits
over traditional statistical techniques if applied correctly (Suhr, 2006; Chin, 1998). For
example, SEM incorporates observed (measured) and unobserved (latent constructs) variables
in a multivariate environment to perform path analytic modeling, it allows researchers to
carry out multiple tests to determine the best model fit, it allows to take multiple
measurements for latent constructs to resolve multicollinearity problems, it represents
complex relationships among several variables in a visual way instead of only describing
them in complex statistical statements, and it tests a priori theoretical assumptions against
empirical data (Suhr, 2006; Chin, 1998).
In addition, SEM has the unique ability to analyze a number of interdependent relationships
in which dependent variables become independent variables simultaneously, it analyzes
multiple dependent variables at the same time (Shook et al., 2004) and has excellent
prediction and explanatory capability of endogenous (dependent) latent variables in model
estimation (Ringle et al., 2012).
Critics argue that SEM is a mathematically complex package which many researchers find
difficult to use and to make sense of their data (Chin, 1998; Shook et al., 2004). To address
the issues of technical complexities of this as well as other modern statistical analysis
9
packages future research is required to guide researchers in three areas. First, how to make
sure their data meet the assumed multi-variability of normal distribution of indicator
variables? Second, how to measure indicators appropriately as week measures may result in
inappropriate modifications to structural models and false findings. Third, what will be an
acceptable sample size to assess model fit because larger samples may produce trivial
differences and smaller samples may lead to suspicious model fit.
Validity and Reliability in Qualitative Research
Validity and reliability, in other words rigour, refer to the extent whether the object under
research has been truly measured and whether the process or method of measurement gives
the same results over and over again when repeated by others for similar research objects
(Drost, 2011; Stenbacka, 2001). This type of language typically exists in quantitative research
because the overall aim of a qualitative research is to explain/explore, rather than to measure,
why a social reality is the way it is. Trustworthiness, however, in qualitative research is the
equivalent to validity and reliability in quantitative research (Munhall, 2011).
Nevertheless, critics of qualitative research believe that small samples in qualitative research
lead to either poor and non-generalizable results (Stenbacka, 2001; Morse et al., 2008; Mays
and Pope, 1995) or, in extreme cases, idiosyncratic theory building where each individual
phenomenon, case, person or situation needs a theory for its own explanation (Eisenhardt,
1989). In addition to small samples, qualitative research paradigm is mostly criticized for
three reasons: First, research bias (due to close involvement of the researcher in the data
collection and therefore his/her personal impression and interpretation); second, lack of
reproducibility (due to the personal understanding of the researcher as other researchers using
the same methods/data may not come to the same conclusion); and third, generalisability
(because of large amounts of data from small samples) (Mays and Pope, 1995).
However, admirers (e.g., Stenbacka, 2001; Krefting, 1991) argue that unlike quantitative
research in which quality of data is sacrificed for quantity or standardization, qualitative
research focuses on small samples to gather a rich set of data which lead to deep
understanding of a social phenomenon. This point has been highlighted by a number of
scholars (e.g., Britten and Fisher, 1993; Stenbacka, 2001; Payne and Williams, 2005) as they
argue that smaller sample size in qualitative research is in fact not a generalizability problem.
This is done, as they believe, on purpose for three reasons. First, to include those groups
which are disadvantaged or in minority or hard to reach; second, to maximize diversity in
order to describe the range of phenomena; third, to go beyond mere numbers, yes/no answers
and superficially measured likert scales in uncovering deep feelings based on lived
experiences of people. The notion that research bias negatively affects research quality is
refuted by Tongco (2007) as she argues that it contributes positively to reliability of the
research process and the data. Her argument is significant as it contributes to the debate that
quality of the overall research should be considered instead of focusing only on results. This
line of argument was echoed by Stenbacka (2001). She argued that the overall quality of
qualitative research can be achieved if the research process and methods are sufficiently
explained and documented so as to help others to trace the results back to each step of the
research in order to have confidence in the overall research.
I argue that the main problem in qualitative research, with reference of validity and
reliability, is the use of competing and sometimes conflicting jargons in the literature. For
example, Dellinger and Leech (2007) used terms such as truth value, credibility,
10
dependability, trustworthiness, generalisability, legitimation and authenticity to refer to the
overall quality of qualitative research. Onwuegbuzie and Johnson (2006) identified the
flowing five types of validities in qualitative research in order to simplify this concept. (1)
descriptive validity (factual accuracy of the research); (2) interpretive validity (accuracy of
the interpretation of the data from the respondents’ standpoint); (3) theoretical validity (the
consistency of theatrical explanation with data); (4) evaluative validity (whether an
evaluative framework can be applied to the researched); and (5) generalizability (whether
results of the research can be generalised to other contexts, individual, times etc). Mays and
Pope (1995) suggested the following six steps to be followed in qualitative research to ensure
reliability and rigor as well as to minimize research bias. (1) A systematic and self-conscious
research design; (2) accuracy in data collection; (3) a thorough data analysis by involving
other researchers and consulting with previous relevant research findings; (4) effective
communication with respondents in data collection and with other researchers in
interpretation; (5) detailed documentation of the research process; and (6) meticulous
documentation of the data and process.
The endeavours made by the above scholars are indeed significant contributions to the
literature of qualitative social research but they made the process of selecting methods
complicated due to use of conflicting terminologies. Therefore, their efforts are not helpful
enough in qualitative social research training and for novice researchers in selecting suitable
methods that can lead them in achieving research outcomes effectively in the context of their
research questions. This paper does not aim to narrow down the research of methodological
improvement, it rather posits that further research is required in this field as the qualitative
research tradition is still evolving. Therefore, it needs refinement and it remains somehow
under-researched.
Validity and Reliability in Quantitative Research
The issues of validity and reliability are generally less complicated in quantitative research
than they are in qualitative research (Onwuegbuzie and Johnson, 2006). It is perhaps due to
this reason that the recommendations to improve validity and reliability in quantitative
research did not change much during the last half a century. For example, the four steps
recommended, in this regard, by Campbell and colleagues in the 1960s has been repeated by
others in 2010s (Campbell et al., 1963; Drost, 2011). These four steps are: (1) Statistical
conclusion validity - it refers to the fact that if a relation exists between the variables that can
be tested statistically. (2) Internal validity - it refers if a causal relationship exists between
variables and if the samples are representative and are not biased. (3) Construct validity - it
refers to the fact that how well a concept or construct is interpreted into a functioning and
understandable reality. (4) External validity - to what extent a causal relationship between
concepts or variables is generalisable.
However, some quantitative researchers (Dellinger and Leech, 2007; Onwuegbuzie and
Johnson, 2006) in the recent past have focused on three types of validities. They are: (1)
content validity - this refers to the fact that whether items on a measurement instrument
represented the specific domain which was aimed to be studied; (2) criterion validated - to
check if scores from the test correlated to other constructs or if the scores were able to predict
future scores to hypothetically related constructs; and (3) construct validity - it refers to the
fact if the instrument measured the construct as it was claimed. Dellinger and Leech (2007)
particularly argue that in fact construct validity covers all types of validities including
measurement-related validity, design-related validity and statistical inference validity. I found
11
their argument significant due to the fact that construct validity overlaps with all other types
of validities and if the construct has been measured and meaning of data has been interpreted
properly that would mean other types of validities are covered.
In addition, social researchers have increasingly used a wide variety of computer software
programmes in justifying validity and reliability of their research (Guest et al., 2011; Mays
and Pope, 1995). This help, however, has been greater in quantitative than in qualitative
research. This is because quantitative research deals with numerical data that is easier to
manipulate or analyse using statistical software packages such as SPSS, SEM and LISREL.
Contrary to this, researchers find it difficult to analyse qualitative data (such as text, pictures,
videos etc.) using word processors or other computerised packages, such as Nvivo and Atlas-
ti.
Due to the inherent limitations of qualitative and quantitative methods, scholars tend to shift
their attention towards adopting mixed methods to maximise research rigour and to minimise
various methodological weaknesses which are difficult to avoid if either a qualitative or
quantitative method alone is used.
Mixed Methods Research
Although the rise of qualitative social research methods increased our ability to better
understand social constructs, researchers increasingly found themselves inclined, specifically
since 1990s (Alasuutari, 2010), towards using both qualitative and quantitative methods in
the same research. Some researchers (e.g., Symonds and Gorard, 2008) believe that this shift
in researchers’ attitude was triggered by the categorizations that all numerical research is
quantitative and all other research is qualitative. They argue that any research involving both
numerical data and other types of data necessitated the birth of mixed methods as a third
enquiry paradigm. A similar point was raised by Alasuutari (2010), as he believes that since
1990s the ambition among social researchers to make research more robust than before
resulted in the widespread use of mixed methods as a recognized research methodology.
Likewise, others (e.g., Brannen and Moss, 2012) posit that mixed methods came to existence
due to the potential it offers us to see social relations and complexities clearer by combining
methods and by recognizing the limitations that each method has.
Yet there are those who believe that the driving force for the popularity of mixed methods
was the concept of triangulation in social research (Symonds and Gorard, 2008; Jick, 1979;
Bryman, 2004). Triangulation increases the validity of research findings as it mitigates both
the inherent bias associated with individual methods and the possibility of making
inappropriate generalizations. Triangulation also offers the opportunity to present multiple
findings about the same phenomenon due the deployment of various elements of both
qualitative and quantitative methods in the same piece of research.
In practice, mixed methods research generally refers to the processes of collecting and
analyzing both qualitative and quantitative data in a single research project (Driscoll et al.,
2007; Jogulu and Pansiri, 2011). In other words, mixed methods research involves combining
research techniques, methods, approach as well as language from both qualitative and
quantitative research traditions in a single research enquiry (Symonds and Gorard).. Creswell
and Plano Clark (2007, p. 5) define mixed methods as:
“As a methodology, it involves philosophical assumptions that guide the direction of
the collection and analysis of data and the mixture of qualitative and quantitative
12
approaches in many phases in the research process. As a method, it focuses on
collecting, analyzing, and mixing both quantitative and qualitative data in a single
study or series of studies. Its central premise is that the use of quantitative and
qualitative approaches in combination provides a better understanding of research
problems than either approach alone”.
Similarly, Creswell and Tashakkori (2007, p. 3) offer the following definition:
“... the use of mixed methods means that the studies report both qualitative and
quantitative research and include both approaches in the data collection, analysis,
integration, and the inferences drawn from the results.”
In the light of the above definitions, this paper recognizes that the use of more than one
method in the same research is not a new idea because researchers prior to 1990s did make
use of more than one method (Alasuutari, 2010; Brannen and Moss, 2012). However, social
researchers since then started using both qualitative and quantitative methods in the same
research more frequently than before and used the term mixed methods more assertively than
they did in the past (Alasuutari, 2010). The increased use of mixed methods is attributed to its
nature. This is a flexible research approach which involves both qualitative and quantitative
methods (or their elements) and which is directed by the requirement of the research question
instead of what a researcher wants to know or explore (Muijs, 2010; Bronstein and Kovacs,
2013). In mixed methods, researchers have the freedom to use all necessary data collection
and analysis tools rather than confining themselves to only one method. Moreover, mixed
methods provide more comprehensive insights of a given social phenomenon than either
quantitative or qualitative research methods offer (Creswell and Plano Clark, 2007; Creswell
and Garrett, 2008). In short, a mixed methods research design is employed when both breadth
and depth are required to be looked at or when relationships between variables, as well as
meanings of specific social phenomena, are required to be determined and
explained/explored.
Benefits of Mixed Methods
As mentioned above, mixed methods research usually provides more rigorous results than
adopting either qualitative or quantitative data collection and analysis methods alone (Shah
and Corley, 2006; Jick, 1979; Driscoll et al., 2007; Bryman, 2006). This is because most
social issues cannot be measured directly nor can they be calculated accurately using direct
variables or by adopting either a quantitative or a qualitative research approach. Take the
example of the amount and level of human capital in an organization which is the sum total
of individual level qualifications, skills, trainings and abilities which are not solely owned by
the firm but can be used by the firm in attaining strategic objectives. Although this construct
has been researched in the past with the help of a number of proxy indicators or by using
indirect variables (Mahsud et al., 2011; Unger et al., 2011; Basu, 1998), no research has ever
been able to capture the impact of this socially complex and causally ambiguous construct on
individual- and/or firm-level performance using direct variables. Moreover, the continued
divide between qualitative and quantitative research methods in social sciences further
pushed measurement/assessment of complex social constructs beyond the bounds of
possibility. However, it is possible to draw insights about this and other complicated social
phenomena by using a combination of qualitative and quantitative research methods.
In addition to adding value to research results, mixed methods research provides important
insights that might be missed when only a single method is used and relied on (Cronholm and
13
Hjalmarsson, 2011). Moreover, use of mixed methods eliminates a number of unnecessary
and unproductive controversies between qualitative and quantitative research which are
currently present in the literature. For example, as expressed by Press (2005), some scholars
on both sides of the divide mistakenly assume that quantitative social researchers presuppose
regularity in social behavior similar to that found in natural sciences whereas qualitative
social scientists supposedly believe that all social events are idiosyncratic therefore, any kind
of replication is not possible. This article adds to her assertion: while theses extreme views
might not exist in practice, no research is fully generalizable or replicable because of social
dynamics; and no single truth is possible in social research because some generalization is
always possible as one individual’s experience can match, to some extent, to other
individuals’ experiences in a similar situation under similar conditions. Because of these
reasons, mixed methods in social research is a better choice than using either qualitative or
quantitative methods alone in explaining complex social phenomena in a meaningful way
(Gioia and Thomas, 1996; Jick, 1979). Moreover, researchers have the unique opportunity to
build and test theory at the same time in the same research by mixing qualitative and
quantitative research methods as traditionally theory building happens in qualitative research
whereas theory testing is the domain of quantitative research (Shah and Corley, 2006; Curry
et al., 2009).
Those who criticize the mixed methods paradigm base their critique on the perceived
epistemological and empirical underpinnings of mixed methods (Symonds and Gorard,
2008). They argue that since the combination of elements from the two traditional paradigms
in social research (qualitative and quantitative) forms a third paradigm (mixed methods)
which is philosophically considered pragmatic, therefore it inhibits further development of
new and more effective social research methods. However, some researchers counter argue
that since social researchers bring their own predispositions to their research, it is not possible
to be totally subjective or objective in collecting, analyzing and interpreting their data without
having some sort of inherent bias no matter what methodologies they adopt (Symonds and
Gorard, 2008; Charmaz, 2014). Similarly, some researchers argue that such extreme views,
being either a pure positivist or interpretivist, are not actually practiced in social research for
at least two reasons. First, laws of natural sciences are not applicable in social sciences and
that an idiosyncratic piece of research does not provide any useful insights about how a given
society/culture operates (Press, 2005; Muijs, 2010). Second, interpretation is required in
quantitative research and counting can be adopted as a way of analysis in qualitative research
(Bazeley, 2002; Curry et al., 2009).
In view of the significance of the above counter arguments, it makes sense to say that mixed
methods, as a social research paradigm, is neither a proxy nor it is a duplication of any of the
two traditional methods. Furthermore, the use and presence of mixed methods does not
hinder the potential of further developing and refining social research methodologies. This is
because the widespread use of qualitative methods since 1960s and mixed methods since
1990s did not cause demise of quantitative and qualitative methods respectively. Therefore,
contrary to Symonds & Gorard’s (2008) suggestion any new developments in social research,
including but also beyond and above mixed methods, is not likely to cause death of mixed
methods any time soon.
However, while it is undisputed that mixed methods yields more in-depth and more reliable
research results than relying on either qualitative or quantitative methods alone (Denzin,
2010), how and when methods should be mixed remains a question in the literature. A variety
of mixed methods designs, which scholars have used in the past, are given next.
14
Types of Mixed Methods
Methods can be combined in different ways depending on the research objectives
(Tashakkori and Teddlie, 2010; Teddlie and Tashakkori, 2012; Onwuegbuzie and Johnson,
2006; Bronstein and Kovacs, 2013) as well as researchers’ interests and experiences. A brief
outline of different mixed methods in social research is given below.
Concurrent: qualitative and quantitative data are collected and analyzed independently but
concurrently.
Sequential: qualitative and quantitative data are collected and analyzed separately in a
sequence, one after the other.
Conversion: data are converted or transformed from one form to another (e.g., quantifying
qualitative data) and then analyzed.
Parallel: both types of data are collected but analysis is done in an integrated way in a
parallel design.
Fully mixed: in a fully mixed design, qualitative and quantitative approaches are mixed in an
interactive way at all stages of the research.
Transformative: in this approach qualitative and quantitative data collection methods are
chosen based on data requirements and changes are made in the research process during the
project when and as required.
Sequential Mixed Methods
While methods can be mixed in a variety of designs, evidence suggests that a sequential
mixed method is the most widely used design in mixed methods research (Driscoll et al.,
2007). This argument is strengthened by the analysis of 232 journal articles by Bryman
(2006) in which he found that more researchers used sequential mixed methods than all other
forms of mixed methods. Furthermore, Onwuegbuzie and Johnson (2006) posited that the two
approaches which preponderate mixed methods research are actually variations of sequential
mixed methods. Researchers can collect and analyze one type of data in the first phase that
may direct and inform the type of data to be collected and analyzed and the type of method to
be adopted in the second phase. This approach is also called non-fixed and emergent mixed
methods (Bronstein and Kovacs, 2013). Alternatively, researchers can mix the two research
approaches (qualitative and quantitative) at each phase of the research in a sequential way to
inform the next phase.
Whatever tradition is adopted, mixed methods in general and sequential mixed methods in
particular, though based on the research question(s) and context, offer a great deal of
confirmation and validation of the research process and results (Onwuegbuzie and Johnson,
2006; Dellinger and Leech, 2007; Jick, 1979; Curry et al., 2009). Explanation how each of
these types of mixed methods work or how ought they should be adopted is not in the scope
of this paper. However, a typical sequential mixed methods design, in which qualitative data
is collected and analyzed before quantitative data is collected, is conceptualized in Figure 1
as an exemplar.
The main reason to choose this exemplar is that this type of mixed methods is the most
widely used mixed method design in social research, as given above. The second reason is
that it matches the author’s own ontological and epistemological position: knowledge about a
social reality can be better accessed and understood from different angles and by adopting
both qualitative and quantitative data collection and analysis tools and techniques in the same
15
social enquiry. I also believe that there can exit more than one reality about a given social
construct and each reality can be correct in its own right as well as in the context where it
exists and where it is explicated. All these realties can also be fallible at the same time.
On a practical level, this type of mixed methods design is particularly useful as the codes and
themes indentified during the qualitative phase provide basis for variables and constructs for
the quantitative phase of the research, even though some scholars (Bazeley, 2002) have
questioned that this type of mixed methods design may not represent a fully integrated mixed
methods. However, their own recognition that this is the widely used mixed methods design
in practice and their use of the word ‘integrated mixed methods’ is indicative of this design
being a useful mix of methods. In addition, while I accept that this is not the only design of
mixed methods, and also it may not be the best form of mixed methods, this type of design is
used by more researchers than any other forms of mixed methods. This is due to the fact that
this type of mixed methods approach provides the opportunity for researchers to collect and
analyze data and judge each method without diluting the other (Brannen and Moss, 2012).
Figure 1: A typical example of Sequential Mixed Methods Research Design
Stages Qualitative data collection and analysis Quantitative data collection
and analysis
6 Final research results, testing or verification of theory/hypothesis and
triangulation
5 Quantitative data analysis
(Structured Equation Modeling)
4 Quantitative data collection
(structured questionnaire survey)
3 Interpretation of Qualitative data
(thematic analysis)
2 Qualitative data Analysis (coding and
identification of core categories)
1 Qualitative data collection (semistructured
interviews)
Sequential mixed methods technique also offers the process of triangulation as data are
collected from different sources about the same phenomenon or at different points from the
same source. Such multi-lens and multi-angle data can be merged into a single point or
compared with each other to obtain more reliable outcomes than results derived from data
16
collected using a single source or at a single point in time and/or space. This is because what
people say about a situation often differs from what they would do if they were faced with the
same situation (Goffman, 1989; Press, 2005; Charmaz, 2004). In short, triangulation reduces
uncertainty and enhances confidence and thus provides a rational for use of mixed methods
(Jick, 1979; Bryman, 2004). To illuminate this argument further, I now discuss four
exemplars where sequential mixed methods were used as a social research design.
In their study, Driscoll et al. (2007) collected quantitative data using a survey followed by
qualitative data using a semi-structured interview technique. They then transformed
qualitative data into quantitative data before final analysis. With the help of embedded
qualitative data they were able to explain and augment many contradictions they found in the
quantitative data. Similarly, Jogulu and Pansiri (2011) analyzed two doctoral research
projects which used sequential mixed methods data collection and analysis techniques. In
both projects, the doctoral researchers collected and analyzed quantitative followed by
qualitative data using in-depth semi-structured interviews to explain quantitative results. By
mixing methods, the doctoral researchers were able to expand their statistical data
interpretation as the qualitative data enabled them to elaborate quantitative results and
address the research questions more accurately which otherwise might have not been possible
if only a single research method was used. Similarly, Jick (1979) used a mixed methods
approach in his PhD research in which he found that mixing methods was a stronger research
design in finding out employees’ anxiety in uncertain and apparently insecure situations. He
collected and analyzed quantitative data using a survey design followed by qualitative
interview data, as well as archival data collection from within the company reports/registers,
which he sued to triangulate the quantitative survey analysis results.
Likewise, Gioia and Thomas (1996) also used sequential mixed methods, where they
collected qualitative data by interviewing all three members of the top management of a large
public university several times. They interviewed top management team members of other
similar institutions too and analyzed 11 out of a total of 25 in-depth interviews. Based on the
core categories identified, they developed a questionnaire which they sent out to a much
larger sample for quantitative data collection. This technique can be particularly useful if the
social phenomenon at hand is an abstract and if it is difficult to know whether the right
questions were asked in the first phase of the research. By adopting a sequential design, they
were able to reflect the emerging themes from the first phase in the questionnaire which
helped them in further analysis using statistical tools and thus in confirmation and
triangulation of their results. This type of mixed methods design can strengthen the overall
research results and improve trustworthiness of the research process and its results (Shah and
Corley, 2006; Jick, 1979; Gioia and Thomas, 1996).
Validity and Reliability in Mixed Methods Research
The issues of validity and reliability in mixed methods research are not the same which
researchers face in qualitative and quantitative research as separate methods. The process of
combining qualitative with quantitative methods although creates complementary strengths, it
also results in non-overlapping weaknesses (Onwuegbuzie and Johnson, 2006; Dellinger and
Leech, 2007). Complementary strengths refer to the value added to the overall research when
qualitative and quantitative methods are mixed, whereas nonoverlapping weaknesses refer to
the difficulties of: analysing qualitative textual data (also called problem of representation);
and making credible, transferable and confirmable inferences (also called problem of
legitimisation or integration).
17
As qualitative and quantitative research approaches complement each other (Britten
and Fisher, 1993), the concept of trustworthiness can somehow be compatible when
appropriate methods are employed, such as sequential mixed methods. Moreover, scholars
adopting mixed methods have a degree of leverage over qualitative and quantitative
researchers because use of mixed methods minimizes research bias (a criticism raised by
positivist paradigm) and the risk of survey questionnaires not being uniformly understood by
respondents and researchers (a criticism raised by interpretivist paradigm) (Mays and Pope,
1995). In addition, employing verification and self-correcting strategies at each stage of the
research process in mixed methods, which are considered measures of validity and reliability
improvement (Morse et al., 2008), may further contribute positively to research rigour. These
strategies, coupled with meticulous documentation of the research process, can add value to
the overall research as readers will be able to verify the research process should they wish so.
Stop.
To be specific, the following two paragraphs shed some light on how validity and reliability
can be dealt with in a sequential mixed methods approach. The hypothetical explication given
below is based on the conceptual framework given in Figure 1.
In the first phase of the research, qualitative data are collected from purposefully selected
individuals using a semi-structured interview technique, which provides a rich source of text
data leading to the identification of codes, categories and themes directed by the research
question. These codes, categories and themes would provide basis for interpretation of lived
experiences of the participants. This method of data collection is particular useful as it is nonforcing
(Stenbacka, 2001) and in which participants voluntarily share their real life
experiences in their own words. In this example (Figure 1), the qualitative thematic data
analysis would be followed by a close-ended questionnaire-based survey in which variables
and constructs will be defined based on the identified codes and themes. By doing so, both
internal and external validity can be increased as samples would be representative of the
population and assessing/measuring the phenomenon would not be flawed. In addition, the
second phase would compliment and triangulate results from the first phase. In order to
improve overall research quality, other researchers may be involved and asked to critically
evaluate each stage of the research and provide feedback, which can be considered and acted
upon.
Moreover, this type of mixed methods research can also achieve analytical generalization
(Stenbacka, 2001) as well as intermediate type of limited generalization (also called
moderatum generalization) which are acceptable norms in qualitative research as well as in
mixed methods (Payne and Williams, 2005). This point has been elaborated by Cronholm and
Hjalmarsson (2011) as they believe that in a mixed methods approach, quantitative data are
used to add precision to qualitative analysis results to verify or falsify empirical grounded
hypotheses, which is essentially the approach of sequential mixed methods.
Discussion and Conclusion
Although the divide between the two traditional research methods (qualitative versus
quantitative) resulted in a rich amount of scholarly investigation, the two traditions seem to
remain as apart as ever. As a result, social researchers started using mixed methods
increasingly as an alternative approach (Caruth, 2013) which overcomes many of the
weaknesses of qualitative and quantitative methods.
18
Although no method is better or more accurate than any other methods in social enquiry as
the use of a particular method depends on the nature and objectives of the investigation, there
are greater benefits of using mixed methods than relying on either qualitative or quantitative
methods alone. Nevertheless, scholars need to develop some form of consensus in terms of
when mixed methods should be adopted and how they should be mixed. This is because a
number of mixed methods are discussed in the literature without providing any clear
guidelines of how and when to adopt them keeping in mind the nature of the issues and the
overall research goals. In other words, methodological literature does not provide a step-bystep
guide by which researchers (especially novices and students) would be able to choose
data collection and analysis methods most suitable for their research.
Having said that and building on past published research (Onwuegbuzie and Johnson,
2006; Bryman, 2006; Jogulu and Pansiri, 2011), this article argues that mixed methods,
specifically sequential mixed methods, contribute more towards overall research quality than
other methods.
With respect to limitations in the existing literature and possible future directions, it is found
in this review that some scholars have suggested various steps to be taken to improve quality
of research mostly from qualitative and to some extent mixed methods research standpoints.
Most of such evidence in the literature is either confusing due to jargons used or simply
scant. The lack of scientific evidence of methodological appropriateness and understanding of
social research methods, especially among novice social scientists and students, always
results in difficulty to adopt research methods suitable for their unique research questions and
contexts. Future theorizing and imperial investigation is required to further the field of
methodological appropriateness in order to expand our understanding about the role data
collection and analysis methods play in conducting trustworthy and reliable social research.
In particularly, future research is required in the following areas three areas. (1) While
research suggests that mixed methods produce more accurate and reliable results than
qualitative and quantitative methods, there is little evidence of when and how to combine
methods keeping in mind the nature of the research problem and the available resources.
Addressing the above issue is of paramount importance due to the follow three sub-questions.
(i) the classification of methods without elaborating of why and how boundaries are drawn
between them and what will be the possible consequences in case if these boundaries are
violated (Symonds and Gorard, 2008)? (ii) What are the underlying reasons whereby
researchers tend to adopt sequential mixed methods than other designs? (iii) Whether or not
samples be repeated in mixed methods from one phase to another as repetition of samples
may add value to data due to respondents’ knowledge about the research and non–repetition
may contribute to control research bias and over exposure of the respondents. (2) While
qualitative research can produce high quality research results, there is hardly any research
suggesting a step-by-step approach in adopting and executing qualitative research methods
which are practicable and understandable by scholars including novices and students. (3) As
it is assumed SEM is widely used in quantitative research, evidence is scant to address
limitations of SEM. i.e., how to make sure quantitative data meet the assumed multivariability
of normal distribution of indicator variables, how to measure indicators
appropriately and what will be an acceptable sample size to assess model fit when SEM is
used for data analysis.
This article has some limitations. The author’s predisposition is that quantitative social
research stresses more on standardization of research process whereas qualitative and mixed
19
methods research emphasis more on the quality of data and process than protocols. Therefore,
quantitative methods are typically less complicated than qualitative and mixed methods are.
Part of this presumption is that quantitative research is less dynamic than qualitative and
mixed methods social research. Future social scholarship can shed some light on this
assumption so as to contribute to the whole debate of methodological appropriateness in
social scientific enquiry.
In summary, building on published research this review draws three conclusions. First,
although quantitative research is simple and is capable of producing superficially robust
results which can be generalised from samples to population, it does not necessarily provide
better or more valid explanations than qualitative research does. Second, while qualitative
research is complex and offers poor generalisation, it has the ability to capture deep meanings
of a social phenomenon thus explains social realities more accurately. Third, all research is
selective as no researcher can capture the factual truth of events by following either a
qualitative or quantitative tradition. In any case, quality of any research is an important issue
which can be assured by collecting sufficient, accurate and rich data to explain the social
phenomenon in consideration as well as to help readers to form an independent assessment of
the results. Such a broad objective is more likely achievable when appropriate mixed methods
are adopted. Although past research shows that use of sequential mixed methods is more
popular among social researchers than other forms of mixed methods, there are gaps and
limitations in using mixed methods in general and sequential mixed methods in particular. In
order to narrow the gaps and overcome the limitations, this article highlighted the need of a
dialogue among social researchers to further the debate of methodological appropriateness
with the aim to widen the choice of adopting suitable data collection and analysis methods in
the context of research questions, as well as physical and social conditions.
References
Alasuutari, P. (2010) The rise and relevance of qualitative research. International journal of social
research methodology, 13 (2), 139-155.
Anderson, J. C. and Gerbing, D. W. (1988) Structural equation modeling in practice: A review and
recommended two-step approach. Psychological bulletin, 103 (3), 411-423.
Arregle, J. L., Hitt, M. A., Sirmon, D. G. and Very, P. (2007) The Development of Organizational
Social Capital: Attributes of Family Firms. Journal of management studies, 44 (1), 73-95.
Basu, A. (1998) An exploration of entrepreneurial activity among Asian small businesses in Britain.
Small Business Economics, 10 (4), 313-326.
Bazeley, P. (2002) Issues in mixing qualitative and quantitative approaches to research. AIDS, 21 (2),
S91-S98.
Brannen, J. and Moss, G. (2012) Critical issues in designing mixed methods policy research.
American Behavioral Scientist, 0002764211433796.
Braun, V. and Clarke, V. (2006) Using thematic analysis in psychology. Qualitative research in
psychology, 3 (2), 77-101.
Bricki, N. and Green, J. (2007) A guide to using qualitative research methodology. London, London
School of Hygiene and Tropical Medicine.
Britten, N. and Fisher, B. (1993) Qualitative research and general practice. The British Journal of
General Practice, 43 (372), 270-271.
Bronstein, L. R. and Kovacs, P. J. (2013) Writing a Mixed Methods Report in Social Work Research.
Research on Social Work Practice, 23 (3), 354-360.
Bryman, A. (2004) Triangulation and Measurement, Department of Social Sciences, Loughborough
University Loughborough, Leicestershire. [Online] Available at:
20
http://www.referenceworld.com/sage/socialscience/triangulation.pdf [Accessed 26 October
2013].
Bryman, A. (2006) Integrating quantitative and qualitative research: how is it done? Qualitative
research, 6 (1), 97-113.
Bryman, A. and Bell, E. (2007) Business research methods. Oxford: Oxford university press.
Campbell, D. T., Stanley, J. C. and Gage, N. L. (1963) Experimental and quasi-experimental designs
for research. Houghton Mifflin Boston.
Caruth, G. D. (2013) Demystifying Mixed Methods Research Design: A Review of the Literature.
Mevlana International Journal of Education, 3 (2).
Charmaz, K. (2004) Premises, principles, and practices in qualitative research: Revisiting the
foundations. Qualitative Health Research, 14 (7), 976-993.
Charmaz, K. (2014) Constructing grounded theory. Sage.
Chin, W. W. (1998) Commentary: Issues and opinion on structural equation modeling. MIS quarterly,
vii-xvi.
Creswell, J. W. (2002) A Framework for Design. In Research Design: Qualitative, Quantitative and
Mixed Methods Approaches. 2nd. ed. Thousand Oaks: Sage.
Creswell, J. W. and Garrett, A. L. (2008) The" movement" of mixed methods research and the role of
educators. South African Journal of Education, 28 (3), 321-333.
Creswell, J. W. and Plano Clark, V. L. P. (2007) Designing and conducting mixed methods research.
Wiley Online Library.
Creswell, J. W. and Tashakkori, A. (2007) Editorial: Developing publishable mixed methods
manuscripts. Journal of Mixed Methods Research, 1 (2), 107-111.
Cronholm, S. and Hjalmarsson, A. (2011) Experiences From Sequential Use of Mixed Methods.
Electronic Journal of Business Research Methods, 9 (2), 87-95.
Crook, T. R., Todd, S. Y., Combs, J. G., Woehr, D. J. and Ketchen Jr, D. J. (2011) Does human
capital matter? A meta-analysis of the relationship between human capital and firm
performance. Journal of Applied Psychology, 96 (3), 443-456.
Curry, L. A., Nembhard, I. M. and Bradley, E. H. (2009) Qualitative and mixed methods provide
unique contributions to outcomes research. Circulation, 119 (10), 1442-1452.
Dellinger, A. B. and Leech, N. L. (2007) Toward a unified validation framework in mixed methods
research. Journal of Mixed Methods Research, 1 (4), 309-332.
Denscombe, M. (2010) The good research guide: for small-scale social research projects. Open
University Press.
Denzin, N. K. (2010) Moments, mixed methods, and paradigm dialogs. Qualitative inquiry.
Driscoll, D. L., Appiah-Yeboah, A., Salib, P. and Rupert, D. J. (2007) Merging qualitative and
quantitative data in mixed methods research: How to and why not. Ecological and
Environmental Anthropology (University of Georgia), 18-28.
Drost, E. A. (2011) Validity and reliability in social science research. Education Research and
Perspectives, 38 (1), 105-123.
Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R. and Kwan, I. (2002)
Increasing response rates to postal questionnaires: systematic review. Bmj, 324 (7347), 1183.
Eisenhardt, K. M. (1989) Building theories from case study research. Academy of management
review, 14 (4), 532-550.
Eisenhardt, K. M. and Graebner, M. E. (2007) Theory building from cases: opportunities and
challenges. Academy of management journal, 50 (1), 25-32.
Elliott, R., Fischer, C. T. and Rennie, D. L. (1999) Evolving guidelines for publication of qualitative
research studies in psychology and related fields. British Journal of Clinical Psychology, 38
(3), 215-229.
Elo, S. and Kyngäs, H. (2007) The qualitative content analysis process. Journal of advanced nursing,
62 (1), 107-115.
Feilzer, M. Y. (2010) Doing mixed methods research pragmatically: Implications for the rediscovery
of pragmatism as a research paradigm. Journal of mixed methods research, 4 (1), 6-16.
Fox, R. J., Crask, M. R. and Kim, J. (1988) Mail survey response rate a meta-analysis of selected
techniques for inducing response. Public Opinion Quarterly, 52 (4), 467-491.
21
Gilbert, N. (2008) Researching social life. Sage.
Gioia, D. A. and Thomas, J. B. (1996) Identity, image, and issue interpretation: Sensemaking during
strategic change in academia. Administrative science quarterly, 370-403.
Goffman, E. (1989) On Fieldwork. Journal of Contemporary Ethnography, 18 (2), 123-132.
Graneheim, U. H. and Lundman, B. (2004) Qualitative content analysis in nursing research: concepts,
procedures and measures to achieve trustworthiness. Nurse education today, 24 (2), 105-112.
Guba, E. G. (1990) The paradigm dialog. Sage Publications.
Guest, G., MacQueen, K. M. and Namey, E. E. (2011) Applied thematic analysis. Sage.
Harris, M. (1976) History and significance of the emic/etic distinction. Annual review of
anthropology, 5, 329-350.
Hatch, N. W. and Dyer, J. H. (2004) Human capital and learning as a source of sustainable
competitive advantage. Strategic management journal, 25 (12), 1155-1178.
Holloway, I. and Wheeler, S. (1996) Qualitative Research for Nurses Blackwell Science Ltd. Oxford.
Hollway, W. and Jefferson, T. (2008) The free association narrative interview method. 296-315.
Jick, T. D. (1979) Mixing qualitative and quantitative methods: Triangulation in action.
Administrative science quarterly, 24 (4), 602-611.
Jogulu, U. D. and Pansiri, J. (2011) Mixed methods: a research design for management doctoral
dissertations. Management research review, 34 (6), 687-701.
Krefting, L. (1991) Rigor in qualitative research: The assessment of trustworthiness. The American
journal of occupational therapy, 45 (3), 214-222.
Lewis-Beck, M., Bryman, A. E. and Liao, T. F. (2004) The Sage encyclopedia of social science
research methods. Vol. 1 Sage.
Mahsud, R., Yukl, G. and Prussia, G. E. (2011) Human capital, efficiency, and innovative adaptation
as strategic determinants of firm performance. Journal of Leadership & Organizational
Studies, 18 (2), 229-246.
Marshall, M. N. (1996) Sampling for qualitative research. Family practice, 13 (6), 522-526.
Maxwell, J. A. (2012) Qualitative research design: An interactive approach. Vol. 41 Sage
publications.
May, T. (2011) Social Research: Issues, Methods and Research. McGraw-Hill International.
Mays, N. and Pope, C. (1995) Rigour and qualitative research. BMJ: British Medical Journal, 311
(6997), 109-112.
Morris, M. W., Leung, K., Ames, D. and Lickel, B. (1999) Views from inside and outside: Integrating
emic and etic insights about culture and justice judgment. Academy of Management Review,
24 (4), 781-796.
Morse, J. M., Barrett, M., Mayan, M., Olson, K. and Spiers, J. (2008) Verification strategies for
establishing reliability and validity in qualitative research. International journal of qualitative
methods, 1 (2), 13-22.
Muijs, D. (2010) Doing quantitative research in education with SPSS. Sage.
Munhall, P. (2011) Nursing research. Jones & Bartlett Learning. Chapter 3 (Quantitative Versus
Qualitative Research, or Both?), pp. 35-52.
Newby, R., Watson, J. and Woodliff, D. (2003) SME survey methodology: Response rates, data
quality, and cost effectiveness. Entrepreneurship Theory and Practice, 28 (2), 163-172.
Onwuegbuzie, A. J. and Johnson, R. B. (2006) The validity issue in mixed research. Research in the
Schools, 13 (1), 48-63.
Orb, A., Eisenhauer, L. and Wynaden, D. (2001) Ethics in qualitative research. Journal of nursing
scholarship, 33 (1), 93-96.
Payne, G. and Williams, M. (2005) Generalization in qualitative research. Sociology, 39 (2), 295-314.
Press, N. (2005) Qualitative research: thoughts on how to do it; how to judge it; when to use it.
Genetics in Medicine, 7 (3), 155-158.
Ringle, C., Sarstedt, M. and Straub, D. (2012) A Critical Look at the Use of PLS-SEM in MIS
Quarterly. MIS Quarterly (MISQ), 36 (1), iii-xiv.
Scotland, J. (2012) Exploring the Philosophical Underpinnings of Research: Relating Ontology and
Epistemology to the Methodology and Methods of the Scientific, Interpretive, and Critical
Research Paradigms. English Language Teaching, 5 (9), 9-16.
22
Shah, S. K. and Corley, K. G. (2006) Building Better Theory by Bridging the Quantitative–
Qualitative Divide. Journal of Management Studies, 43 (8), 1821-1835.
Shook, C. L., Ketchen, D. J., Hult, G. T. M. and Kacmar, K. M. (2004) An assessment of the use of
structural equation modeling in strategic management research. Strategic Management
Journal, 25 (4), 397-404.
Stenbacka, C. (2001) Qualitative research requires quality concepts of its own. Management Decision,
39 (7), 551-556.
Suhr, D. (2006) The basics of structural equation modeling. [Online] Available at:
http://www.lexjansen.com/wuss/2006/tutorials/tut-suhr.pdf [Accessed on 3 November 2013].
Suter, W. N. (2011) Introduction to educational research: A critical thinking approach. Sage.
Symonds, J. E. and Gorard, S. (2008) The death of mixed methods: research labels and their
casualties. Jossey-Bass.
Tashakkori, A. and Teddlie, C. (2010) Sage handbook of mixed methods in social & behavioral
research. Sage.
Teddlie, C. and Tashakkori, A. (2012) Common “Core” Characteristics of Mixed Methods Research
A Review of Critical Issues and Call for Greater Convergence. American Behavioral
Scientist, 56 (6), 774-788.
Thornberg, R. (2010) Schoolchildren's social representations on bullying causes. Psychology in the
Schools, 47 (4), 311-327.
Tongco, M. D. C. (2007) Purposive sampling as a tool for informant selection. Ethnobotany Research
& Applications, 5, 147-158.
Unger, J. M., Rauch, A., Frese, M. and Rosenbusch, N. (2011) Human capital and entrepreneurial
success: A meta-analytical review. Journal of Business Venturing, 26 (3), 341-358.
Wikgren, M. (2005) Critical realism as a philosophy and social theory in information science?
Journal of Documentation, 61 (1), 11-22.
Zhang, Y. and Wildemuth, B. M. (2006) Unstructured interviews. [Online] Available at:
http://hsmi.psu.ac.th/upload/forum/Unstructured_interviews.pdf [Accessed on 28 September
2013].