Getting started:
reviewing the
literature
Chapter outline
Reviewing the existing literature 98
Getting the most from your reading 98
Systematic review 102
Narrative review 110
Searching the existing literature 113
Electronic databases 113
Keywords and defining search parameters 118
Referencing your work 120
The role of the bibliography 123
Avoiding plagiarism 124
Checklist 127
Key points 127
Questions for review 128
5
Getting started: reviewing the literature98
Chapter guide
The goal of this chapter is to provide guidance for students on how to get started on their research
project. Once you have identified your research questions (see Chapter 4), the next step in any research
project is to search the existing literature and write a literature review. The principal task at this early
stage involves reviewing the main ideas and research relating to your chosen area of interest. This
provides the basis for the writing of a literature review, which forms an important part of the dissertation.
This chapter will advise students on how to go about searching the literature and engaging critically with
the ideas of other writers. It will also help you to understand some of the expectations of the literature
review and give you some ideas about how to assess the quality of existing research.
Reviewing the existing literature
Why do you need to review the existing literature? The
most obvious reason is that you want to know what is
already known about your area of interest so that you do
not simply ‘reinvent the wheel’. Your literature review is
where you demonstrate that you are able to engage in
scholarly review based on your reading and understanding
of the work of others in the same field. Beyond this,
using the existing literature on a topic is a means of
developing an argument about the significance of your
research and where it leads. The simile of a story is also
sometimes used in this context (see Thinking deeply 5.1).
Whatever different understandings of the literature review
process you adopt, it is important to be clear about
the goal that the process is directed towards achieving.
A competent review of the literature is at least in part a
means of affirming your credibility as someone who is
knowledgeable in your chosen area. This is not simply a
matter of reproducing the theories and opinions of other
scholars, but also being able to interpret what they have
written, possibly by using their ideas to support a particular
viewpoint or argument. The purpose of exploring
the existing literature should be to identify the following
issues.
• What is already known about this area?
• What concepts and theories are relevant to this area?
• What research methods and research strategies have
been employed in studying this area?
• Are there any significant controversies?
• Are there any inconsistencies in findings relating to
this area?
• Are there any unanswered research questions in this
area?
This last issue points to the possibility that you will be
able to revise and refine your research questions in the
process of reviewing the literature.
Getting the most from your reading
Since a great deal of time during the early stages of your
research project will be taken up with reading the existing
literature in order to write your review, it is important
to make sure that the process of reading is also preparing
you for this. Getting the most out of your reading involves
developing your skills in being able to read actively and
critically. When you are reading the existing literature
try to do the following.
• Take good notes, including the details of the material
you read. It is infuriating to find that you forgot to
record the volume number of an article you read and
that needs to be included in your Bibliography. This
may necessitate a trip to the library on occasions when
you are already hard pressed for time.
• Develop critical reading skills. In reviewing the literature
you should do more than simply summarize what
you have read. You should, whenever appropriate, be
critical in your approach. It is worth developing these
skills and recording relevant critical points in the
course of taking notes. Developing a critical approach
is not necessarily one of simply criticizing the work of
others. It entails moving beyond mere description and
Getting started: reviewing the literature 99
asking questions about the significance of the work. It
entails attending to such issues as: How does the item
relate to others you have read? Are there any apparent
strengths and deficiencies—perhaps in terms of methodology
or in terms of the credibility of the conclusions
drawn? What theoretical ideas have influenced
the item? What are the implications of the author’s
ideas and/or findings? What was the author’s objective
in conducting the research? What are the main
conclusions and are they warranted on the basis of
the data provided in the item? What are the author’s
assumptions?
• Your search for literature should be guided by your
research questions, but as well you should use your
review of the literature as a means of showing why
your research questions are important. For example, if
one of your arguments in arriving at your research
questions is that, although a lot of research has been
done on X (a general topic or area, such as the
secularization process, female entrepreneurship, or
employee absenteeism), little or no research has been
done on X1 (an aspect of X), the literature review
is the point where you can justify this assertion.
Alternatively, it might be that there are two competing
positions with regard to X1 and you are going to investigate
which one provides a better understanding. In
the literature review, you should outline the nature
of the differences between the competing positions.
The literature review, then, allows you to locate your
own research within a tradition of research in an area.
Indeed, reading the literature is itself often an important
source of research questions.
• Bear in mind that you will want to return to much
of the literature that you examine in the discussion of
your findings and conclusion.
• Do not try to get everything you read into a literature
review. Trying to force everything you have read into
Tips and skills
Ways of conceptualizing a literature review
Bruce’s (1994) study of research students’ early experiences of the dissertation literature review identified six
qualitatively different ways in which the review process was experienced or understood by postgraduates.
The six conceptions included:
1. List. The literature review is understood as a list comprising pertinent items representing the literature of the
subject.
2. Search. The review is a process of identifying relevant information and the focus is on finding or looking, which
may involve going through sources (for example, article, database) to identify information.
3. Survey. Students also see the literature review as an investigation of past and present writing or research on
a subject; this investigation may be active (critical/analytical) or passive (descriptive).
4. Vehicle. The review is also seen as having an impact on the researcher, because it is seen as a vehicle for
learning that leads to an increase in his or her knowledge and understanding. Within this conception the
review acts as a sounding board through which the student can check ideas or test personal perceptions.
5. Facilitator. The literature review can be understood as directly related to the research that is about to be or
is being undertaken, the process helping the researcher to identify a topic, support a methodology, provide
a context, or change research direction. The review thus helps to shape the course of the student’s research.
6. Report. The review is understood as a written discussion of the literature, drawing on previously conducted
investigations. The focus is on ‘framing a written discourse about the literature which may be established as
a component part of a thesis or other research report’ (Bruce 1994: 223).
These six conceptions reflect the varying relationship between the student and the literature, the earlier ones
being more indirect—the student works with items that represent the primary literature, such as bibliographic
citations—and the latter conceptions being more direct—the student works with source material, rather than, for
example, a representative abstract. The conceptions can also be seen as cumulative, since a student who adopts
the facilitator conception may also continue to hold the conception of the literature review as a survey. Bruce
therefore recommends that students be encouraged to adopt the higher-level conceptions (3–6), because
through these the other ways of experiencing the literature review (1–3) become more meaningful.
Getting started: reviewing the literature100
your review (because of all the hard work involved in
uncovering and reading the material) is not going to
help you. The literature review must assist you in
developing an argument, and bringing in material of
dubious relevance may undermine your ability to get
your argument across.
• Bear in mind that reading the literature is not something
that you should stop doing once you begin
designing your research. You should continue your
search for and reading of relevant literature more
or less throughout your research. This means that, if
you have written a literature review before beginning
your data collection, you will need to regard it as
provisional. Indeed, you may want to make quite substantial
revisions of your review towards the end of
writing up your work.
• Do not just summarize all the literature you have read.
Quite aside from the fact that it is boring to read such
a summary, it does not tell the reader what you have
made of the literature and how it fits into your overall
research project. Try to use the literature to tell a story
about it. Some useful thoughts about how to develop
the literature in this way can be found in Thinking
deeply 5.1. The different ways of construing the literature
that are presented in this box are derived from a
review of qualitative studies of organizations, but the
approaches identified have a much broader applicability,
including quantitative research.
• The study by Holbrook et al. (2007) referred to in
Thinking deeply 5.2 contains some useful implications
from a study of Ph.D. examiners’ reports for conducting
a literature review. One of the most central
implications of it is to emphasize the importance of
having a comprehensive coverage of the literature.
While comprehensive coverage might be an expectation
for Ph.D. candidates, this may be more difficult to
achieve for undergraduate and postgraduate dissertations.
At the very least, it implies that making sure
that key references are included in the review is
essential.
Thinking deeply 5.1
Presenting literature in articles based on
qualitative research on organizations
Further useful advice on relating your own work to the literature can be gleaned from an examination of the
ways in which articles based on qualitative research on organizations are composed. In their examination of
such articles, Golden-Biddle and Locke (1993, 1997) argue that good articles in this area develop a story—that is,
a clear and compelling framework around which the writing is structured. This idea is very much in tune with
Wolcott’s (1990a: 18) recommendation to ‘determine the basic story you are going to tell’. Golden-Biddle and
Locke’s research suggests that the way the author’s position in relation to the literature is presented is an
important component of storytelling. They distinguish two processes in the ways that the literature is conveyed.
1. Constructing intertextual coherence. This refers to the way in which existing knowledge is represented and
organized; the author shows how contributions to the literature relate to each other and the research
reported. The techniques used are:
• Synthesized coherence puts together work that is generally considered unrelated; theory and research
previously regarded as unconnected are pieced together. There are two prominent forms:
i. very incompatible references (bits and pieces) are organized and brought together;
ii. connections are forged between established theories or research programmes.
• Progressive coherence portrays the building up of an area of knowledge around which there is considerable
consensus.
• Non-coherence recognizes that there have been many contributions to a certain research programme, but
that there is considerable disagreement among practitioners.
Each of these strategies is designed to leave room for a contribution to be made.
2. Problematizing the situation. The literature is then subverted by locating a problem. The following techniques
were identified:
Getting started: reviewing the literature 101
• Incomplete. The existing literature is not fully complete; there is a gap (see also Sandberg and Alxesson 2011).
• Inadequate. The existing literature on the phenomenon of interest has overlooked ways of looking at it that
can greatly improve our understanding of it; alternative perspectives or frameworks can then be
introduced.
• Incommensurate. This argues for an alternative perspective that is superior to the literature as it stands. It
differs from ‘inadequate problematization’ because it portrays the existing literature as ‘wrong, misguided,
or incorrect’ (Golden-Biddle and Locke 1997: 43).
The key point about Golden-Biddle and Locke’s account of the way the literature is construed in this field is
that it is used by writers to achieve a number of things.
• They demonstrate their competence by referring to prominent writings in the field (Gilbert 1977).
• They develop their version of the literature in such a way as to show and to lead up to the contribution
they will be making in the article.
• The gap or problem in the literature that is identified corresponds to the research questions.
The idea of writing up one’s research as storytelling acts as a useful reminder that reviewing the literature, which
is part of the story, should link seamlessly with the rest of the article and not be considered as a separate
element.
Thinking deeply 5.2
What do examiners look for in
a literature review?
Holbrook et al. (2007) conducted an analysis of examiners’ reports on Ph.D. theses. They analysed 1,310 reports
relating to 501 theses in Australia (a Ph.D. thesis is examined by at least two examiners). These reports are
naturally occurring documents, in that examiners have to provide these reports as part of the process of
examining a Ph.D. candidate. In the course of writing a report, examiners frequently if not invariably comment on
the literature review. While these findings are obviously specific to a Ph.D., the features that examiners look for
are also applicable in general terms to other kinds of writing, such as an undergraduate or a postgraduate
dissertation.
The reports were analysed using computer-assisted qualitative data analysis software, which will be covered in
Chapter 25. The analysis of these reports suggests that comments concerning the literature review were of three
basic kinds:
1. Comments about coverage of the literature. This was by far the most common type of comment and signals
whether the candidate has covered and made sense of a broad swathe of the literature.
2. Identification of errors. This type of comment relates to such things as references being omitted from the
bibliography, misreporting of references, and inconsistent presentation of referencing and quotations.
3. Comments about ‘use and application’ of the literature. Although this was the least common of the types of
comment made by examiners, it attracts the bulk of the attention of Holbrook et al. It is made up of a number
of subcategories of comment:
• the literature is used (or not used) to develop and sustain an argument;
• clear familiarity with the literature;
• the development of a critical assessment of the literature (the ability to ‘weigh up the literature and subject
it to critical appraisal, ideally to lead to a new or interesting perspective’ (Holbrook et al. 2007: 348));
• connecting the literature to findings;
• demonstrating an appreciation of the disciplinary context of the literature.
Getting started: reviewing the literature102
Most literature reviews take the form of narrative reviews
(see below for more on this notion). This means
that they seek to arrive at an overview of a field of study
through a reasonably comprehensive assessment and
critical reading of the literature. Such literature reviews
might occur as preludes to the presentation of some
empirical findings or they might be works in their own
right (for example, a dissertation or article based entirely
on a review of the literature in an area). While such
reviews continue to be the norm for most purposes when
reviewing the existing literature in an area, there has
been growing interest in a different approach to reviewing
the literature known as systematic review, which is
the focus of the next section.
Systematic review
In recent years, considerable thought has been lavished
on the notion of systematic review (see Key concept
5.1). This is an approach to reviewing the literature that
adopts explicit procedures. It has emerged as a focus of
interest for two main reasons. One is that it is sometimes
suggested (see, e.g., Tranfield et al. 2003) that many
reviews of the literature tend to ‘lack thoroughness’ and
reflect the biases of the researcher. Proponents of systematic
review suggest that adopting explicit procedures
makes such biases less likely to surface. Second, in fields
like medicine, there has been a growing movement
towards evidence-based solutions to illnesses and treatments.
Systematic reviews of the literature are often seen
as an accompaniment to evidence-based approaches, as
their goal is to provide advice for clinicians and practitioners
based on all available evidence. Such reviews are
deemed to be valuable for decision-makers, particularly
in areas where there is conflicting evidence concerning
treatments (as often occurs in the case of medicine).
The systematic review approach is beginning to diffuse
into other areas, like social policy, so that policy-makers
and others can draw on reviews that summarize the balance
of the evidence in certain areas of practice. Tranfield
and colleagues contrast systematic review with what
they describe as ‘traditional narrative reviews’ (the subject
of the following section). An example of systematic
review is given in Research in focus 5.1. However,
One of the main themes running through these latter remarks is that the student does not just summarize the
literature in a routine way, simply because he or she knows that a literature review has to be undertaken. Instead,
examiners look for evidence that the candidate uses the literature—to develop an argument, to connect with
his or her findings, or to develop a distinctive stance on the subject. However, undoubtedly, the thing that
disconcerts examiners most is evidence of poor coverage of the literature, as it signals a lack of engagement
with and full appreciation of the subject.
Key concept 5.1
What is a systematic review?
Systematic review has been defined as ‘a replicable, scientific and transparent process . . . that aims to minimize
bias through exhaustive literature searches of published and unpublished studies and by providing an audit trail
of the reviewer’s decisions, procedures and conclusions’ (Tranfield et al. 2003: 209). Such a review is often
contrasted with the traditional narrative review, which is the focus of the next section. The proponents of
systematic review are more likely to generate unbiased and comprehensive accounts of the literature, especially
in fields in which the aim is to understand whether a particular intervention has particular benefits, than those
using the traditional review, which is often depicted by them as haphazard. A systematic review that includes
only quantitative studies is a meta-analysis (see Key concept 5.2). In recent times, the development of
systematic review procedures for qualitative studies has attracted a great deal of attention, especially in the social
sciences. Meta-ethnography (see Key concept 5.3) is one such approach to the synthesis of qualitative findings,
but currently there are several different methods, none of which is in widespread use (Mays et al. 2005).
Getting started: reviewing the literature 103
advocates of systematic review acknowledge that, unlike
medical science, where systematic reviews are commonplace
and often highly regarded, social scientific fields
are often characterized by low consensus concerning key
research questions, because of the different theoretical
approaches. Moreover, medical science is often concerned
with research questions to do with answers to the
question ‘What works?’ Such questions are fairly well
suited to systematic review in fields like social policy, but
are less often encountered in other social science fields
like sociology.
Nonetheless, systematic review has attracted a great
deal of attention in recent years, so it is worth exploring
some of its main steps. Accounts of the systematic review
process vary slightly, but they tend to comprise the following
steps in roughly the following order.
1. Define the purpose and scope of the review. The review
needs an explicit statement of the purpose of the
review (often in the form of a research question) so
that decisions about key issues such as what kinds of
research need to be searched for and what kinds of
samples the research should relate to can be made in
a consistent way. It is often argued that, for a systematic
review, the researcher and his or her team should
assemble a panel to advise them on the precise formulation
of the research issue(s) to be examined and
also to assist with suggestions for keywords for Step 2
(below).
2. Seek out studies relevant to the scope and purpose of the
review. The reviewer should seek out studies relevant
to the research question(s). The search will be based
on keywords and terms relevant to the purpose defined
in Step 1. The search strategy must be described
in terms that allow it to be replicated. The reviewer
has to consider which kinds of publication outlets
should be incorporated. It is tempting to search for
research published only in articles in peer-reviewed
journals, because they are relatively easy to find using
databases like the Social Sciences Citation Index
(SSCI, about which more will be said below) using
keywords. However, to rely solely on peer-reviewed
journal articles would imply omitting other sources
of evidence, most notably, studies reported in books,
in articles in non-peer-reviewed journals, and in what
is often referred to as ‘grey literature’ (for example,
conference papers and reports by various bodies).
3. Appraise the studies from Step 2. The reviewer might
want to restrict the review to studies published only in
a particular time period or to studies that derive from
one region or nation rather than another. Another
criterion might be the kind of research design or
research method used. In some fields, like medicine,
there is an unequivocal hierarchy of research approaches
that are relevant to the ‘What works?’ question.
This means that only articles that entail a true
experimental design—often called randomized controlled
trials or RCTs—will be included, as only research
based on such designs generates unambiguous
findings about cause and effect. However, in most of
the social sciences there is far less consensus about
what is the appropriate approach to research. Based
on the strict application of the inclusion criteria formulated,
the appraisal process will lead to the production
of a list of all the published outputs on which
the review will be based. Initially, searches at Step 2
will produce a vast number of possible candidates for
inclusion in the review based on the keywords and
hand searching through various possible publication
outlets. These studies will be gradually whittled down
as the research items are examined for their degree of
fit with the research question(s) and with the quality
criteria employed by the researchers. This stage
necessitates a specification of quality criteria. This is
likely to entail criteria such as whether an appropriate
research design and research methods were used
and whether the chosen research design and research
methods were implemented according to the standards
of good research practice for those research design
and research methods. At the same time, the appropriateness
of the study for the research question(s)
will be assessed.
4. Analyse each study and synthesize the results. A formal
protocol should be used to record features like: date
when the research was conducted; location; sample
size; data-collection methods; and the main findings.
A synthesis of the results then has to be produced.
If the findings of a group of studies are quantitative
in character, a meta-analysis will probably be conducted.
This phase will involve producing summary
statistics from the quantitative data supplied with
each study. In the case of other kinds of systematic
review, such as those based on qualitative research or
where there is a combination of both quantitative and
qualitative studies, the results will often be presented
in a report in the form of summary tables and a narrative
that brings together the key findings. Denyer and
Tranfield (2009) propose that the review document
should be structured much like a research report in
which the purpose of the review, its methods, its findings,
the discussion of the findings, and a conclusion
are clearly specified.
Getting started: reviewing the literature104
Research in focus 5.1
Healthy eating among young people
Shepherd et al. (2006) have published an account of the procedures they used to examine the barriers to healthy
eating among young people aged 11–16 years and the factors that facilitate healthy eating. In Table 5.1 I have
outlined the chief steps in doing a systematic review, as outlined in the main text, and the corresponding
procedures and practices in the review by Shepherd et al. These authors used methods for systematic review that
have been developed by the Evidence for Policy and Practice Information and Coordinating Centre (EPPI) at the
Institute of Education, University of London. The EPPI has a very comprehensive website that details its approach
and its main methods and provides full reports of many of the systematic reviews its members have conducted
(http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=53&language=en-US (accessed 2 August 2010)).
One of the features that is especially noteworthy concerning the summary in Table 5.1 is that intervention
studies (for example, training parents in nutrition and evaluating the outcomes of such an intervention) and
non-intervention studies (for example, a cohort or an interview study) were separated out for the purposes of
presenting a summary account of the findings and appraising the quality of the studies, although a final matrix
was formed that synthesized the key elements across both types of study. Assessing the quality of studies is
an important component of a systematic review, so that only reliable evidence forms the basis for such things
as policy changes. Different quality criteria were employed for the two types of study. In the case of the
non-intervention studies, the following seven criteria were used:
(i) an explicit theoretical framework and/or literature review;
(ii) clear statement of aims and objectives of the research;
(iii) clear account of the context within which the research was conducted;
(iv) clear account of the nature of the sample and how it was formed;
(v) clear description of methods of data collection and analysis;
(vi) ‘analysis of the data by more than one researcher’ (Shepherd et al. 2006: 242); and
(vii) whether sufficient information was provided to allow the reader to see how the conclusions were derived
from the data.
The application of the corresponding criteria for the intervention studies resulted in just 7 of the studies being
viewed as methodologically sound. None of the 8 non-intervention studies were methodologically sound in
terms of all seven of the above appraisal criteria, although 4 met six of the seven criteria and a further 2 met five
of the seven criteria. Of the 8 non-intervention studies, 5 used a self-completion questionnaire to generate data,
2 used focus groups, and 1 used interviews. Thus, the category ‘non-intervention study’ includes research
methods associated with both quantitative and qualitative research. It is quite common for systematic reviews to
end up being based on quite small numbers of studies, because the explicit criteria for inclusion coupled with
the quality criteria represent standards that very few investigations can meet. When presenting their synthesis of
their review findings, the authors separated the findings of the 7 methodologically sound intervention studies
from those pertaining to the 15 other intervention studies. Regarding the findings of the non-intervention studies,
the authors report that several barriers to and facilitators of healthy eating were identified. For example, they
write: ‘Facilitating factors included information about nutritional content of foods/better labeling, parents and
family members being supportive; healthy eating to improve or maintain one’s personal appearance, will-power
and better availability/lower pricing of healthy snacks’ (Shepherd et al. 2006: 255). The authors linked such
findings with intervention studies arguing that ‘juxtaposing barriers and facilitators alongside effectiveness
studies allowed us to examine the extent to which the needs of young people had been adequately addressed by
evaluated interventions’ (Shepherd et al. 2006: 255).
Getting started: reviewing the literature 105
Tranfield et al. (2003) suggest that the systematic
review process provides a more reliable foundation on
which to design research, because it is based on a more
comprehensive understanding of what we know about a
subject. It is therefore likely to be relevant to researchers
as a way of summarizing findings, so that it is not
just practitioners who benefit from systematic reviews.
Proponents of systematic review also recommend the
approach for its transparency; in other words, the
grounds on which studies were selected and how they
were analysed are clearly articulated and are potentially
replicable. It has sometimes been suggested that not
all areas of literature lend themselves to a systematic
review approach, because they are not always concerned
with research questions to do with exploring whether
a certain independent variable has certain kinds of
effects. Meta-analysis of quantitative studies requires
this kind of research question, but qualitative studies and
indeed some sorts of quantitative investigation are not
necessarily in this format. This impression may have
been created because many early systematic reviews
were of the ‘what works?’ or ‘does X work?’ kind, where
the literature relating to various kinds of intervention
would be appraised and reviewed. In more recent years,
a wider range of research questions have come within
the purview of systematic review, as it has begun to
include both qualitative studies and quantitative nonintervention
studies.
Table 5.1
Steps in systematic review in connection with a systematic review of barriers to,
and facilitators of, healthy eating among young people (Shepherd et al. 2006)
Steps in systematic review Corresponding practices in Shepherd et al. (2006)
1. Define the purpose and
scope of the review
A. Review question: ‘What is known about the barriers to, and facilitators of, healthy eating
among young people?’ (Shepherd et al. 2006: 243).
2. Seek out studies relevant
to the scope and purpose of
the review
B. The authors employed a combination of terms to do with healthy eating (e.g. nutrition) and
terms to do with health promotion or with the causes of health or ill-health (e.g. at-riskpopulations)
and with terms indicative of young people (e.g. teenager). In addition to being
about ‘the barriers to, and facilitators of, healthy eating among young people’, the review had
to be: either an outcome evaluation (usually to evaluate the outcome of an intervention) or
a non-intervention study (e.g. an interview study) in the UK, in English. Further, guidelines
were formulated separately for these two types of study. In the case of non-intervention studies,
it had to: be about attitudes, views, experiences, etc. of healthy eating; provide insights into
respondents’ own definitions of healthy eating and factors affecting it; and ‘privilege young
people’s views’ (Shepherd et al. 2006: 241). Several online bibliographical databases were
searched (including SSCI and PsycINFO). Lists of references and other sources were also searched.
An initial 7,048 references were gradually trimmed to 135 reports (relating to 116 studies). Of the
116 studies, 75 were intervention studies, 32 were non-intervention studies, and 9 were prior
systematic reviews. Application of the full set of inclusion criteria resulted in just 22 outcome
evaluations and 8 non-intervention studies meeting the criteria for what the authors refer to as
‘in-depth systematic review’ (Shepherd et al. 2006: 242).
3. Appraise the studies from
Step 2
C. ‘Data for each study were entered independently by two researchers into a specialized
computer database’ (Shepherd et al. 2006: 241). In doing so, the reviewers sought to summarize
the findings from each study and appraise its methodological quality. Separate quality criteria
were employed for intervention and non-intervention studies. The application of eight criteria for
the intervention studies resulted in just 7 being regarded as ‘methodologically sound’ and the
results of just these 7 studies are the focus of the authors’ summary.
4. Analyse each study and
synthesize the results
D. Separate syntheses were conducted for the two types of study and a third synthesis for the
intervention and non-intervention studies jointly. The authors write of this third synthesis: ‘a
matrix was constructed which laid out the barriers and facilitators identified by young people [in
the non-intervention studies] alongside descriptions of the interventions included in the in-depth
systematic review of outcome evaluations. The matrix was stratified by four analytical themes to
characterize the levels at which the barriers and facilitators appeared to be operating: the school,
family and friends, the self and practical and material resources’ (Shepherd et al. 2006: 241). In
forming the matrix, one column summarized barriers and facilitators identified in the nonintervention
studies and there were further separate columns for the 7 ‘soundly evaluated
interventions’ and the 15 ‘other evaluated interventions’.
Getting started: reviewing the literature106
Key concept 5.2
What is meta-analysis?
Meta-analysis involves summarizing the results of a large number of quantitative studies and conducting various
analytical tests to show whether or not a particular variable has an effect. This provides a means whereby the
results of large numbers of quantitative studies of a particular topic can be summarized and compared. The aim
of this approach is to establish whether or not a particular variable has a certain effect by comparing the results
of different studies. Meta-analysis thus involves pooling the results from various studies in order to estimate
an overall effect by correcting the various sampling and non-sampling errors that may arise in relation to
a particular study. In a sense, a meta-analysis lies between two kinds of activity covered in this book: doing
a literature review of existing studies in an area in which you are interested (the focus of this chapter), and
conducting a secondary analysis of other researchers’ data (see the section on ‘Other researchers’ data’ in
Chapter 14). However, the technique relies on all the relevant information being available for each of the studies
examined. Since not all the same information relating to methods of study and sample size is included in
published papers, meta-analysis is not always feasible. Meta-analysis is vulnerable to what is known as the ‘file
drawer problem’. This occurs when a researcher conducts a study, finds that the independent variable does not
have the intended effect, but has difficulty publishing his or her findings. As a result, it is often suggested that the
findings are simply filed away in a drawer. If the file drawer problem has occurred in a field of research, the
findings of a meta-analysis will be biased in favour of the independent variable being found to have a certain
effect, as some of the findings that contradict that effect will not be in the public domain.
Research in focus 5.2
A meta-analysis of the impact of leadership
interventions
A meta-analysis conducted by Avolio et al. (2009) sought to examine the research question: ‘do leadership
interventions have the intended impact and if so to what degree?’ This is a significant question, given the
attention that is often lavished on the concept of leadership and the amounts of money spent on training
leaders to exhibit certain kinds of behaviour. Avolio et al. wanted to include in their review all experimental and
quasi-experimental studies of leadership interventions. This meant that all cross-sectional design studies that
addressed leadership and leadership interventions did not qualify, as they do not involve an intervention in which
there is a manipulation of the independent variable. The authors’ search for a comprehensive set of studies
involved the following procedures:
1. Searching eighteen electronic databases using 124 keywords and phrases.
2. An examination of the bibliographies of all studies produced through the electronic database searchers and
an examination of the bibliographies of previous meta-analyses in the field.
3. Emails to 670 leadership researchers asking them to review a proposed list of studies.
4. A manual search of leadership handbooks and other books.
This search process yielded over 500 studies, which were gradually trimmed down to 200 studies that met the
authors’ criteria. The main reason for exclusion was that the research was not an intervention study. Interestingly,
of the 200 studies, 16 per cent were unpublished, suggesting that meta-analyses and other kinds of review that
are based exclusively on published research may be missing a significant number of studies, and this may be a
source of bias. The research by Howell and Frost (1989) that is referred to in Research in Focus 3.4 is one of the
200 included studies. The authors found a strong relationship between leadership interventions and various kinds
of outcomes (such as task performance, as in Research in focus 3.4). In other words, leadership interventions do
have a significant impact on various kinds of dependent variable.
Getting started: reviewing the literature 107
Key concept 5.3
What is meta-ethnography?
Meta-ethnography is a method that is used to achieve interpretative synthesis of qualitative research and other
secondary sources, thus providing a counterpart to meta-analysis in quantitative research (Noblit and Hare 1988).
It can be used to synthesize and analyse information about a phenomenon that has been extensively studied,
such as lay experiences of diabetes (see Research in focus 5.3). However, this is where the similarity ends,
because meta-ethnography ‘refers not to developing overarching generalizations but, rather, translations of
qualitative studies into one another’ (Noblit and Hare 1988: 25). Noblit and Hare base their approach on the
idea that all social science explanation is comparative, involving the researcher in a process of translating
existing studies into his or her own worldview, and through this he or she creates a reading of other people’s
readings about a subject. Meta-ethnography involves a series of seven phases that overlap and repeat as the
synthesis progresses.
1. Getting started. This involves the researcher in identifying an intellectual interest that the qualitative research
might inform by reading interpretative accounts.
2. Deciding what is relevant to the initial interest. Unlike positivists, interpretative researchers are not concerned
with developing an exhaustive list of studies that might be included in the review. Instead the primary intent
is to determine what accounts are likely to be credible and interesting to the intended audience for the synthesis.
3. Reading the studies. This involves the detailed, repeated reading of the studies, rather than moving to analysis
of their characteristics.
4. Determining how the studies are related. This stage entails ‘putting together’ the various studies by
determining the relationships between them and the metaphors used within them.
5. Translating the studies into one another. This phase is concerned with interpreting the meaning of studies in
relation to each other: are they directly comparable or ‘reciprocal’ translations (so that the concepts used by
each study are translated one-by-one into concepts used by the others); do they stand in opposition to each
other as ‘refutational’ translations; or do they, taken together, represent a line of argument that is neither
‘reciprocal’ nor ‘refutational’?
6. Synthesizing translations. The researcher compares the different translations and shows how they relate to
each other. This may involve grouping them into different types.
7. Expressing the synthesis. This involves translating the synthesis into a form that can be comprehended by the
audience for which it is intended.
Crucial to understanding this approach is that the synthesis is focused primarily on the interpretations and
explanations offered by studies that are included, rather than on the data that these studies are based on.
Meta-ethnography thus translates the interpretations of one study into the interpretations of another one.
Research in focus 5.3
A meta-ethnography of lay experiences
of diabetes
Campbell et al. (2003) report their approach to conducting a meta-ethnography of studies within the medical
sociology field of lay experiences of diabetes and diabetes care. A search came up with ten articles based on
qualitative research that addressed this area. Three were excluded for quite different reasons: one turned out
not to be based on qualitative research; the evidence in another was appraised as being too weak to warrant
inclusion; and the findings of the third paper turned out to be in one of the seven papers that would be included.
The seven papers could be grouped into three ‘clusters’: response to diabetes and treatment; how patients and
Getting started: reviewing the literature108
However, one of the limitations of systematic review
stems from situations where research questions are not
capable of being defined in terms of the effect of a particular
variable, or when the subject boundaries are more
fluid and open or subject to change. This is often the case
in many areas of social research. Another criticism of the
approach is that it can lead to a bureaucratization of the
process of reviewing the literature, because it is more
concerned with the technical aspects of how it is done
than with the analytical interpretations generated by it.
A third potential limitation of the approach relates to its
application to qualitative research studies and in particular
to the methodological judgements that inform decisions
about quality and so determine the inclusion or exclusion
of an article from a literature review. These stem from
differences between qualitative and quantitative research
in relation to the criteria used to assess their
methodological quality (see Chapters 7 and 17). The systematic
approach assumes that an objective judgement
about the quality of an article can be made. Particularly
in relation to qualitative research, there is little consensus
on how the quality of studies should be carried out,
an issue that will be returned to in Chapter 17. Moreover,
some researchers would say that they measure the
quality of published research in terms of what they find
interesting—this may or may not include empirical study,
but such a view is not compatible with the systematic
approach, which requires articles to be evaluated in terms
practitioners differ in perceptions of the disease; and the connections between beliefs about the causes of
diabetes and how they managed the disease. One of the themes to emerge among the four articles in the first of
these three clusters was the link between control and ‘strategic cheating’. Campbell et al. note that one study
noted the significance of people’s sense of control of the disease, which they accomplished through managing it
strategically. Such people are referred to as ‘copers’. Another study made a similar point between those who felt
they were in control of their diet and those described as ‘buffeted’ by it. Their stance on this issue affected their
perception of diabetes, with the former group having a less negative image of it. Some people were able to
manage their diet strategically in a flexible way, which was sometimes perceived as ‘cheating without guilt’.
These reflections were then linked to findings across the two other studies in this group. The authors write:
Looking across these four studies it would seem that strategic cheating, departing from medical advice in
a thoughtful and intelligent way, in order to achieve a balance between the demands of diabetes and the way
the person wants to live their life, was associated with a feeling of confidence, less guilt, acceptance of the
diabetes and improved glucose levels. (Campbell et al. 2003: 678)
In addition, six concepts were found from the seven studies to be significant for the diabetes sufferers in terms of
helping them to achieve a balance between controlling the disease and also having some control over their
lives—for example, the need to adopt a less subservient approach to medical practitioners. Interestingly, the
authors were able to derive insights from their meta-ethnography that were not present in any of the articles.
Tips and skills
Using systematic review in a student
research project
The systematic review approach does contain some elements that cannot easily be applied in a student research
project because of limitations of time and resources. For example, you are unlikely to be able to assemble a panel
of experts in methodology and theory to meet you regularly and discuss the boundaries of the review. However,
there are some aspects of the approach that can be applied to students’ research. For example, meeting your
supervisor regularly during the planning stage of your literature review to define the boundaries of the subject
and to come up with likely search terms is extremely useful. Your supervisor’s knowledge of the subject can be
invaluable at this stage. Also, a systematic review approach to the literature requires a transparent way of
searching for and examining the literature as well as keeping records of what you have done. These practices
are feasible for a student research project.
Getting started: reviewing the literature 109
of methodological criteria. In addition, researchers in the
medical sciences have found that the process of identifying
relevant qualitative studies is more time consuming
and cannot be done on the basis of the abstract or summary
in the way that quantitative research studies can
(M. L. Jones 2004). Finally, whether or not the systematic
review approach makes sense to you depends somewhat
on your epistemological position (see Chapter 2).
As Noblit and Hare (1988: 15) state: ‘Positivists have had
more interest in knowledge synthesis than interpretivists.
For them, knowledge accumulates. The problem has
been how best to accomplish that accumulation.’ For
these reasons, researchers who adopt an interpretative
approach to understanding the social sciences and use
qualitative methods may find the systematic review
approach more problematic. Similar concerns have been
expressed by educational researchers about the suitability
of systematic review in an area of study that is quite
different from the medical field where it was developed
(see Thinking deeply 5.3).
Thinking deeply 5.3
Debates about the role of systematic review in
education research
Debates about the role of systematic review in education research are of potential relevance to social policy
researchers because of the similarities shared between these two applied fields of study. Both education and
social policy research draw on a range of social science disciplines, involve the study of practitioners, and are
sometimes criticized for not focusing sufficiently on the concerns of practitioners and policy-makers. Evans and
Benefield (2001) have argued that the medical model of systematic review can be adapted for application in
education research. This would enable researchers to ‘say something more precise and targeted’ about the
effectiveness of specific interventions, or in other words to provide evidence about ‘what works’ (Evans and
Benefield 2001: 538). Systematic reviews would thus help to make research evidence more usable.
However, Hammersley (2001) criticizes the assumption in systematic review about the superiority of the positivist
model of research, which is expressed through the methodological criteria applied in evaluating the validity of
studies (experiments being more highly valued), and through the explicit procedures used to produce reviews
that are intended to be ‘objective’. This ‘takes little or no account of the considerable amount of criticism that has
been made of that model since at least the middle of the twentieth century’ (Hammersley 2001: 545). Moreover,
Hammersley suggests that the dichotomy portrayed between rational rule-following systematic review and
irrational judgement narrative review is overstated, because even the simplest rule-following involves an element
of interpretation. He concludes:
What all this means, I suggest, is that producing a review of the literature is a distinctive task in its own right.
It is not a matter of ‘synthesising data’; or, at least, there is no reason why we should assume that reviewing
must take this form. Rather, it can involve judging the validity of the findings and conclusions of particular
studies, and thinking about how these relate to one another, and how their interrelations can be used to
illuminate the field under investigation. This will require the reviewer to draw on his or her tacit knowledge,
derived from experience, and to think about the substantive and methodological issues, not just to apply
replicable procedures. (Hammersley 2001: 549)
Pearson and Coomber (2009) provide some evidence that supports Hammersley’s contention that systematic
review necessarily entails an element of interpretation. They report the results of a participant observation study
of a systematic review process. The domain with which the reviewers were concerned was the development of
guidance in connection with substance misuse. Pearson and Coomber found that the reviewers prioritized
internal validity over external validity considerations in selecting studies for inclusion. Also, the reviewers elected
to play down the significance of one kind of intervention—life skills training—because a report was made
available to them that provided a strong critique of it. However, Pearson and Coomber note that an examination
of the summaries of research on life skills training generated by the reviewers suggests there was a good case for
including it in the guidance on treatment. Thus, a report that had not been selected through the systematic
review process seems to have been instrumental in the lack of attention given to life skills training, implying
a degree of subjectivity to the review process.
Getting started: reviewing the literature110
Narrative review
Rather than reviewing the literature to find out what
their research project can add to existing knowledge
about a subject, interpretative researchers (see Chapter 2
for an explanation of interpretivism) can have quite different
reasons for reviewing the literature on a particular
subject, since their purpose is to enrich human discourse
(Geertz 1973a) by generating understanding rather than
by accumulating knowledge. The literature review is for
them a means of gaining an initial impression of the topic
area that they intend to understand through their research.
The process of reviewing the literature is thus a
more uncertain process of discovery, in that you might
not always know in advance where it will take you!
Narrative reviews therefore tend to be less focused and
more wide-ranging in scope than systematic reviews.
They are also invariably less explicit about the criteria for
exclusion or inclusion of studies. An example of a narrative
review is given in Research in focus 5.4.
MacLure (2005: 409) suggests that the prioritization of systematic review in education research is worrying
because ‘it is hostile to anything that cannot be seen, and therefore controlled, counted and quality assured’;
it thus degrades the status of reading, writing, thinking, and interpreting as activities that are crucial to the
development of analysis and argument. Although systematic review has so far not been as widely adopted in
social research, the concerns expressed by education researchers are of potential relevance, particularly to
qualitative researchers. However, one of the most interesting aspects of Hammersley’s (2001) critique is that he
implies that systematic review is inconsistent with its own principles in that there appears to be no or very little
evidence that systematic reviews lead to better evidence (and therefore presumably to better evidence-based
practice)!
Research in focus 5.4
A narrative review of qualitative research
on leadership
Some years ago, I conducted a literature review of qualitative research that had been undertaken on leadership
(Bryman 2004b). Leadership research is a field that has been dominated over the years by quantitative
investigations, so it struck me as potentially interesting to examine the growing number of qualitative studies that
were appearing. I decided to examine articles that had appeared in journals that publish only articles that have
been reviewed by peers. There were two main reasons for this: peer-reviewed articles can be searched relatively
easily through online databases like the SSCI, and peer review offers an element of quality control, since only
articles that have gone through the process of peer review are accepted for publication. Peer review weeds out
articles that are not of sufficient quality for a journal either by rejecting them outright or by insisting that authors
implement substantial revisions in response to referees’ concerns. In addition, I included in my review articles
that I already knew and I hand searched The Leadership Quarterly, one of the main outlets for research articles.
I also examined the bibliographies of some articles for further candidates for inclusion. This general area was of
interest to me as I have long been interested in both qualitative research and the field of leadership. I did not
have a specific focus to my review, although I was interested in general terms in the question of how similar
qualitative research was to the quantitative research that dominated the field of leadership.
I presented my main findings in a table that outlined: the year of publication; the sector in which the research
was conducted; the research design; the research methods used; the nature of the key findings; and the kinds of
leadership style and leader behaviour that were emphasized in each study. In the subtitle of my article, I called
it a ‘critical but appreciative review’. It was critical in that it pointed to some overall deficiencies in qualitative
research on leadership but it was also appreciative, because I pointed to some of the distinctive contributions
that qualitative research has made to the field. The chief flaw with my review is that, by focusing on published
research, my conclusions may have been influenced by the file drawer problem.
Getting started: reviewing the literature 111
If your approach to the relationship between theory
and research is inductive rather than deductive (see
Chapter 2), setting out all the main theoretical and conceptual
terms that define your area of study prior to data
collection is extremely problematic, because theory is
the outcome of the study, rather than the basis for it.
Hence, in the process of researching a topic, researchers
may discover issues that they did not previously anticipate
as likely to be important to their area of study. As a
result, they become aware of the limitations of the topic
area that they originally intended to inform, and this can
lead them towards an unanticipated understanding of it
(Noblit and Hare 1988). Interpretative researchers are
thus more likely than deductive researchers to change
their view of the theory or literature as a result of the
analysis of collected data and so they require greater flexibility
to modify the boundaries of their subject of study
as they go along. This means that narrative review may
be more suitable for qualitative researchers whose research
strategy is based on an interpretative epistemology, and
for them systematic review should not be automatically
accepted as a better way of dealing with the literature.
Most reviews are of the narrative kind, regardless
of whether they are meant to be springboards for the
reviewer’s own investigation (for example, when the
literature is reviewed as a means of specifying what is
already known in connection with a research topic, so
that research questions can be identified that the reviewer
will then examine) or are ends in their own right
(as a means of summarizing what is known in an area).
When we examine some examples of writing up research
in Chapter 29, we will see that the literature relevant to
the researcher’s area of interest is always reviewed as a
means of establishing why the researcher conducted the
research and what its contribution is likely to be. Such
reviews are still mainly narrative reviews. Compared to
systematic reviews, narrative reviews can appear rather
haphazard (thus making them difficult to reproduce), of
questionable comprehensiveness, and lacking in discrimination
in terms of the kind of evidence used, though
such a view is by no means always held (see Thinking
deeply 5.3). It may be that this accounts for the growing
incorporation of procedures associated with systematic
reviews into narrative reviews (see Thinking deeply 5.4).
Thinking deeply 5.4
Incorporating systematic review practices into
narrative reviews
It is always risky to speculate, but I have a hunch that some narrative reviews will incorporate some of the
practices associated with systematic review. Even though some writers like those mentioned in Thinking deeply
5.3 object to systematic review for its tendency towards a mechanical approach to reviewing the literature,
it could be that some reviewers will be attracted to its emphasis on such features as transparency about
how searches were conducted and/or comprehensiveness in the literature search. This is especially likely to
be the case when reviewers work on their own, as systematic review requires more than one person to assist
in such steps as: the formulation of research questions, the selection of keywords, and the assessment of
quality.
I tried to incorporate some systematic review practices into a narrative literature review I carried out on
leadership effectiveness at departmental level in higher education (Bryman 2007c). The systematic review
practices were apparent in:
• use of an explicit research question to guide the review. The question was: ‘’What styles of or approaches to
leadership are associated with effective leadership in higher education?’ (Bryman 2007c: 693).
• the specification of the literature search procedures so that they were reproducible, the combination of key
terms for searching for the literature in more than one online database (SSCI, Educational Resources
Information Center, Google Scholar, and others) and hand searching through the bibliographies of numerous
key articles. The terms used were: leader* or manage* or administrat* plus higher education* or university*
or academic plus effective* (the asterisks are ‘wild cards’ so that ‘leader*’ will pick up ‘leader’, ‘leaders’,
‘leading’, and ‘leadership’).
Getting started: reviewing the literature112
• the use of quality appraisal criteria to decide which articles should be within the review’s scope. The quality
appraisal criteria were: ‘the aims of the research were clearly stated; they made clear the ways in which data
were collected (sampling, research instruments used, how data were analysed), did so in a systematic way,
and indicated how the methods were related to the aims; provided sufficient data to support interpretations;
and outlined the method of analysis’ (Bryman 2007c: 695). From many hundreds of ‘hits’, only twenty articles
both related to the research question and met the appraisal criteria.
• the display of the leadership styles associated with leadership effectiveness in a table with an indication of
which articles they had been identified in.
This review did not conform to systematic review procedures in several ways, such as the fact that the literature
reviewed comprised almost exclusively peer-reviewed articles in academic journals, so that ‘grey literature’ was
not included, as it was in the meta-analysis reported in Research in focus 5.2. The vast majority of articles found
in the searches were not included in the review because they did not relate to the research question rather than
because they failed to meet the quality criteria. This was probably because most of the articles identified through
the online bibliographic and hand searches were articles published in journals that peer-review articles prior to
publication, so that these studies had already gone through a quality appraisal process.
Tips and skills
Reasons for writing a literature review
The following is a list of reasons for writing a literature review.
• You need to know what is already known in connection with your research area, because you do not want to
be accused of reinventing the wheel.
• You can learn from other researchers’ mistakes and avoid making the same ones.
• You can learn about different theoretical and methodological approaches to your research area.
• It may help you to develop an analytic framework.
• It may lead you to consider the inclusion of variables in your research that you might not otherwise have
thought about.
• It may suggest further research questions for you.
• It will help with the interpretation of your findings.
• It gives you some pegs on which to hang your findings.
• It is expected!
Student experience
Importance of doing a literature review
Lily Taylor does not appear to need convincing about the necessity of doing a literature review. As she notes:
Looking at significant work that related to mine was good in the sense that it enabled me to look at the use of
methodology and access key concepts and characteristics of the work.
For several of the students, the literature in their chosen area had an influence on their research questions.
For example, Alice Palmer writes about her dissertation research on the changing role of the modern housewife:
Getting started: reviewing the literature 113
Usually, students will have in mind a few initial references
when they begin on a project. These will probably
come from recommended reading in course modules,
or from textbooks. The bibliographies provided at the
end of textbook chapters or articles will usually provide
you with a raft of further relevant references that can
also be followed up. A literature search relies on careful
reading of books, journals, and reports in the first
instance. After identifying a few keywords that help to
define the boundaries of your chosen area of research
(see below), electronic databases of published literature
can be searched for previously published work in
the field.
Electronic databases
Online bibliographical databases accessible on the
Internet are an invaluable source of journal references.
An increasing number of these will also provide access
to the full text of an article in electronic format—these
are usually referred to as e-journals. You will need to
find out whether your institution can give you a user
name and password to gain access to these databases,
so look on your library’s homepage, or ask a member of
library staff.
Probably the single most useful source for the social
sciences is the Social Sciences Citation Index (SSCI),
which fully indexes over 1,700 major social science journals
covering all social science disciplines dating back
to 1970. To gain access to this website, most UK users
will need an Athens username and password. It can be
accessed from the ISI Web of Knowledge (WoK) home
page at the following address:
http://wok.mimas.ac.uk/ (accessed 3 August 2010).
The Citation indexes collectively are also known as Web
of Science.
The SSCI database provides references and abstracts,
and some libraries add full-text links for articles from
some of the most important social science journals published
worldwide. It is therefore very useful as an initial
source in your literature search, because, if you search
the database effectively, you can be relatively confident
that you have covered the majority of recent academic
journals that may have published articles on your topic
of interest. Here are some introductory guidelines for
searching SSCI.
• Select the ISI Web of Knowledge Service.
• Select the Web of Science tab
• Choose Social Science Citation Index by unticking the
other indexes below Citation Databases:
• You can then search by Topic and/or by Author by
entering the appropriate terms or names into the
appropriate boxes below Search for:
• Click on Search. Note that the default is to search
1970 to the current date; you can change this by using
the pull down menus below Timespan:
A feature of SSCI is its complete coverage of journal
contents, so, in addition to research and scholarly articles,
it also contains book reviews and editorial material,
which invariably can be identified through keyword
Lots of reading to identify gaps in previous research was the most important way of formulating research
questions. However, it is also important to follow ‘gut feelings’ about what needs investigating, even if it has
been done before, because things could have changed over time.
Amy Knight wrote in connection with her project on recycling and gender differences:
I completed extensive reading focusing on the topics of recycling and gender differences. In previous studies
gender differences regarding levels of environmental concern tended to be similar (that females demonstrated
higher levels of environmental concern than men). However, previous published research was inconclusive
regarding recycling habits and gender differences. I was interested to see whether levels of environmental
concern could also link to recycling habits hence the two research questions.
To read more about Lily’s, Alice’s, and Amy’s research experiences, go to the Online Resource Centre that
accompanies this book at: www.oxfordtextbooks.co.uk/orc/brymansrm4e/
Searching the existing literature
Getting started: reviewing the literature114
searches. You will need to experiment with the use
of keywords, because this is usually the way in which
databases like these are searched, though author searches
are also possible. Finally a feature that is often useful is
the ‘Times cited’ link. If you find an article that is relevant
to your dissertation, then you can click to see which other
articles have cited it. This does two things. First it allows
you to see how an article has been used in more recent
research, and in particular whether it has been challenged.
Second, it gives an impression of whether the
article and the ideas in it have been developed with new
data. For example, at the time of writing this chapter
(17 January 2011), my paper published in 1999 in the
journal Sociological Review on the Disneyization of society
has been cited in twenty-nine other papers about
related subjects, such as emotional labour and retailing.
However, it is important to realize that articles in other
journals may have cited the article. The reason that these
would not turn up in an SSCI search is that those responsible
for it operate a screening process, which means that
by no means all journals achieve entry into the database.
The screening process takes into account the reputation
and impact of the journal concerned.
You can also use the Cited Reference Search to search
for articles that cite an article that you know about
already. This can help you find other related research
and also see what other authors thought of your original
article. This is particularly useful if your article is a few
years old.
Also very useful is Scopus, which is available at:
www.scopus.com/scopus/home.url (accessed 3 August
2010).
Scopus describes itself as ‘the largest abstract and
citation database of research literature and quality web
sources’. You will need an Athens or other username and
password to get into the database. The ‘General Search’
may meet your initial needs. This allows you to search in
terms of keywords and/or authors. You need to specify
the date range of articles you wish to search for (it goes
back to 1960) and to untick the subject areas not relevant
to your search. Scopus tends to include a wider range
of journals than SSCI. Like SSCI, it will bring up the
abstract, as well as the full reference when a particular
item is selected for further examination.
Also useful for searching for references is Google
Scholar—see Tips and skills ‘Using information on the
Web’ for details of how to use this search tool.
Nowadays, many academic publishers have begun
to offer full-text versions of articles in their journals
Student experience
Strategies for finding references
The students who supplied information concerning their strategies for doing their literature reviews used a
variety of approaches. As well as searching the journals, Erin Saunders got help from her supervisor and others.
I was recommended a number of relevant texts by my supervisor—and from there I located other sources by
using the bibliographies of these texts. As well, I did an extensive journal search for articles that were related
to my topic. I also contacted a number of academics in the field to ask for specific suggestions. Then I read as
much of the literature as I could, identifying key themes and ideas.
Hannah Creane’s approach was to focus on key names in the sociological literature on childhood.
Initially I read a few core textbooks that cover the general aspects of sociology, and picked out from them the
main names of sociologists who have written about childhood and, in particular, childhood as a social
construction. From there I read the books of some of the key names within the field of childhood study, and
just simply kept looking up the names of sociologists whom they had referenced. I kept going like this until
I felt I had enough literature to back up my findings and theories that I made in the light of my own research.
Rebecca Barnes proceeded by identifying key texts and then using bibliographies.
Once I started to locate the core texts, this process gathered more momentum, since I was able to draw on
bibliographies in those sources to identify other relevant references.
To read more about Erin’s, Hannah’s, and Rebecca’s research experiences, go to the Online Resource Centre that
accompanies this book at: www.oxfordtextbooks.co.uk/orc/brymansrm4e/
Getting started: reviewing the literature 115
through their own websites; Cambridge University Press
(Cambridge Journals Online) and Sage (HighWire) are
the two most prominent examples. Again you will need
to check with your librarian to find out which of these
resources you can use and how to access them. The
INGENTA website offers full-text versions from various
publishers, and you will be able to access full-text versions
of articles in journals to which your library subscribes.
In addition to scholarly books and journals,
newspaper archives can provide a valuable supplementary
resource through which to review the emergence of
new topics in areas of social concern. Most newspapers
require subscription to be able to search their online databases
(for example, Financial Times, Daily and Sunday
Telegraph, The Times). However, most academic libraries
will have a subscription to some individual newspapers
or to a service such as Proquest or Lexis Nexis, which
allows you to search several newspapers at once; you
may need a password to access them. Newspapers and
periodicals can be a rich source of information about
certain topics that make good stories for journalists, such
as social problems, policy initiatives, or trade union disputes.
The level of analysis can also be high. For an
academic dissertation they should always be seen as
secondary to published literature in books and journals,
but it takes some time for academic articles to be published,
so for recent events newspapers may be the only
source of information.
A word of warning about using Google and other
search engines for research. Internet search engines are
very useful for researching all sorts of things. However,
they merely find sites; they do not evaluate them. So
be prepared to look critically at what you have found.
Remember that anyone can put information on the Web,
so, when looking at websites, you need to evaluate
whether the information you have found is useful. The
following points are worth considering.
• Who is the author of the site and what is his or her
motive for publishing?
• Where is the site located? The URL can help you here.
Is it an academic site (.ac) or a government site (.gov),
a non-commercial organization (.org) or a commercial
one (.com or .co)?
• How recently was the site updated? Many sites will
give you a last updated date, but you can get clues as
to whether a page is being well maintained by whether
the links are up to date and by its general appearance.
Try to confine your literature search to reliable websites,
such as those mentioned in this chapter. For more
on this issue, see Tips and skills ‘Using information on
the Web’.
The catalogue of your own institution is an obvious
route to finding books, but so too are the catalogues
of other universities. COPAC contains the holdings of
twenty-seven of the largest university research libraries
plus the British Library. It can be found at:
http://copac.ac.uk (accessed 17 January 2011).
A well-known website like amazon can also be extremely
helpful for searching for books.
Tips and skills
Using email alerts
One way of expanding your literature search is through email alerts. These supply you with an email when an
issue of a journal that you are interested in is published. You can also be sent email alerts when articles with
certain keywords or written by particular authors are published. One of the main ways of setting up email alerts
is through Zetoc, through the British Library. You will need to sign in with a username and password. An Athens
username and password will usually achieve this. To find Zetoc, go to:
http://zetoc.mimas.ac.uk/ (accessed 3 August 2010).
Alternatively, you can use Scopus for sending alerts when articles on nominated topics or by nominated authors
are published. Go to:
www.scopus.com/alert/form/MyAlerts.url (accessed 3 August 2010).
There is also a Scopus app for the iPhone, iPod Touch, and iPad that can be downloaded from:
http://itunes.apple.com/app/scopus-alerts-lite-take-your/id365300810?mt=8 (accessed 3 August 2010).
Getting started: reviewing the literature116
Tips and skills
Using information on the Web
The Internet provides an enormous and richly varied source of freely available information about social research
that can be quickly and easily accessed without the need for university agreements to gain access to them.
However, there is a difficulty in relying on this, because the strength of the Internet in providing access to huge
amounts of information is also its weakness, in that it can be very difficult to differentiate what is useful and
reliable from that which is too simplistic, too commercially oriented, too highly opinionated, or just not
sufficiently academic. The worst thing that can happen is that you end up quoting from sources from the Web
that are quite simply misleading and incorrect. Therefore, it is important to be selective in your use of information
on the Internet and to build up a list of favourite websites that you can check regularly for information.
However, such sources have to be evaluated critically. For example, while writing this chapter for the third edition
of this book, I encountered the following definition of qualitative research in Wikipedia, which is very popular
among students.
Qualitative research is one of the two major approaches to research methodology in social sciences.
Qualitative research involves an indepth understanding of human behaviour and the reasons that govern
human behaviour. Unlike quantitative research, qualitative research relies on reasons behind various aspects
of behaviour. Simply put, it investigates the why and how of decision-making, as compared to what, where, and
when of quantitative research. Hence, the need is for smaller but focused samples rather than large random
samples, which qualitative research categorizes into patterns as the primary basis for organizing and reporting
results. (http://en.wikipedia.org/wiki/Qualitative_research (accessed 12 February 2007))
This is a very misleading characterization of both quantitative and qualitative research. It implies that quantitative
researchers are not concerned with examining the ‘reasons behind various aspects of behaviour’. This is a quite
extraordinary notion. The whole point of the preoccupation with causality and the very notions of independent
and dependent variables that are part of the basic vocabulary of quantitative research (see Chapter 7) would
suggest the opposite: quantitative researchers are deeply concerned about exploring the reasons behind
behaviour. Also, qualitative researchers are concerned to explore ‘what, where, and when’, in that they frequently
engage in descriptions of what is happening at certain events or on particular occasions, where they take place,
and often draw inferences about their timing. Further, quantitative researchers ‘categorize . . . data into patterns’,
but the nature and character of those patterns assume a different form. This is a very poor definition and
characterization of qualitative research and demonstrates the risk of using Web sources in an unquestioning way.
Wikipedia contains some very good entries, but it has to be treated with caution, as do Web sources generally.
Interestingly, the above quotation can no longer be found at:
http://en.wikipedia.org/wiki/Qualitative_research (accessed 3 August 2010).
Searching tool
Google has a really useful product called ‘Google Scholar’, which can be accessed from the Google home page.
This product provides a simple way to search broadly for academic literature. Searches are focused on
peer-reviewed papers, theses, books, abstracts, and articles, from academic publishers, professional societies,
preprint repositories, universities, and other scholarly organizations. Google Scholar also enables you to see how
often an item has been cited by other people. This can be very useful in assessing the importance of an idea or
a particular scholarly writer. See:
http://scholar.google.com.
Current affairs
For case study analyses and keeping up to date on current social issues, the BBC News website is reasonably well
balanced and quite analytical:
www.bbc.co.uk.
Statistics on social trends
The National Statistics office makes a huge amount of data about social trends available on its website:
www.statistics.gov.uk
Getting started: reviewing the literature 117
The statistics on Internet use at the beginning of Chapter 28 are gleaned by examining this website.
European statistics relating to specific countries, industries, and sectors can be found on Europa, the portal to the
European Union website:
http://europa.eu/index_en.htm
Other useful websites that are relevant to research methods
Teaching Resources and Materials for Social Scientists:
http://tramss.data-archive.ac.uk/
Intute:
www.intute.ac.uk/services.html
Exploring online research methods:
www.geog.le.ac.uk/ORM/
Qualitative data analysis:
http://onlineqda.hud.ac.uk/
Research ethics:
www.ethicsguidebook.ac.uk and
www.lancaster.ac.uk/researchethics/index.html
Access to various data that can be used for secondary analysis:
www.esds.ac.uk
(All websites mentioned in this box were accessed 3 August 2010 unless stated otherwise.)
Student experience
Literature review as ongoing
The literature review is often viewed as a distinct phase in the research process, but in fact it is invariably an
ongoing component of a research project. While email alerts like Zetoc alerts (see Tips and skills ‘Using email
alerts’) may be a useful way of keeping on top of the literature, they also mean that the literature review may
not draw to a close at an early stage. Rebecca Barnes found that searching the literature was an ongoing
process.
Although at the beginning of my Ph.D., I dedicated a more prolonged period of time to searching for and
reviewing literature, this process has been an ongoing part of the research process. I used electronic databases
such as Cambridge Sociological Abstracts to identify sources which could be useful, and I was also fortunate in
stumbling across a bibliography of sources for same-sex domestic violence on the Internet. . . . I also subscribe
to Zetoc alerts, which means that rather than having to spend time regularly updating the literature which
I have, I am informed of many new articles as soon as they are published.
Rebecca’s experience is not unusual. Isabella Robbins, who was doing a Ph.D. at the time, describes the literature
review as feeling like ‘a process that has been ongoing for about six years’, while Sarah Hanson suggests that it
can be difficult to bring the review to a close.
The only difficulty I encountered was that I couldn’t stop reading; I had finished my literature review and had
started writing my dissertation, but I kept stumbling upon book after book, which then had to be encompassed
into the literature review. I ended up writing and rewriting my literature review.
In a similar vein Jonathan Smetherham wrote of the literature review for an undergraduate dissertation that he
began with some material with which he was familiar and then:
Getting started: reviewing the literature118
Keywords and defining search
parameters
For all these online databases, you will need to work out
some suitable keywords that can be entered into the
search engines and that will allow you to identify suitable
references. Journal articles often include lists of
keywords. When you find two or three articles that are
relevant to your research and that have lists of keywords,
it may be useful to use some of these keywords that are
relevant to your research for searching for other articles.
You will also need to think of synonyms or alternative
In most databases, typing in the title of your project,
or a sentence or long phrase, as your search term is not
advisable, as, unless someone has written something
with the same title, you are unlikely to find very much.
You need to think in terms of keywords (see Tips and
skills ‘Keywords’).
Use the HELP provided in the databases themselves
to find out how to use your keywords to best effect. The
advice on using ‘operators’ such as AND, OR, and NOT
can be especially helpful.
terms and try to match your language to that of the
source you are searching. For example, in the example in
Research in focus 5.4, I used ‘manage*’ and ‘administrat*’
as well as ‘leader*’ (see earlier in this chapter for
the use of asterisks as wild cards). This is not because I
think that management and administration are the same
as leadership but because I realized quite early on that
some authors use these terms either as synonyms for
leadership or in very similar ways. Be prepared to experiment
and to amend your keywords as your research progresses;
you may find as you search the literature that
there are other ways of describing your subject.
In some areas of research, there are very many references.
Try to identify the major ones and work outwards
from there. Move on to the next stage of your research at
the point that you identified in your timetable, so that
you can dig yourself out of the library. This is not to say
that your search for the literature will cease, but that you
need to force yourself to move on. Seek out your supervisor’s
advice on whether you need to search the literature
much more. Figure 5.1 outlines one way of searching the
literature. The most important thing to remember, as the
I developed research questions and then used these as the basis for doing a more probing lit review. By this
stage, I had seen a few of the ‘big names’ cropping up repeatedly, so I began searching out their scholarly work
for greater insight. . . . However, after the actual research project had been conducted in the field, I did
essentially rewrite the literature review, as the scope of my study changed so considerably during the
data-collection process. However, this was a much more focused and efficient exercise—in part due to the
impending deadline, and in part because the review was no longer an exploratory exercise but something
which was sharp, crisp and focused.
To read more about Rebecca’s, Isabella’s, Sarah’s and Jonathan’s research experiences, go to the Online
Resource Centre that accompanies this book at: www.oxfordtextbooks.co.uk/orc/brymansrm4e/
Tips and skills
Keywords
For all kinds of review—narrative or systematic—using keywords for searching online databases of articles is
crucial. However, it is not as easy as it seems. For example, though the authors of the article in Research in
focus 5.3 searched the literature thoroughly using keywords, they note that, after they had completed the
meta-ethnography on lay experiences of diabetes, they ‘were made aware of a meta-ethnography based on
43 qualitative reports concerned with the “lived experience of diabetes”’ (Campbell et al. 2003: 683). Not only
were they unable to uncover this article, which had been published in 1998, through their search, but also the
authors of the other meta-ethnography had included only three of the seven articles Campbell et al. had used.
Searching for keywords requires some experimentation and should not be regarded as a one-off exercise.
Getting started: reviewing the literature 119
gu e 5.Figure 5.1
One way of searching the literature
1. Read books or articles known to you or recommended by others related to your research questions
2a. Keep notes based on your reading of this literaturg e
b. Note keywords used in this literature
c. Make a note of other literature referred to that may be relevant and worth following upg
3. Generate keywords relevant to your research questions
4a. Search the library for literature relating tog your subject
b. Conduct an online search using an appropriate electronic databasg e
5a. Examine titles and abstracts for relevance
b. Retrieve selected items (back up to item 2a)
c. Check regularly for new publications
THEN
AND
AND
THEN
THEN
AND
THEN
AND
AND
Note: At each stage, keep a record of what you have done and your reasons for certain decisions. This will be useful to you for remembering how you
proceeded and for writing up a description and justification of your literature search strategy, which can form part of your methods section. When
making notes on literature that you read, make notes on content and method, as well as relevance, and keep thinking about how each item will
contribute to your critical review of the literature.
Getting started: reviewing the literature120
note at the end of the figure suggests, is to keep a good
record of the process so that you can keep track of what
you have done. Also, when you give your supervisor
drafts of your literature review, make sure you include
all the references and their details so that he or she can
assess the coverage and quality of your review adequately.
Referencing your work
Referencing the work of others is an important academic
convention because it emphasizes that you are aware of
the historical development of your subject, particularly
if you use the Harvard (or author–date) method, and
shows that you recognize that your own research builds
on the work of others. Referencing in your literature
review is thus a way of emphasizing your understanding
and knowledge of the subject. In other parts of your
dissertation referencing will serve somewhat different
purposes—for example, it will show your understanding
of methodological considerations or help to reinforce
your argument. A reference is also sometimes described
as a citation and the act of referencing as citing.
As I mentioned earlier on in this chapter, a key skill in
writing your literature review will therefore be to keep a
record of what you have read, including all the bibliographic
details about each article or book that will go into
your bibliography or references. For larger research projects
it can be useful to use note cards or software packages
that are designed specifically for this purpose such
as Procite or Endnote, but for a student research project
it will probably be sufficient to keep an electronic record
of all the items that you have read in a Word document,
although you should bear in mind that you may not
include all of these in your final bibliography. However,
the main thing to make sure of is that you keep your
bibliographic records up to date and do not leave this
until the very end of the writing-up process, when you
will probably be under significant time pressure.
Your institution will probably have its own guidelines
as to which style of referencing you should use in your
dissertation and if it does you should definitely follow
them. However, the two main methods used are:
• Harvard or author–date. The essence of this system is
that, whenever you paraphrase the argument or ideas
of an author or authors in your writing, you add in
parentheses immediately afterwards the surname of
the author(s) and the year of publication. If you are
quoting the author(s), you put quotation marks around
the quotation and after the year of publication you
include the page number where the quotation is from.
All books, articles, and other sources that you have
cited in the text are then given in a list of references
at the end of the dissertation in alphabetical order by
author surname. This is by far the most common referencing
system in social research and the one that we
follow in this book. It is, therefore, the style that we
would encourage you to use if your university does not
require you to follow its own guidelines.
• Footnote or numeric. This approach involves the use of
superscript numbers in the text that refer to a note at
the foot of the page or the end of the text, where the
reference is given in full together with the page number
if it is a direct quotation. If a source is cited more
than once, an abbreviated version of the reference is
given in any subsequent citation, which is why this is
often called the short-title system. As well as being
used to refer to sources, footnotes and endnotes are
often used to provide additional detail, including
comments from the writer about the source being
cited. This is a particular feature of historical writing.
One of the advantages of the footnote or numeric
method is that it can be less distracting to the reader
in terms of the flow of the text than the Harvard
method, where sometimes particularly long strings of
references can make a sentence or a paragraph difficult
for the reader to follow. Furthermore, software
packages like Word make the insertion of notes relatively
simple, and many students find that this is a
convenient way of referencing their work. However,
when students use this method, they often use it incorrectly,
as it is quite difficult to use it well, and they
are sometimes unsure whether or not also to include a
separate bibliography. The footnote approach to referencing
does not necessarily include a bibliography,
but this can be important in the assessment of students’
work (see Thinking deeply 5.2). As not having
a bibliography is a potential disadvantage to this style
of referencing, your institution may recommend that
you do not use it.
Getting started: reviewing the literature 121
Tips and skills
The Harvard and note approaches to referencing
The examples below show some fictitious examples of referencing in published work. Note that in published
articles there is usually a list of references at the end; books using the Harvard system usually have a list of
references, whereas a bibliography is used with the short-title system of notes. The punctuation of references—
such as where to place a comma, or whether to capitalize a title in full or just the first word—varies considerably
from source to source. For example, with Harvard referencing, in some books and journals the surname of the
author is separated from the date in the text with a comma—for example (Name, 1999)—but in others, like this
book, there is no comma. However, the main thing is to be consistent. Select a format for punctuating your
references, such as the one adopted by a leading journal in your subject area, and then stick to it.
An example of a Harvard reference to a book
In the text:
As Name and Other (1999) argue, motivation is a broad concept that comprises a variety of intrinsic and extrinsic
factors . . .
. . . and in the bibliography or list of references:
Name, A., and Other, S. (1999). Title of Book in Italics. Place of Publication: Publisher.
An example of a Harvard reference with a direct quotation from a book
In the text:
However, the importance of intrinsic factors often tends to be overlooked since ‘studies of motivation have
tended predominantly to focus on the influence of extrinsic factors’ (Name and Other 1999: 123).
. . . and in the bibliography or list of references:
Name, A., and Other, S. (1999). Title of Book in Italics. Place of Publication: Publisher.
An example of a Harvard reference to a journal article
In the text:
Research by Name (2003) has drawn attention to the importance of intrinsic factors in determining employee
motivation.
. . . and in the bibliography or list of references:
Refers to volume (issue) numbers
º
Name, A. (2003). ‘Title of Journal Article’, Journal Title, 28(4): 109–38.
Issue numbers are often not included, as in the case of the References in this book.
An example of a Harvard reference to a chapter in an edited book
In the text:
As Name (2001) suggests, individual motivation to work is affected by a range of intrinsic and extrinsic factors . . .
. . . and in the bibliography or list of references:
Name, A. (2001). ‘Title of Book Chapter’, in S. Other (ed.), Title of Book in Italics. Place of Publication: Publisher,
pp. 124–56. 0
Abbreviation for ‘Editor’
An example of a secondary reference using the Harvard method
In the text:
Individual motivation to work is affected by a range of intrinsic and extrinsic factors (Name 1993, cited in Other
2004).
. . . and in the bibliography or list of references:
Name, A. (1993). Title of Book in Italics. Place of Publication: Publisher, cited in S. Other (2004), Title of Textbook
in Italics. Place of Publication: Publisher.
An example of a Harvard reference to an Internet site
In the text:
Scopus describes itself as ‘the largest abstract and citation database of research literature and quality web
sources’ (Scopus 2007).
. . . and in the bibliography or list of references:
Scopus (2007). www.scopus.com/scopus/home.url (accessed 5 August 2010).
Getting started: reviewing the literature122
Note: it is very important to give the date of access, as some websites change frequently (or even disappear!
See Tips and skills ‘Using information on the Web’ for an example).
An example of a note reference to a book
In the text:
On the other hand, research by Name3
has drawn attention to the influence of intrinsic factors on employee
motivation . . .
. . . and in the notes:
3
A. Name, Title of Book in Italics. Place of Publication, Publisher, 2000, pp. 170–7.
An example of a note reference to an Internet site
In the text:
Scopus describes itself as ‘the largest abstract and citation database of research literature and quality web
sources’.39
. . . and in the notes:
39
Scopus (2007). www.scopus.com/scopus/home.url (accessed 5 August 2010).
Bear in mind that it is essential when preparing your own referencing in the text and the bibliography or list of
references that you follow the conventions and style that are recommended by your institution for preparing an
essay, dissertation, or thesis.
Tips and skills
Using bibliographic software
ProCite, EndNote, and Reference Manager are three of the leading Windows-based software tools used for
publishing and managing bibliographies. Your university may have a site licence for one of these packages. They
are used by academic researchers, information specialists, and students to create bibliographic records equivalent
to the manual form of index cards. This allows you to compile your own personal reference database. These
records can then be automatically formatted to suit different requirements—for example, to comply with the
referencing requirements of a particular scholarly journal. A further advantage to the software is that it can
enable you to export references directly from databases such as the Social Sciences Citation Index (SSCI). The
software also has search options that help you to locate a particular reference, although the extent of these
features varies from one package to another.
In the long run, this can save you time and effort and reduce the possibility of errors. However, for a student
research project it may not be worthwhile for you to take the time to learn how to use this software if it is only to
be used for the dissertation. On the other hand, if knowledge of the software may be useful to you in the longer
term, for example, if you are thinking of going on to pursue an academic career by doing a Ph.D., or if you are
intending to work in a field where research skills are valued, then it may be worth learning how to use the
software. More details about these products can be found on the following websites:
www.procite.com
www.endnote.com
www.refman.com
However, if you do not have access to one of these packages, similar software is offered free to students and can
be downloaded from the Internet. One of these is BiblioExpress, a simplified version of the package Biblioscape.
This package offers the main features associated with bibliographic referencing software and provides extensive
user support from its website, which includes a free downloadable user manual. BiblioExpress enables you to do
most of the main things needed for a student research project. For more details go to:
www.biblioscape.com/biblioexpress.htm
All web pages mentioned in this box were accessed 5 August 2010.
Getting started: reviewing the literature 123
The role of the bibliography
What makes a good bibliography or list of references?
You might initially think that length is a good measure,
since a longer bibliography containing more references
might imply that the author has been comprehensive
in his or her search of the existing literature. This is undoubtedly
true, but only up to a point, since it is also important
for the bibliography to be selectively focused—it
should not include everything that has ever been written
about a subject but instead should reflect the author’s
informed judgement of the importance and suitability of
sources. This incorporates some of the judgements about
quality that were discussed earlier on in this chapter. One
common proxy for quality is the reputation of the journal
in which an article is published. However, although this
is a useful indicator, it is not one that you should rely on
exclusively, since there might be articles in lesser-status
journals—for instance, those targeted at practitioners—
that have relevance to your subject. But it is important
to be aware of these judgements of quality and to seek
the advice of your supervisor in making them. Another
important feature of a good bibliography relates to
secondary referencing. This is when you refer to an article
or book that has been cited in another source such as a
textbook and you do not, or cannot, access the original
article or book from which it was taken. However, relying
heavily on secondary references can be problematic,
because you are dependent upon the interpretation of
the original text that is offered by the authors of the secondary
text. This may be adequate for some parts of your
literature review, but it is a bit like the game Chinese
Whispers, in that there is always the potential for different
interpretations of the original text, and this increases
the further removed you are from the original source.
So it is a good idea to be cautious in the use of secondary
references and to go back to the original source if you
can, particularly if the reference is an important one for
your subject. Thinking deeply 5.5 gives an example of
how an author’s work can be referenced in ways that
involve reinterpretation and misquotation long after the
date of publication. A further feature of a good bibliography
stems from the relationship between the list of
references at the end and the way in which they are used
in the main body of the text. It should go without saying
that it is not very helpful to include references in a list
of references that are not even mentioned in the text.
If references are integrated into the text in a way that
shows you have read them in detail and understood the
theoretical perspective from which they are written, this
is much more impressive than if a reference is inserted
into the text in a way that does not closely relate to what
is being said in the text. Finally, Barnett (1994) argues
that a good bibliography gives no indication of the quality
of a piece of work, pointing out that some of the most
influential academic books ever written do not even
Thinking deeply 5.5
The problem of using secondary
literature sources
Be careful when using second-hand accounts of theories or findings. It is well known that these are sometimes
misleadingly represented in publications—though hopefully not in this book! An interesting case is the Affluent
Worker research that is described later in this book in Research in focus 24.8. This research entailed a survey in
the 1960s of predominantly affluent workers in three firms in Luton. It is regarded as a classic of British sociology.
One of the authors of the books that were published from this research conducted a search for books and articles
that discussed the findings of this research. Platt (1984) shows that several authors misinterpreted the findings.
Examples of such misinterpretation follow.
• The study was based on just car workers. It was not—only one of the three companies was a car firm.
• The study was based on just semi-skilled or mass production workers. It was not—there were a variety of skill
levels and technological forms among the manual sample.
• The research ‘found’ instrumentalism—that is, an instrumental orientation to work. This is misleading—
instrumentalism was an inference about the data, not a finding as such.
The point of this discussion is the need to be vigilant about possibly recycling incorrect interpretations of
theoretical ideas or research findings.
Getting started: reviewing the literature124
include one. Drawing on the ideas of Bourdieu (1984),
he suggests that the main purpose of the bibliography is
to enable you to understand the habitus that the author
is claiming to reside in, this being about understanding
the beliefs and dispositions of the author combined with
the constraints associated with his or her situation.
Avoiding plagiarism
An issue to bear in mind when writing up your literature
review is the need to avoid plagiarizing the work that you
are reading. Plagiarism is a notoriously slippery notion.
To plariarize is defined in The Concise Oxford Dictionary
as to ‘take and use another person’s (thoughts, writings,
inventions . . . ) as one’s own’. Similarly, the online
Encarta UK English Dictionary defines it as ‘the process
of copying another person’s idea or written work and
claiming it as original’. Plagiarism does not just relate to
the literature you read in the course of preparing an
essay or report. Taking material in a wholesale and unattributed
way from sources like essays written by others or
from websites is also a context within which plagiarism
can occur. Further, it is possible to self-plagiarize, as
when a person lifts material that he or she has previously
written and passes it off as original work. Plagiarism is
commonly regarded as a form of academic cheating
and as such differs little if at all in the minds of many
academics from other academic misdemeanours such as
fabricating research findings.
There is a widespread view that plagiarism among
students is increasing in incidence, though whether this
is in fact the case is difficult to establish unambiguously.
Indeed, it is difficult to establish how widespread plagiarism
is, and there are quite substantial variations in
estimates of its prevalence. It is widely viewed that the
Internet is one of the main—if not the main—motor
behind the perceived increase in the prevalence of plagiarism.
The ease with which text can be copied from
websites, e-journal articles, e-books, online essays sold
commercially, and numerous other sources and then
pasted into essays is often viewed as one of the main
factors behind the alleged rise in plagiarism cases among
students in UK universities and elsewhere.
There are several difficulties with plagiarism as an
issue in higher education. One is that universities vary
in their definitions of what plagiarism is (Stefani and
Carroll 2001). Further, they vary in their response to it
when it is uncovered. They also vary in both the type and
the severity of punishment. Further, within any university,
academic and other staff differ in their views of the
sinfulness of plagiarism and how it should be handled
(Flint et al. 2006). There is also evidence that students
are less convinced than academic staff that plagiarism is
wrong and that it should be punished. Research at an
Australian university implies that staff are more likely
than students to believe that plagiarism is common
among students (J. Wilkinson 2009). Major reasons for
plagiarism on which staff and students largely agreed
were: a failure to understand referencing rules; laziness
or bad time management; and the ready availability of
material on the Internet. Interestingly, students were less
likely than staff to agree with the statement ‘Students
receive adequate guidance from staff about what is an
[sic] isn’t acceptable in terms of referencing in assignments’,
implying that many students feel they do not
receive sufficient advice. These findings point, at the
very least, to the need to be fully acquainted with your
institution’s regulations on plagiarism and its advice on
proper referencing.
In view of all these uncertainties of both the definition
of and the response to plagiarism, students may wonder
whether they should take the issue of plagiarism seriously.
My answer is that they most definitely should take
it seriously. Academic work places a high value on the
originality of the work that is presented in any kind of
output. To pass someone else’s ideas and/or writings off
as your own is widely regarded as morally dubious at
best. Thus, while there are several grey areas with regard
to plagiarism, as outlined in the previous paragraph, it is
important not to overstate its significance. There is widespread
condemnation of plagiarism in academic circles
and it is nearly always punished when found in the work
of students (and indeed that of others). You should
therefore avoid plagiarizing the work of others at all
costs. So concerned are universities about the growth
in the number of plagiarism cases that come before examination
boards and the likely role of the Internet in
facilitating it that they are making more and more use of
plagiarism detection software, which trawls the Internet
for such things as strings of words (for example, Turnitin
UK; see http://turnitin.com/static/index.html (accessed
5 August 2010) for more information). Thus, as several
writers (e.g. McKeever 2006) have observed, the very
Getting started: reviewing the literature 125
technological development that is widely perceived as
promoting the incidence of plagiarism—the Internet—
is increasingly the springboard for its detection. Even
well-known and ubiquitous search engines like Google
are sometimes employed to detect student plagiarism
through the search for unique strings of words.
The most important issue from the student’s point
of view is that he or she should avoid plagiarism at all
costs, as the penalties may be severe, regardless of the
student’s own views on the matter. First, do not ‘lift’ large
sections of text without making it clear that they are in
fact quotations. This makes it clear that the text in question
is not your own work but that you are making a point
by quoting someone. It is easy to get this wrong. In June
2006 it was reported that a plagiarism expert at the
London School of Economics had been accused of plagiarism
in a paper he published on plagiarism! A paragraph
was found that copied verbatim a published source by
someone else and that had not been acknowledged
properly as from another source. The accused person defended
himself by saying that this was due to a formatting
error. It is common practice in academic publications
to indent a large section of material that is being quoted,
thus:
undermined. It is also important to realize that, for many
if not most institutions, simply copying large portions of
text and changing a few words will also be regarded as
plagiarism.
Second, do not pass other people’s ideas off as your
own. This means that you should acknowledge the source
of any ideas that you present that are not your own. It
was this aspect of plagiarism that led to the author of The
Da Vinci Code, Dan Brown, being accused of plagiarism.
His accusers did not suggest that he had taken large
chunks of text from their work and presented it as his
own. Instead, they accused him of lifting their ideas from
a non-fiction book they had written (The Holy Blood and
the Holy Grail). However, Dan Brown did acknowledge
his use of their historical work on the grail myth, though
only in a general way in a list of acknowledgements, as
novelists mercifully do not continuously reference ideas
they use in their work. Brown’s accusers lost their case,
but there have been other high-profile cases of plagiarism
that have been proved. For example, in 2003, the
UK Prime Minister’s Director of Communications and
Strategy issued a briefing to journalists on the concealment
of weapons in Iraq. This was found to have been
copied from several sources and became known as the
‘dodgy dossier’. The fact that so much of it had been
taken from the work of others undermined its credibility
in the eyes of others.
One of the most important messages of this section
will hopefully be that you should guard against plagiarism
at all costs. But it should also be clear that you
should find out what your university and possibly departmental
guidelines on the matter are. Quite aside from the
rights and wrongs of plagiarism, it is not likely to impress
your tutor if it is clear from reading the text that large
chunks of your essay or report have been lifted from
another source with just your own words interspersing
the plagiarized text. In fact, that is often in my experience
a giveaway—the contrast in styles is frequently very
apparent and prompts the tutor to explore the possibility
that some or much of the assignment you submit has
in fact been plagiarized. Nor is it likely to impress most
tutors if much of the text has been lifted but a few words
changed here and there, along with a sprinkled few
written by you. However, equally it has to be said that
frequent quoting with linking sentences by you is not
likely to impress either. When I have been presented with
essays of that kind, I have frequently said to the student
concerned that it is difficult to establish just what his or
her own thoughts on the issue are.
Try therefore to express your ideas in your own words
and acknowledge properly those ideas that are not your
The most important issue from the student’s point of
view is that they should avoid plagiarism at all costs, as
the penalties may be severe, regardless of the student’s
own views on the matter. First, do not ‘lift’ large sections
of text without making it clear that they are in fact
quotations. This makes it clear that the text in question
is not your own work but that you are making a point by
quoting someone. It is easy to get this wrong. In June
2006 it was reported that a plagiarism expert at the
London School of Economics had been accused of
plagiarism in a paper he published on plagiarism! A
paragraph was found that copied verbatim a published
source by someone else and that had not been
acknowledged properly as from another source. The
accused person defended himself by saying that this was
due to a formatting error. It is common practice in
academic publications to indent a large section of
material that is being quoted. (Bryman 2012: 125)
The lack of indentation meant that the paragraph in
question looked as though it was his own work. While it
may be that this is a case of ‘unintentional plagiarism’
(Park 2003), distinguishing the intentional from the
unintentional is by no means easy. Either way, the credibility
and possibly the integrity of the author may be
Getting started: reviewing the literature126
own. Plagiarism is something you may get away with
once or twice, but it is so imprinted on the consciousness
of many of us working in universities nowadays that you
are unlikely to get away with it regularly. It is also extremely
irritating to find that your own work has been
plagiarized. I was asked to act as an external examiner of
a doctoral thesis and found that large sections of one of
my books had been taken and presented as the student’s
own work. I found this extremely annoying. A colleague to
whom I mentioned the incident remarked that the only
thing worse than plagiarism is incompetent plagiarism
—incompetent because the student had plagiarized the
work of someone he or she knew would be the external
examiner. However, on reflection, the colleague was mistaken.
Plagiarism is wrong—regardless of whether it is
competently implemented or not. It is precisely for this
reason that, in May 2007, Google banned advertisements
from companies that write customized essays for students
(http://news.bbc.co.uk/1/hi/education/6680457.stm
(accessed 5 August 2010)). Advice on plagiarism can
usually be found in handbooks provided by students’
institutions, as well as from www.plagiarism.org/ (accessed
5 August 2010).
One final point to note is that plagiarism is like a moving
target. What it is, how it should be defined, how it can
be detected, how it should be penalized: all these issues
and others are in a state of flux as I write this chapter. It
is very much a shifting situation, precisely because of
the perception that it is increasing in frequency. The
penalties can be severe, and, as I have witnessed when
students have been presented with evidence of their
plagiarism, it can be profoundly embarrassing and distressing
for them. The message is simple: do not do it and
make sure that you know exactly what it is and how it is
defined at your institution, so that you do not inadvertently
commit the sin of plagiarism.
Thinking deeply 5.6
Plagiarism and copyright in the case of a novel
Teenage American novelist Kaavya Viswanathan, author of How Opal Mehta Got Kissed, Got Wild, and Got a Life,
was accused of plagiarizing sections of passages from another novel by Kinsella called Can you Keep a Secret,
including entire sentences that were found to be virtually identical. Viswanathan claimed that the similarity was
unintentional and attributed it to her photographic memory. The book was subsequently withdrawn from sale
and the author’s $500,000 contract with the publisher Little Brown & Company cancelled, after it was found that
there were also passages by other writers, including work by Salman Rushdie and Megan McCafferty. The key
question, according to Lawson (2006), is whether the young novelist knew what she was doing and whether she
accepts it was plagiarism. He also contends that the case highlights some of the pressures that novelists are
placed under by publishers to make their mark in a market where they are competing against other forms of
entertainment.
Although this case highlights the contested nature of charges of plagiarism, including the importance of
ascertaining the author’s intent, which is very difficult to do, it also draws attention to the moral judgement
and significant penalties that may be levelled at an author if plagiarism is shown to have occurred. Although
university students are not in a situation of risking multi-million-dollar deals in the same way as these novelists,
the impact of plagiarism if it is shown to be significant can be highly detrimental in terms of their education and
career prospects.
Sources: S. Goldenberg, ‘Star Young Author Admits Unconscious Plagiarism’, Guardian, 26 Apr. 2006; M. Lawson, ‘Fingers in
the Word Till’, Guardian, 6 May 2006.
Getting started: reviewing the literature 127
Checklist
Questions to ask yourself when conducting and writing
a literature review
᭺ Have you reflected on what your audience is expecting from the literature review?
᭺ Is your list of references up to date in your current areas of interest? Are there new areas of interest
that you need to search on? Is it reasonably comprehensive?
᭺ What literature searching have you done recently?
᭺ What have you read recently? Have you found time to read?
᭺ What have you learned from the literature? Has this changed in any way your understanding of the
subject in which you are working?
᭺ Is your search for the literature and the review you are writing being guided by your research
questions? Has your reading of the literature made you think about revising your research questions?
᭺ Is what you have read going to influence or has it influenced your research design in any way? Has it
given you ideas about what you need to consider and incorporate?
᭺ Have you addressed any key controversies in the literature and any different ways of conceptualizing
your subject matter?
᭺ Have you been writing notes on what you have read? Do you need to reconsider how what you have
read fits into your research?
᭺ Have you adopted a critical approach to presenting your literature review?
᭺ What story are you going to tell about the literature? In other words, have you worked out what is
going to be the message about the literature that you want to tell your readers?
᭺ Has someone read a draft of your review to check on your writing style and the strength of your
arguments about the literature?
Source: adapted from Bruce (1994); Holbrook et al. (2007); Reuber (2010).
Key points
● Writing a literature review is a means of reviewing the main ideas and research relating to your
chosen area of interest.
● A competent literature review confirms you as someone who is competent in the subject area.
● A great deal of the work of writing a literature review is based upon reading the work of other
researchers in your subject area; key skills can be acquired to help you get the most from your
reading.
● Systematic review is a method that is gaining in popularity in social research as a way of enhancing
the reliability of literature searching and review.
● Narrative review is a more traditional approach that has advantages of flexibility, which can make it
more appropriate for inductive research and qualitative research designs.
Getting started: reviewing the literature128
Questions for review
Reviewing the existing literature
● What are the main reasons for writing a literature review?
● How can you ensure that you get the most from your reading?
● What are the main advantages and disadvantages associated with systematic review?
● What type of research questions is systematic review most suited to addressing?
● What are the main reasons for conducting a narrative literature review?
● In what type of research is narrative review most appropriate?
Searching the existing literature
● What are the main ways of finding existing literature on your subject?
● What is a keyword and how is it useful in searching the literature?
Referencing your work
● Why is it important to reference your work?
● What are the main referencing styles used in academic work and which of these is preferred by your
institution?
● What is the role of the bibliography and what makes a good one?
Avoiding plagiarism
● What is plagiarism?
● Why is it taken so seriously by researchers?
Online Resource Centre
www.oxfordtextbooks.co.uk/orc/brymansrm4e/
Visit the Online Resource Centre that accompanies this book to enrich your understanding of
reviewing the literature. Consult web links, test yourself using multiple choice questions, and
gain further guidance and inspiration from the Student Researcher’s Toolkit.