Current Sociology
2016, Vol. 64(5) 691­–698
© The Author(s) 2016
Reprints and permissions:
sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/0011392116656711
csi.sagepub.com
CS
How double-blind peer
review works and what it
takes to be a good referee
Eloísa Martín
Federal University of Rio de Janeiro, Brazil
Abstract
This editorial describes the double-blind peer review process used by academic journals
in the social sciences with a particular focus on the process at Current Sociology. Guidance
is also provided to help referees write an appropriate and constructive review. Finally,
it addresses some of the major challenges of the peer review process at the practical,
epistemic and political level.
Keywords
Academic publication, double-blind peer review, international publication,
productivism, referee, the gift
Like other prestigious academic journals, Current Sociology uses double-blind peer
review for all articles it publishes. Each article is evaluated by at least two scholars in
a procedure that keeps both the author and the reviewers anonymous, hence the term
‘double-blind.’ There are several steps in the review process, which can be longer or
shorter depending on the journal.
At Current Sociology, the editor does an initial review of an article, sometimes with
the help of the Editorial Board, to determine whether it meets the journal’s requirements.
This means that the article must comply with the journal’s guidelines (format, minimum
and maximum number of words, and style) and fit the journal’s goals and editorial policy.
If the article is not aligned with the journal’s interests or objectives or if a very preliminary
version has been submitted (with errors, shortcomings or a poorly developed
argument), it will be desk-rejected. At Current Sociology, a desk rejection is normally
decided within the first week after submission.
How peer review works
Once the editor has green lighted an article, the review process begins. It is the editor’s
task to find referees who read the language the article is written in and who are capable
656711CSI0010.1177/0011392116656711Current SociologyEditorial
research-article2016
Editorial
by guest on August 15, 2016csi.sagepub.comDownloaded from
692	 Current Sociology 64(5)
of gauging its suitability, relevance and originality. Potential referees also need time and
energy available for this activity, which can take several months. As a result, finding
referees is never easy. First and foremost, this is because Current Sociology receives
articles on different topics with widely different theoretical and methodological
approaches from all over the globe. Second, Current Sociology receives articles in the
three officials ISA languages (English, French and Spanish) and in any other language in
which suitable reviewers can be found. And finally, in spite of the best efforts by the
editorial office, potential reviewers may not be available or may not be able to review the
article in time. Although the period between manuscript submission and the first feedback
from reviewers is an average of 45 days, in some cases it is necessary to ask 10, 15
or even more colleagues before two agree to review the article. This search can take up
to a month but is a process invisible to the author, who simply experiences this as a long
wait. Once the referees have been chosen, there are usually two or three review rounds
before an article submitted to Current Sociology is ultimately approved. Rejections usually
occur after the first round or much less frequently, after the second round.
Current Sociology referees are experts in the topics discussed in the manuscript and they
are active in debates in their respective fields at the international, regional and local level.
In order to help ensure an article is relevant, we try to find at least one referee who works
or specializes in the region analyzed; the other referee must be capable of evaluating
whether the manuscript’s analysis is suitable for other contexts or for an international
debate. For example, a study on reproductive healthcare policies in France might be
reviewed by a specialist who works on policies like these in France – ideally, someone who
works at a French institution or in another country of Europe, and who reads and publishes
in French. This person would be entrusted with evaluating the suitability of the data and the
originality of the argument vis-a-vis local and regional publications on the topic. The second
reviewer would ideally be a specialist in reproductive healthcare policies from another
part of the world – Canada, Venezuela, Japan,Australia or SouthAfrica, for example – who
is able to say whether the work contributes to the debate in other contexts and whether it
reflects the current debate internationally. This geopolitical approach towards peer review
has helped us avoid cases of republication, simultaneous submission and plagiary (these
are few and far between, fortunately, but they do occur). On the other hand, with this
approach we also encourage authors to present discussions and ideas in their texts that they
had not spotted in the first drafts but which would be fruitful for international Current
Sociology readers. We consciously seek to merge local and regional debates with international
currents, thus contributing to a richer and more cosmopolitan production within sociology.
This goes beyond merely reproducing mainstream debates and applying established
theories to local data; it is not about imposing a single style of academic writing that could
limit original ideas and approaches. Instead, Current Sociology seeks to create spaces of
intelligibility where local data and theoretical discussions may not necessarily be ‘applicable’
or able to be imported to other contexts but where the diversity of topics and debates
sparked by authors across the globe can shed light on discussions elsewhere.
At Current Sociology, we look for referees who are capable of using their experience
in a specific niche within their field to help authors develop and express ideas in the best
way possible in their manuscript. Referees are thus expected to be more than mere gatekeepers
of established knowledge; they should be committed to the collective production
by guest on August 15, 2016csi.sagepub.comDownloaded from
Martín	 693
of knowledge, based on constructive criticism and proposals to improve the work, even
in the case they suggest rejecting the article.
In general, referees tend to agree on the shortcomings of an article, although they
may draw the author’s attention to different issues in the manuscript. At the end of each
round of revision, when I send my decision to the authors, I usually comment on the
referee reports, guiding authors in what needs to be reworked. It is less common for us
to receive two different recommendations from the referees, with one suggesting minor
revision or acceptance and the other suggesting rejection. When this does occur, the
editorial decision at Current Sociology will always be based on whether the referee
who suggested rejection will agree to read a second version. In some cases, the referee
considers that the modifications that the article requires cannot be done within the time
allotted for the review process and this is the reason for rejection. If this referee decides
to give the author another chance and read a second version, the author will be invited
to resubmit after a major revision. In some cases, the referee who suggests rejection
believes the issues in the manuscript are insurmountable and thus refuses to read any
new version.
In such cases, and considering that another peer reviewer has given a vote of confidence
to the manuscript, we invite a third (and sometimes, a fourth) referee to evaluate
the article. The goal is not only to break the ‘deadlock’but also to offer the author more
resources to improve his/her work while giving the editor enough information to decide
whether the article in publishable. Thus, and as I have shown earlier (Martín, 2015),
during the revision process there is (or should be) a conversation behind the scenes that
actively involves the editor. In cases in which a referee is inflexible when it comes to
his/her belief that a manuscript should be rejected but when other reviewers gave positive
evaluations and the author provided a convincing counterargument, I as an editor
follow the logic of in dubio, pro reo. I would rather publish an article that isn’t that
good – an article that could eventually be criticized by the academic community due to
its shortcomings – than miss out on a ground-breaking work just because it didn’t convince
one referee. Luckily, thinking back on the referees who have collaborated with
Current Sociology over the past six years, I have encountered only a handful of cases
in which a referee seemed closed off to new perspectives. In most cases, even if the
Current Sociology referees don’t agree with the theoretical perspective or believe that
the analytical framework of an article could be enhanced, they usually forgo their individual
preferences (the article they would like to read or the article they themselves
would have written) to focus on their colleague’s work. Generosity, which has become
increasingly rare in academia, continues to be a value for the production of knowledge
at Current Sociology.
In addition to the referees, the authors play a fundamental role in the process: after all,
it’s their article that will ultimately be published. During the review, an academic debate
involving the author, referees and the editor takes place. Authors should be prepared to
take criticism and to defend their ideas – with clear sociological arguments – if they disagree
with referees’ recommendations. It is fundamental to understand that the review
is an ongoing process where referees point out the stronger and weaker points of a
manuscript. The editor serves as a mediator in this process and ultimately decides if the
manuscript is publishable.
by guest on August 15, 2016csi.sagepub.comDownloaded from
694	 Current Sociology 64(5)
How to be a good peer reviewer
Although the entire publication system is based on peer review, there is no specific training
for refereeing. Any scholar could be asked – and is expected to give – their impression
of a colleague’s work, and their comments are taken so seriously by editors than the
fate of this work often depends on their review. This is a job with great responsibility
though it is not given any special training, but a scientific vocation. Though it is supposed
to be a very bureaucratic and objective work, it is one that requires a good deal of
intuition. All in all, the system has worked quite well, like other tasks most scholars learn
through trial and error. But considering the importance of peer review, and the fact that
as we are learning how to do our job as referees, we could have a negative impact on
someone else’s work, I would like to offer a general overview on doing a peer review.
The main concept is that referees should be helpful and constructive in their comments
and concerns, so authors can improve – and not just abandon – their manuscripts.
The first condition for doing a good job as a referee is to leave aside one’s own beliefs
and enter the author’s logic. Peer review isn’t about coming up with a list of changes the
author must make in order to make the article interesting for the referee: it’s about collaborating
with the author so he or she can get the most out of the manuscript’s reflections.
Participating in the peer review process is – or should be – the most generous task within
the academic work. Referees, after all, help authors to improve their work at no cost and
also anonymously, and the only one who directly benefits from this work is the author,
whose work eventually gets published. True, the name of the referees may appear in a long
list of acknowledgments the journal releases at the end of the year and there may be a few
perks (like a copy of the journal, a discount or a freebie). Some new websites even list all
of a scholar’s work as a reviewer. However, the time and energy a reviewer puts into the
colleague’s work (even if their evaluation is negative, even if a reviewer hasn’t had time to
go over the adjectives and forgets to be gracious in his or her comments) is time and energy
she or he takes away from other potential activities (work related or otherwise).
To do the sort of review that helps the editor to make a decision while allowing the
author to improve a manuscript, the referees must evaluate the style, content and relevance
of the article.
In terms of style, the referee must evaluate whether the article makes an academic
contribution or if it is more of a research report, a piece for public policy or a manifesto.
Even if the manuscript presents a rigorous methodology for gathering data or a coherent
political analysis, scientific articles have particularities that distinguish them from other
texts and make them publishable exclusively in academic journals. At the same time, the
referee will check to ensure that the text is well written and coherent and determine
whether it develops a clear argument throughout. Here the referee notes whether the text
is grammatically correct and if the argument flows. The reader should be gracefully led
into the reasoning of the proposal; otherwise, the manuscript proves a hard read, the main
argument is lost amid overlapping thoughts, or the text is redundant (or the author has
merely copy-pasted certain sections).
In terms of the content, the referee should evaluate whether there are gaps in the
empirical research or the secondary data and whether the theoretical-conceptual discussions
of the topic are up-to-date. Have key authors and contemporary issues been
by guest on August 15, 2016csi.sagepub.comDownloaded from
Martín	 695
included? In this regard, I consider it critical for there to be both a referee from the
region analyzed (or from the region where the author is writing) as well as a referee
from an entirely different part of the world. This way, the referees – from their respective
locations – will evaluate whether the author adequately explains the principal contributions
his or her article makes to local and international debates. This doesn’t mean
that referees expect a quote from every author who has ever addressed the subject (or
even a certain topic within the debate) but they must be aware of the originality and the
suitability of certain authors to certain debates. An author should never present an existing
analysis as original – even if this analysis has never been published in English, for
example. In addition, if the article to be reviewed is research-based, the referees must
also evaluate the methodology used, gauging whether the data the author presents suffice
and are relevant.
When it comes to evaluating the contents of an article, some referees want to see
their own works cited by the author. Evidently, when an editor invites a particular
scholar to review a manuscript, it is because the editor considers that the reviewer’s
own academic work contributes to the topics addressed in the manuscript. This doesn’t
necessarily mean that a recently published doctoral thesis or article by the referee is an
essential reference to a given debate or in the article to be reviewed. In some cases,
however, the referees are key authors in certain discussions and their publications
should be included. In general, these referees are more cautious about recommending
their own work, which will probably be suggested by the other referee or referees in
their own evaluation of the manuscript. My advice to the referee in this case is to
evaluate whether his/her work is actually essential to the manuscript, that is, whether
there are overlapping themes or some explicit and unquestionable points in common
between the manuscript and his/her own publication. If there are, the referee must
explain to the editor, in the confidential report, why the reference is essential; and
ultimately it would be up to the editor to decide whether or not to suggest that the
author include the reference.
On the other hand, because of the anonymity requirement, authors are asked to remove
any reference from their text that could identify them as the authors. In many cases –
obviously, unbeknownst to them – the reviewers recommend that the author add a work
that he/she was forced to omit to preserve anonymity. I believe that this is proof that
double-blind peer review works and is essential for the production of scientific
knowledge.
Finally, referees must analyze whether the article is relevant to both the journal in
question and to the current debate surrounding the topic. This means assessing whether
the article looks like something that particular journal would publish, whether the
author has considered the journal’s publishing tradition and its specific audience. On
the other hand, this means that the reviewer must analyze the originality of the contribution
and its potential contribution to the community of readers. In some cases, an
article can be well-written with rigorous data and cite the right authors without presenting
anything new. If it doesn’t make a contribution to the current debate – if it does
not provide an answer to the question ‘so what?’– reviewers probably should not recommend
it be published.
by guest on August 15, 2016csi.sagepub.comDownloaded from
696	 Current Sociology 64(5)
Major challenges
The peer review process can be slow, complicated and not always beneficial for authors –
or, obviously, for journals. It could be argued that a good part of the academic system,
including publishing and funding for research is, in Weberian terms, rational, bureaucratic
and goal-oriented. However, the system rests on a merit-based distribution process based
on a pre-modern institution: the gift. Generally, referees receive no payment for evaluating
a colleague’s work; they justify their neutral stance and the good faith of their reviews on
the fact that the manuscript they are reading aims to expand scientific knowledge in a given
area; and they trust that they will receive this same treatment from their peers when it is
their turn to be evaluated. In this regard, both the author and the reviewer expect and trust
in the generosity of all parties in order for the system to work. Curiously, generosity is not
a scientific value, just as the gift is not a rational procedure, at least in the Weberian sense.
In spite of these particularities – and even considering the issues that have arisen – doubleblind
peer review has been working reasonably well and is considered the most appropriate
and efficient type of review by authors, referees and editors alike. However, as many editors
frequently lament, it has proven increasingly difficult to make a peer review process
reliable and productive, basically because those who are entrusted with doing the reviews
often find themselves in situations that force them to renounce their generosity.
On the other hand, for editors, it has become more difficult to find peers who agree to do
a review. This is because the scholars who are invited as referees are also researchers, professors,
authors, editors and contributors to professional associations; as a result, they
receive many invitations to review articles. Not to mention the fact that the referee’s task is
a difficult one that requires time and dedication; it is done at no cost but is not considered in
a professional’s evaluations of performance or productiveness. Perhaps that is the reason for
the recent appearance of websites that report on a scholar’s activities as a reviewer, acknowledging
an activity that is otherwise anonymous. These ‘gold stars’ can thus be enumerated
and included on CVs and job applications, providing validation for a task which otherwise
remains invisible.Although this is a positive step, as it recognizes the efforts of referees and
makes them quantifiable, measurable and comparable – thus encouraging rational scholars
to ‘invest’ in the activity – it also entails the same risks that have been reported in terms of
the productivism of journals: quantity above quality, speed above depth.
Therefore, because of either the number of a scholar’s commitments or because of this
new drive to achieve more goals, review reports have become shorter and less detailed.
As a result, they don’t necessarily help an author to improve his/her article. At the same
time, the referees who agree to do reviews are getting younger: most senior scholars
seem to have less time, mainly due to the obligations that correspond to higher posts. The
number of invitations that go out for peer reviews could explain the fact that sometimes
scholars don’t even respond to the requests sent out by editors.
Perhaps the lack of time and an excessive number of obligations also explain - in addition
to the rules of etiquette that apply in each national academie - a certain disregard in
reviewer comments. When pressed for time, some reviewers use harsh adjectives when
describing problems in a colleague’s work. This lack of courtesy can turn the review process
into a question of personal honor as opposed to an exchange on how to best improve
the text’s arguments. The review process, especially if one referee proposes a major
by guest on August 15, 2016csi.sagepub.comDownloaded from
Martín	 697
revision, could be interrupted by an author’s urge to defend his or her work when an
unconstructive comment is made. The focus of the debate thus shifts away from what really
matters, which is the central argument of the article and its strengths and weaknesses.
Conversely, the author may have a different response if a reviewer writes, ‘This article
needs more work’ as opposed to ‘This article is terrible’. In both cases, the idea is the
same, i.e. the manuscript’s current version is unacceptable. Yet unnecessarily aggressive
comments tend to be discouraging to authors, especially younger ones.
On the other hand – and here I must acknowledge the always-frustrating task of
reviewers –the pressure to publish (a lot) or perish (in agony) has produced a great number
of articles sent in for publication when they still clearly need several months of work.
For that reason, there are confidential comments for the editor: the referees can vent their
frustration and freely state that they have ‘wasted hours reading this horrible article’.
With the editor, reviewers are free to clarify that beyond their attempts to offer constructive
criticism, the article is ‘the worst thing I’ve read in years’. The editor needs to know
this and will obviously consider such comments, comparing them with what the other
referees have shared before making an informed decision: one complaint does not suffice
and the referee needs to specify the reasons for a negative evaluation. Yet the author
doesn’t need to know that his or her work has upset the reviewer this much: being
informed won’t necessarily help an author improve the work and will certainly not predispose
him or her to react positively during the back-and-forth of the review process.
Final remarks
Given all of the challenges and issues that come up during peer review, the editor is the
one who is ultimately responsible for making decisions on whether an article is publishable
and whether a referee report is suitable. The editor serves as a bridge between the
authors, the referees and the expectations of the journal’s readers. He or she must work
with authors to help them communicate their analysis and reflections as effectively as
possible. An editor must reach out to referees when they propose changes that are not
associated with the style or the theoretical framework to keep the author from losing
sight of his/her original text; the editor must be attentive to comments that ultimately
reflect what the referee would have liked to have read or written. The editor also needs
to understand that national academic cultures in each country might determine a certain
writing style (of the author) and a certain style of critique (of the reviewer) and work with
them both to generate fruitful spaces of intelligibility. Here consensus is forged not by
common visions or overlapping interests but by finding a path where disagreements can
eventually lead to a collective production of knowledge.
Reference
Martín E (2015) Letters of rejection. Current Sociology 63(7): 937–942.
Résumé
Cet éditorial décrit l’évaluation à double insu par des pairs des revues universitaires de
sciences sociales, tout en mettant l’accent sur la procédure suivie par Current Sociology.
by guest on August 15, 2016csi.sagepub.comDownloaded from
698	 Current Sociology 64(5)
Cet article prodigue aussi des conseils pour aider les arbitres scientifiques à rédiger des
évaluations appropriées et constructives. Finalement, il évoque les principales difficultés
rencontrées au cours du processus d’examen, sur les plans pratique, épistémique et
politique.
Mots-clés
Arbitre scientifique, cadeau, evaluation à double insu par les pairs, productivisme,
publication internationale, publication scientifique
Resumen
Este editorial describe el proceso de dictamen doble ciego que caracteriza a las revistas
científicas, y analiza como funciona dicho proceso en periódicos de ciencias sociales
en general y en Current Sociology en particular. Ofrece, además, orientaciones para la
elaboración de un dictamen apropiado y constructivo. Finalmente, discute algunos de los
principales desafíos – prácticos, epistémicos y políticos – del proceso de revisión de pares.
Palabras clave
Dádiva, dictamen doble ciego, dictaminador, productivismo, publicación académica,
publicación internacional
by guest on August 15, 2016csi.sagepub.comDownloaded from