Introduction

In recent years authors of scholarly publications face the risk of publishing their work with a bogus publisher or in an untrustworthy journal. In practice, the term "predatory" is widely used to denote these publishers. However, with regard to the fact that such publishers do not comply with the principles of transparency and best practices in scholarly publishing, the use of terms such as untrustworthy, bogus, and suspicious is increasing. For that reason we use the terms untrustworthy journal and bogus publisher in this study material.

The main aim of bogus publishers and untrustworthy journals is easy financial gain. Therefore, they eschew the usual peer-review process before publishing a text, they create fictitious editorial boards, imitate the names of well-established, credible journals, etc. As a result of this behaviour, the academic community is deluged with a large amount of unverified and even distorted or false information. Now more than ever, there is an urgent need for institutions and researchers to be informed about the vital necessity of evaluating a publisher and journal before submitting their article. For example, already today the Committee for Evaluation of Research Organisations and Completed Programmes has information that in the past, even texts published by bogus publishers have been evaluated. For example, in March 2016, the Student Chamber of the Council of Higher Education Institutions informed the Council about such a case at a meeting with the members of the Council.

With regard to the above-mentioned facts, informing academics about how to detect untrustworthy journals is the most effective protection against publishing in such journals. Only then will awareness spread through academia about the risks of publishing in untrustworthy journals that were created merely for the purpose of financial gain.

The aim of this study material is to summarize the basic facts about where untrustworthy journals come from and possible ways for detecting them.

1 The origin of untrustworthy journals

Untrustworthy journals sponge on the otherwise noble concept of the Open Access movement, which aims to disseminate credible specialized information freely to the research community. Open Access strives for unrestricted access to literature, limited neither by financial nor by technical barriers, and its aim is to publish research results that underwent a regular peer-review process before being published.

A part of publishing in the Open Access mode is covering the costs for editing and typesetting of the texts, fees for server operation, etc. Therefore, today authors can use two modes of Open Access publishing for submitting their work to a scholarly journal:

Open-Access Journal

Such journals offer full texts of their articles to readers on the internet free of charge while the costs for publication of the article are covered either by the author (so-called Article Processing Charges or APCs), or by the institution which publishes the respective journal (for example Central European Journal of Nursing and Midwifery published by the University of Ostrava).

Hybrid Open-Access Journal

In these journals, access to the journal’s content is not free of charge by default, but authors have the possibility to publish their article in the Open Access mode if they pay a fee for it.

The Emergence of Untrustworthy Journals

Untrustworthy journals emerged with the aim to exploit the Open Access mode. While proper journals in the OA mode observe the common practices of scholarly publishing (peer-review, specialists in their international editorial board, etc.), untrustworthy journals not only violate these practices, but they also often try to trick potential authors into publishing their articles with them (e.g. they create fictitious editorial boards, imitate titles of prestigious journals, perform a speedy review process, etc.). The sole aim of all these practices is to make authors publish in their journal in order to collect a fee from these authors.

The year 2008 brought the first mention of untrustworthy journals, although they were not yet termed untrustworthy. At that time, the person who drew attention to them was Tim Hill, the owner of the New Zealand publishing house Dove Medical Press publishing in the Open Access mode. In 2010, Jeffrey Beall, a librarian from the American University of Colorado Denver, published on his (now-defunct) blog Scholarly Open Access a list of bogus journals’ publishers and a list of untrustworthy journals, the so-called Beall’s list. Two years later, he proposed criteria by which untrustworthy publishers could be identified. Beall updated these criteria and lists until mid-January 2017 when he cancelled the blog (therefore only a link to its archived version is provided).

Although Beall’s List gained popularity in the academic community because of the possibility of determining quickly and easily whether a journal is listed there or not, its main flaw was a lack of transparency. Jeffrey Beall defined 55 criteria of untrustworthy journals, but many of these criteria proved to be controversial, because they are either difficult to verify or their evaluation is subjective.

The method for journal evaluation that we have provided since 2017 to scholars and PhD students at the University Campus Bohunice has repeatedly undergone critical discussion, which resulted in its current 10 objectively verifiable criteria. In the following parts of this study material, we present these 10 criteria in context with the respective criteria from Beall’s list. In this way we want to help you understand that journal evaluation is not just black and white, and a complex approach is necessary.

Použitá literatura

Fact SheetMEDLINE® Journal Selection [Internet]. NIH: U.S. National Library of Medicine. 2019 [cited 2020 Feb 4]. Available from: https://www.nlm.nih.gov/lstrc/jsel.html
Akshantala Education and Charitable Trust. IBI Factor [Internet]. InfoBase Index. c2013 [cited 2020 Feb 6]. Available from: http://www.infobaseindex.com/ibifactor.php
Andrew C, Traynor V, Iverson D. An integrative review: understanding driving retirement decisions for individuals living with a dementia. J Adv Nurs [Internet]. 2015 Dec [cited 2019 Nov 17];71(12):2728–40. Available from: https://doi.org/10.1111/jan.12727
Beall J. List of Publishers [Internet]. Scholarly Open Access: Critical analysis of scholarly open-access publishing. 2012 [cited 2017 January 31]. Available from: http://web.archive.org/web/20160524111242/https://scholarlyoa.com/publishers/
Beall J. Hijacked Journals [Internet]. Scholarly Open Access: Critical analysis of scholarly open-access publishing. 2014 [cited 2017 January 31]. Available from: http://web.archive.org/web/20160604161447/https://scholarlyoa.com/other-pages/hijacked-journals/
Beall J. Cardiology Journal’s Decline is Heartbreaking [Internet]. Scholarly Open Access: Critical analysis of scholarly open-access publishing. 2014 [cited 2017 January 31]. Available from: http://web.archive.org/web/20160608065500/https://scholarlyoa.com/2014/07/08/cardiology-journals-decline-is-heartbreaking/
Beall J. Criteria for Determining Predatory Open-Access Publishers [Internet]. 3rd ed. [Denver]; 2015. 6 p. [cited 2016 May 1]. Available from: http://www.ufv.ca/media/assets/research/workshops/Criteria-for-Determining-Predatory-Publishers.pdf
Beall J. More Duplication of Journal Titles and Conference Names by Predatory Publishers [Internet]. Scholarly Open Access: Critical analysis of scholarly open-access publishing. 2015 [cited 2017 January 31]. Available from: http://web.archive.org/web/20161224064542/https://scholarlyoa.com/2015/10/06/more-duplication-of-journal-titles-and-conference-names-by-predatory-publishers/
Beall J. Another Article about Plagiarism — That Contains Plagiarism [Internet]. Scholarly Open Access: Critical analysis of scholarly open-access publishing. 2016 [cited 2017 January 31]. Available from: http://web.archive.org/web/20160503043821/https://scholarlyoa.com/2016/02/16/another-article-about-plagiarism-that-contains-plagiarism/
Beall J. List of Standalone Journals [Internet]. Scholarly Open Access: Critical analysis of scholarly open-access publishing. 2016 [cited 2017 January 31]. Available from: https://web.archive.org/web/20170111172309/https://scholarlyoa.com/individual-journals/
Beall J. Misleading Metrics [Internet]. Scholarly Open Access: Critical analysis of scholarly open-access publishing. 2016 [cited 2017 January 31]. Available from: https://web.archive.org/web/20170111172311/https://scholarlyoa.com/other-pages/misleading-metrics/
Beall J. Scholarly Open Access: Critical analysis of scholarly open-access publishing. 2016 [cited 2017 January 31]. Available from: http://web.archive.org/web/20170103170903/https://scholarlyoa.com/
Beall J. Predatory Publisher Organizes Conference Using Same Name as Legitimate Conference | Scholarly Open Access [Internet]. Scholarly Open Access: Critical analysis of scholarly open-access publishing. [cited 2017 January 31]. Available from: http://web.archive.org/web/20160503043451/https://scholarlyoa.com/2014/08/28/predatory-publisher-organizes-conference-using-same-name-as-legitimate-conference/
Beall J. Proposed Criteria for Identifying Predatory Conferences [Internet]. Scholarly Open Access: Critical analysis of scholarly open-access publishing. 2016 [cited 2017 January 31]. Available from: http://web.archive.org/web/20160624083058/https://scholarlyoa.com/2016/06/23/proposed-criteria-for-identifying-predatory-conferences/
Beall J. Don’t Use PubMed as a Journal Whitelist [Internet]. Scholarly Open Access: Critical analysis of scholarly open-access publishing. 2016 [cited 2017 February 3] Available from: http://web.archive.org/web/20170114052258/https://scholarlyoa.com/2016/10/20/dont-use-pubmed-as-a-journal-whitelist/
Bloudoff-Indelicato M. Backlash after Frontiers journals added to list of questionable publishers. Nature [Internet]. 2015 jen [cited 2016 May 1];526(7575):613–613. Available from: http://www.nature.com/doifinder/10.1038/526613f
Butler D. Investigating journals: The dark side of publishing. Nature News [Internet]. 2013 Mar 28 [cited 2016 May 5];495(7442):433. Available from: http://www.nature.com/news/investigating-journals-the-dark-side-of-publishing-1.12666
Carafoli E. Scientific misconduct: the dark side of sicence. Rend. Fis. Acc. Lincei [Internet]. 2015 [cited 2016 August 8];26(3):369-382. Available from: http://link.springer.com/content/pdf/10.1007%2Fs12210-015-0415-4.pdf doi: 10.1007/s12210-015-0415-4
CASP. Critical appraisal skills programme checklist [Internet]. Oxford: Critical Appraisal Skills Programme; 2018 [cited 2019 Oct 28]. Available from: https://casp-uk.net/wp-content/uploads/2018/01/CASP-Qualitative-Checklist-2018.pdf
Clarivate Analytics. Title suppressions [Internet]. InCites Journal Citation Reports Help. 2018 [cited 2019 Nov 4]. Available from: http://help.incites.clarivate.com/incitesLiveJCR/JCRGroup/titleSuppressions.html
Coughlan M, Cronin P, Ryan F. Step-by-step guide to critiquing research. Part 1: quantitative research. Br J Nurs [Internet]. 2007 [cited 2019 Nov 17];16(11):658–63. Available from: https://doi.org/10.12968/bjon.2007.16.11.23681
Edie AH, Conklin JL. Avoiding predatory journals: Quick peer review processes too good to be true. Nurs Forum. 2019;54(3):336–9.
Elsevier. Scopus: Content coverage guide [Internet]. [Amsterdam]: Elsevier; 2017 [cited 2019 Oct 29]. Available from: https://www.elsevier.com/__data/assets/pdf_file/0007/69451/0597-Scopus-Content-Coverage-Guide-US-LETTER-v4-HI-singles-no-ticks.pdf
Fórum Věda žije! Akademicky antipredátor [Internet]. Akademický antipredátor. 2016 [cited 2016 May 1]. Available from: http://antipredator.vedazije.cz/
Frandsen TF. Why do researchers decide to publish in questionable journals? A review of the literature: Why authors publish in questionable journals. Learned Publishing [Internet]. 2019 [cited 2019 Nov 3];32(1):57–62. Available from: http://doi.wiley.com/10.1002/leap.1214
Harzing A-W, Adler NJ. Disseminating knowledge: From potential to reality—New Open-Access journals collide with convention. AMLE [Internet]. 2016 [cited 2019 Apr 5];15(1):140–56. Available from: https://doi.org/10.5465/amle.2013.0373
Huisman J, Smits J. Duration and quality of the peer review process: the author’s perspective. Scientometrics [Internet]. 2017 [cited 2019 Apr 4];113(1):633–50. Available from: http://link.springer.com/10.1007/s11192-017-2310-5
International DOI Foundation. The DOI System: ISO 26324 [Internet]. doi. 2016 [cited 2016 May 5]. Available from: http://www.doi.org/index.html
Joanna Briggs Institute. Critical appraisal tools [Internet]. Joanna Briggs Institute. [c2019] [cited 2019 Nov 12]. Available from: https://joannabriggs.org/critical_appraisal_tools
John Wiley & Sons. World Psychiatry: Editorial Board [Internet]. Wiley Online Library. 2019 [cited 2020 Feb 20]. Available from: https://onlinelibrary.wiley.com/page/journal/20515545/homepage/editorialboard.html
John Wiley & Sons. CA: A Cancer Journal for Clinicians: Editorial board [Internet]. Wiley Online Library. c2019 [cited 2019 Oct 22]. Available from: https://onlinelibrary.wiley.com/page/journal/15424863/homepage/editorialboard.html
Komise pro hodnocení výsledků výzkumných organizací a ukončených programů. Zápis z 84. jednání KHV. Výzkum a vývoj v ČR. 2016 [Internet]; 2016 [cited 2016 May 11]. Available from: http://www.vyzkum.cz/FrontClanek.aspx?idsekce=772589
Komm M. Predátoři a upíři aneb horor vědeckých publikací. Deník Referendum [Internet]. 2015 Dec 8 [cited 2016 May 1]; Available from: http://denikreferendum.cz/clanek/21850-predatori-a-upiri-aneb-horor-vedeckych-publikaci
McGlinchey N. When should reviewers receive reminders? [Internet]. Wiley. 2017 [cited 2019 Apr 4]. Available from: https://www.wiley.com/network/researchers/being-a-peer-reviewer/when-should-reviewers-receive-reminders
McKenna M. Students Block UP Rector’s Office And Get Arrested [Internet]. Kosovo 2.0. 2014 [cited 2016 May 5]. Available from: http://www.kosovotwopointzero.com/en/article/983/students-block-up-rectors-office-and-get-arrested
Nadelson S, Nadelson LS. Evidence-Based Practice Article Reviews Using CASP Tools: A Method for Teaching EBP. Worldviews on Evidence-Based Nursing [Internet]. 2014 [cited 2019 Nov 17];11(5):344–6. Available from: https://doi.org/10.1111/wvn.12059
Poynder R. The Open Access Interviews: Dove Medical Press [Internet]. Open and Shut? 2008 [cited 2016 May 5]. Available from: http://poynder.blogspot.com/2008/11/open-access-interviews-dove-medical.html
Ryan F, Coughlan M, Cronin P. Step-by-step guide to critiquing research. Part 2: Qualitative research. Br J Nurs [Internet]. 2007 [cited 2019 Nov 17];16(12):738–44. Available from: https://doi.org/10.12968/bjon.2007.16.12.23726
Schmidt B, Ross-Hellauer T, van Edig X, Moylan EC. Ten considerations for open peer review. F1000Res [Internet]. 2018 [cited 2019 Oct 11];7:969. Available from: https://f1000research.com/articles/7-969/v1
Stoye E. Predatory publisher OMICS fined $50 million by US court for defrauding scientists [Internet]. Chemistry World. 2019 [cited 2020 Jan 29]. Available from: https://www.chemistryworld.com/news/predatory-publisher-omics-fined-50-million-by-us-court-for-defrauding-scientists/3010348.article
U.S. National Library of Medicine. Fact Sheet: MEDLINE® Journal Selection [Internet]. U.S. National Library of Medicine 2016 [cited 2017 February 3]. Available from: https://www.nlm.nih.gov/pubs/factsheets/jsel.html
Ústřední knihovna Vysoké školy báňské - Technické univerzity Ostrava. Co je to otevřený přístup k výsledkům vědeckého výzkumu [Internet]. 2016 [cited 2016 May 1]. Available from: http://knihovna.vsb.cz/open-access/ open-access.htm
Vláda ČR. Metodika hodnocení výsledků výzkumných organizací a hodnocení výsledků ukončených programů (platná pro léta 2013 až 2016) [Internet]. [Brno]: [Vláda ČR]; 2014 [cited 2016 May 5]. 60 p. Available from: http://www.vyzkum.cz/storage/att/471EC8E44A7C3AA09C01B666F1ED6B30/M2013-0815-kor2.pdf
Web of Science Group. Journal Citation Reports: Suppression policy [Internet]. [USA]: Clarivate Analytics; 2019 [cited 2019 Oct 11]. Available from: https://clarivate.com/webofsciencegroup/essays/journal-self-citation-jcr/
Web of Science Group. Web of Science journal evaluation process and selection criteria [Internet]. Web of Science Group. c2019 [cited 2019 Nov 3]. Available from: https://clarivate.com/webofsciencegroup/journal-evaluation-process-and-selection-criteria/

Zdroje ukázek v textu

Araliya. Home [Internet]. Indian Journal of Advanced Nursing. c2014 [cited 2020 Feb 6]. Available from: http://ijan.co.in/
Caister Academic Press. Instructions for Authors (CIMB) [Internet]. Current Issues in Molecular Biology. 2020 [cited 2020 Jan 31]. Available from: https://www.caister.com/cimbauthor
David Publishing Company. China-USA Business Review: Indexing [Internet]. David Publishing Company. c2015 [cited 2020 Feb 4]. Available from: http://www.davidpublisher.org/index.php/Home/Journal/detail?journalid=13&jx=CUBR&cont=indexing
European Medical Group. Authors - EMJ [Internet]. European Medical Journal. c2020 [cited 2020 Jan 29]. Available from: https://www.emjreviews.com/contributors/authors/
Hindawi. Editorial board [Internet]. Parkinson’s Disease. 2020 [cited 2020 Jan 31]. Available from: https://www.hindawi.com/journals/pd/editors/
John Wiley & Sons. Editorial Board [Internet]. CA: A Cancer Journal for Clinicians. 2019 [cited 2020 Jan 31]. Available from: https://acsjournals.onlinelibrary.wiley.com/hub/journal/15424863/homepage/editorialboard.html
Lambert Academic Publishing. Why you should publish with Lap Publishing [Internet]. Lambert Academic Publishing. 2020 [cited 2020 Feb 21]. Available from: https://www.lap-publishing.com/site/why-choose-us/47
Open Access Publisher. Home [Internet]. Vitamins & Minerals. c2020 [cited 2020 Feb 4]. Available from: https://www.omicsonline.org/vitamins-minerals.php
SciMed Central. About the Journal [Internet]. JSM Bioavailability and Bioequivalence. c2016 [cited 2019 Dec 12]. Available from: https://www.jscimedcentral.com/Bioavailability/
Springer Nature. Editorial criteria and processes [Internet]. Nature. c2020 [cited 2020 Jan 29]. Available from: https://www.nature.com/nature/for-authors/editorial-criteria-and-processes
Wolters Kluwer. Login [Internet]. Noise and Health. 2019 [cited 2019 Dec 12]. Available from: http://www.noiseandhealth.org/subscriberlogin.asp?rd=article.asp?issn=1463-1741;year=2018;volume=20;issue=97;spage=217;epage=222;aulast=Lim;type=2
Wolters Kluwer. November-December 2018 | Vol 20 | Issue 97 [Internet]. Noise and Health. 2019 [cited 2019 Dec 12]. Available from: http://www.noiseandhealth.org/
Wolters Kluwer. Home [Internet]. Wolters Kluwer. 2020 [cited 2020 Jan 31]. Available from: https://wolterskluwer.com/
Wolters Kluwer. Medicine [Internet]. Wolters Kluwer. c2020 [cited 2020 Jan 31]. Available from: https://journals.lww.com/md-journal/pages/default.aspx
American Journal of Analytical Chemistry: Indexing [Internet]. Scientific Research: An Academic Publisher. 2020 [cited 2020 Feb 4]. Available from: https://www.scirp.org/journal/AJAC/
Home [Internet]. Neuropsychiatry (London). c2020 [cited 2020 Feb 3]. Available from: http://www.jneuropsychiatry.org/

2 The recommended approach for evaluating a journal

If you want to eliminate the risk of publishing in an untrustworthy journal, it is indispensable that you perform these three steps:

Check objectively verifiable formal criteria of transparency principles and ethics of scholarly publishing. This step includes checking 10 criteria which are described in detail in the following sections. We recommend recording the resulting number of points for each criterion on an evaluation chart.
Perform a content analysis of the journal, i.e. read a couple of the journal’s volumes and, based on your own expert knowledge or with the help of check lists (see section 2.2), evaluate the professional quality of the published articles.
Search on the internet. There are various websites where scholars share experience with publishing. In this way you can gain insight into the quality of the respective journal’s editorial work (see section 2.3). As a part of this evaluation step you should also familiarize yourself with how the databases Web of Science, JCR, and Scopus evaluate the journal.

The following sections describe the individual, above-mentioned steps of the approach for a detailed evaluation of a journal.

2.1 Checking formal criteria

In the first step, you should check ten criteria which can be objectively verified and are based on transparency principles and ethics of scholarly publishing determined by authorities in publication ethics, namely COPE (Committee on Publication Ethics), DOAJ (Directory of Open Access Journals), OASPA (Open Access Scholarly Publishers Association) and WAME (World Association Medical Editors). When evaluating a journal, we recommend using an evaluating chart either in its online version or as an Excel file.

In the following subchapters, it is explained why the respective criteria should be evaluated and how to proceed. We have also included examples of journals that violate the respective criterion. When evaluating these criteria with the help of the evaluation chart, you should proceed as follows. Start by verifying on the journal’s website whether a criterion is met or not (e.g. you can verify the presence of an ISSN by looking at the main page of the website or subpages thereof with information about the journal). When the criterion is met, the journal gains 1 point, while when the criterion is not met (even partly) the journal gains 0 points. A journal with 10 points can be considered fully transparent.

However, in real life scholars often learn that some of the criteria are occasionally violated even by well-established journals. This is why the journal evaluation approach recommended in this study material includes two more evaluation steps: an assessment of the professional quality of the journal and an effort to find information about the journal’s operation and how it solves possible violations of publication ethics.

2.1.1 Access to full text

The first criterion to check is the free access to full texts as the primary goal of Open Access journals. When validating this criterion you should focus on whether a journal in Open Access mode really provides access to the full texts of its articles. Keep in mind that a number of journals provided access to their full texts on the basis of subscription in the past, and only later did they adopt the Open Access mode. Therefore, it is advisable to focus on availability of full texts in the current issue or volume.

Although evaluating this criterion may seem straightforward, unfortunately cases may appear when you hesitate whether you should deem it fulfilled or not. For example, in the journal JSM Bioavailability and Bioequivalence, this criterion will be clearly evaluated as unfulfilled because, after selecting the current issue in the menu on journal’s website, the homepage "About the Journal" loads again instead of the page with articles for download.

However, the evaluation of the journal Noise & Health, may be controversial, because it provides access to its articles in HTML format for free while for a PDF version of articles it charges 20 USD. One person may consider the availability of full text in HTML format to represent a fulfilled criterion while another person might assess the criterion more strictly and expect completely free access to the full text regardless of its format in the open-access mode.

2.1.2 Article processing charges

In the context of article processing charges (APCs), untrustworthy journals are connected with practices such as non-transparent information about APCs or providing information about them only after the fact (a surprise in the form of unexpected invoices). The amount of APCs may also be very low and therefore cannot cover the costs for publishing an article.

Although it was found that the average cost imposed by western publishers to publish an article is approximately 500 EUR, it cannot be considered a universal guideline which distinguishes untrustworthy journals from the others, because the economies of various countries differ. The amount of APCs as a criterion is disputable for one more reason: today there are many untrustworthy journals that require an APC amount similar to that of traditional publishers (see below the example Journal of Immunobiology x Immunobiology).

When assessing the APC criterion you should determine whether the journal states the exact amount of APCs clearly and intelligibly. It is typical of untrustworthy journals that they provide the amount of APCs together with some note – usually on the same page – in which the publisher reserves the right to charge additional fees.

As an example the publishing house Allied Academies may serve (see below). Publishing an article here requires not only paying APCs but also for membership in an unspecified organization with a yearly fee of 75 USD.

2.1.3 Description of peer-review process

Although it is commonly stated that untrustworthy journals usually have a very short peer-review process (a matter of days), in reality this criterion is questionable as well. On one hand, writing a review may take only a few hours, but on the other hand, the total length of the peer-review process may be influenced by searching for a suitable reviewer or by the reviewers being busy. For example, it follows from the e-mail below that if the editor of the journal had not been on holiday, the review process could have been shorter by a whole month.

When evaluating this criterion, you should find out whether a sample of the peer-review process is provided on the journal’s website, so that you have a precise idea about its course. This means finding a page on the journal’s website with information for authors or with the journal’s ethical principles, and there you should look to see whether the journal describes the course of peer review in detail. There are three types of peer-review process:

Double blind	The text is evaluated by at least two reviewers while neither the reviewers nor the author(s) know each other’s identity. This is the most common type of review process.
Single blind	The text is evaluated by at least two reviewers who know the identity of the author(s) and whose identity is hidden from the author(s).
Open	The text is evaluated by at least two reviewers, both the reviewers and author(s) know each other’s identity.

The editorial board is one of the key parts of a journal. It is a decision-making body determining both the content and thematic focus of the journal. It also develops strategies and visions which the journal follows in its publishing. The editorial board’s prestige reflects the quality of the whole periodical. In the context of untrustworthy journals, the editorial board is connected with the criteria mentioned below, though these criteria are generally problematic.

popis recenzního řízení v pokynech pro autory — In this example from the journal *Nature,* a short sample from a comprehensive description of the peer-review process in the instructions for authors is provided.

2.1.4 Affiliation of editorial-board members

The editorial board is one of the key parts of a journal, it is a decision-making body with respect to the content and thematic focus of the journal. It also develops strategies and visions which the journal follows in its publishing. The editorial board’s prestige reflects the quality of the whole periodical. In the context of untrustworthy journals, editorial board is connected with the below-mentioned criteria; however, they are mostly problematic.

Criteria	Questionability of criteria
The same editorial board for a whole portfolio of journals	As there are several thousand journals, it is practically impossible to ascertain whether one editorial board is connected with multiple journals.
Fictitious members of editorial boards	This criterion can only be verified by searching for the board members on the websites of their institutions. Besides the time-consuming nature of verifying this criterion, especially when the editorial board has many members, not all institutions provide information about their employees or students due to personal data protection.
Well-known and successful researchers are included among the editorial-board members without their knowledge.	Not all scholars provide information about their membership on the website of their institution. Reasons for this may vary (e.g. the design of their institution’s website does not allow it, lack of interest on the side of the author, no obligation to provide such information, etc.). Therefore, this criterion can only be verified by contacting the scholar directly. The question remains to what extent are scholars willing to reply to questions regarding their membership on editorial boards.
The editorial board has only a few members or it is not international and its members come mainly from developing countries	In terms of the number of editorial-board members, authorities such as COPE, DOAJ, OASPA and WAME do not set out any standard for whether this criterion is violated or fulfilled. Even the prevalence of people from developing countries on the editorial board and the implied lower quality of editorial work is controversial. Under globalization, an increasing number of representatives from third countries on editorial boards is a natural development. In particular, there are many regional journals whose editorial-board members are mostly from the respective region, yet such journals are not lacking in professional quality.
Affiliation of the editorial-board members is not accurate	This is the only criterion associated with the editorial board which we recommend checking, although we are aware that even this criterion may be problematic. As follows from the sample below, even such a prestigious journal as CA: A Cancer Journal for Clinicians with the highest impact factor (223.679 v r. 2018) does not provide the full affiliation (i.e. institution and state/city) of its editorial-board members but only their institutions. In the case of strict control, this would mean that in terms of formal criteria, this journal would be evaluated as untrustworthy. Nevertheless, we recommend checking this criterion, because only with full affiliation can an author or reader identify an editorial-board member unimpeachably. At the same time, this is also a criterion set by the COPE, DOAJ, OASPA and WAME authorities.

2.1.5 The name of the editor-in-chief included

Following the preceding criterion, you should also check whether the journal provides clear information about its editor-in-chief either on its website or in the journal itself. Like in any other normally operating organization, journals must have a person responsible for certain processes. Whether an article is accepted or declined is decided by the editor-in-chief of the journal.

2.1.6 Unambiguous identification of the publisher

A common practice of untrustworthy journals is that instead of providing unambiguous information about their publisher, they either do not mention the publisher at all or they replace it with the name of the journal. Being able to unambiguously identify the publisher is vital because it helps the reader or potential author learn who owns the journal. In this way authors immediately get an idea about the expected quality of the editorial work, especially when the owner is a traditional publishing house such as Elsevier, Springer, etc.

When verifying this criterion, the publisher can commonly be found in the footer of the website with information about copyright. In the example below, the website footer of the journal Medicine includes a statement that the publisher is Wolters Kluwer Health, Inc. The name of the publisher serves also as a link to its website.

By contrast, the website of the journal Neuropsychiatry mentions in the footer only the name of the journal and next to it an address without any addressee.

2.1.7 The journal provides an ISSN on its website and the ISSN is valid

The ISSN (International Standard Serial Number) provides important information about a journal, because it is an unambiguous identifier which prevents readers as well as potential authors from confusing journals with similar titles. When verifying this criterion, first the ISSN of the journal must be found and then checked on the ISSN Portal (https://portal.issn.org/). One should verify whether the journal can be retrieved on this portal according to its ISSN and whether the information recorded in the ISSN corresponds to that included on the website of the journal (e.g. information about the publisher, the frequency of the journal’s publication, etc.).

When trying to find the ISSN, you need to inspect the website of the journal very thoroughly, because it may not be easy to find at first sight.

You can see this in the following examples. While the journal World Psychiatry provides its ISSN right in the heading of its website…

… the journal Nature placed this information just above the footer of its website and only in small print.

Sometimes finding the ISSN may be complicated. For example, the journal Cell does not provide its ISSN either on itshomepage, or on any subpages in the About section, where the reader would commonly expect such information, but instead only on the page with information about subscriptions.

2.1.8 Accurate information about the journal’s citation metrics in the Journal Citation Reports and Scopus

One of the most distinctive features of untrustworthy journals is an effort to deliberately confuse authors by falsely proclaiming metrics that have allegedly been assigned to the journal in the databases Journal Citation Reports (JCR) and Scopus. While the JCR database calculates the impact factor for journals indexed there, Scopus calculates three metrics for their journals - CiteScore, SNIP and SJR (we discuss metrics in a separate study material).

Untrustworthy journals present on their websites the value of certain metrics, the names of which are similar to the above-mentioned metrics (e.g. Global Impact Factor, Journal Impact Factor). The aim is to lure authors into publishing their articles in an untrustworthy journal and collect article processing charges. With regard to evaluation of scholarly achievements and the author’s prestige, publishing in a journal with impact factor in particular is a great motivation. Therefore, untrustworthy journals take advantage of this and try to mislead authors by providing metrics with names similar to the metrics provided by JCR and Scopus on their websites.

In this case verifying this criterion is very easy, because any time you encounter a journal providing metrics of a similar name like impact factor, CiteScore, SNIP and SJR, just search for the journal in JCR or Scopus and check whether the journal is listed there and has a current value of the respective metrics. Here you need to remember that the values of the metrics are published with a certain delay. For example, impact factor is commonly published in June or July. Therefore, the most recent value in the first half of 2020 will be for the year 2018, and in the second half for 2019.

Journals commonly provide citation metrics on their website or in the section about indexation in databases. In the below-mentioned example with the journal China-USA Business Review, the metrics are provided in the section Indexing. In this case the metric’s name JIFACTOR may indicate either an effort to mislead authors by deliberately providing metrics with a name similar to the official one or providing false metrics (for more information about false metrics, see below).

Because the text with the name of the metrics is not a link to a website with more information, we need to make sure whether the journal is listed in JCR and whether these metrics are genuine. From the figure below, it is obvious that the journal is not indexed in JCR and therefore did not meet the criterion, because it is either lying about being indexed in JCR or provides false information about the metrics.

Another example of a journal falsely claiming to have citation metrics is the Indian Journal of Advanced Nursing, which declares that it has an impact factor of 2.002 (this is a moving bar where also IBI Factor is provided - see below the section about untrustworthy metrics).

When trying to verify the value of these metrics in JCR, we learnt that this journal is not indexed in this database.

A different situation is shown in the example of the journal Vitamins & Minerals with the metric Journal Impact Factor. This may either be an attempt to imitate the name of the real impact factor from JCR with the aim to confuse authors or providing a misleading metric.

Contrary to the preceding example, here the name of the metric is a link to a website (see below) with a description of its calculation method. Although the calculation itself does not differ from the real impact factor, the basis for calculating this metric are citations from Google Scholar Citation Index database according to the information provided (see below). The journal violated the criterion Accurate information about citation metrics in Journal Citation Reports and Scopus by the fact that it used the official name of the real impact factor for its own metric, namely Journal Impact Factor. At the same time, the journal also did not comply with the criterion Providing misleading citation metrics because basing a metric on a system that uses even citations from presentations is questionable.

2.1.9 Accurate information about indexation in Web of Science and Scopus

Authors are motivated to publish in journals indexed in the Web of Science (WoS) and Scopus because their evaluation is based on that. However, one should always check whether a claim about being indexed in one of these databases is true (medical professionals may also consider MEDLINE PubMed).

A journal’s claim about being indexed in WoS or Scopus can be verified either in the database directly or according to the content of both databases available publicly. In the case of WoS, you can use the online search engine Master Journal List. The list of Scopus indexed journals can be downloaded on the website Elsevier in the section Scopus Content (Solutions > Scopus > How Scopus works > Content coverage > Titles on Scopus).

The necessity to carefully check information on journals’ websites is shown on the example of American Journal of Analytical Chemistry, which gives Web of Science (Clarivate Analytics) at the first place in the section Indexation. This is complemented with data about citation rates and a link to a preview from WoS documenting this indexation.

This text may give you the impression that the journal is indexed in Web of Science, although this is not the case (see below the preview from WoS). In reality, this preview from WoS only proves that the Journal of Analytical Chemistry was cited by other journals in WoS.

2.1.10 Providing misleading citation metrics

Untrustworthy journals try to gain the appearance of being prestigious by providing various citation metrics which have nothing in common with impact factor, CiteScore, SNIP and SJR from JCR and Scopus. Most frequent are metrics combining an adjective with the term impact factor (e.g. Global Impact Factor, General Impact Factor, IBI Factor etc.). The problem of these metrics is their lack of transparency. Their method of evaluation is either not published or is based - even partially - on a subjective evaluation of journals.

This means that if you encounter a different metric than impact factor, CiteScore, SNIP and SJR from JCR and Scopus, you should try to learn more about the metric. In the chapter about citation metrics in JCR and Scopus the journal Vitamins & Minerals with the metric Journal Impact Factor served as an example. We also explained the method by which it is calculated, which includes citation data from Google Scholar Citations. With regard to the fact that Google Scholar Citations does not distinguish whether an article is cited by properly published texts or by various other documents such as presentations, drafts, etc., the data about citation rates can be considered questionable and as a result the entire metric is irrelevant.

Another example of a misleading citation metrics is IBI factor provided by the Indian Journal of Advanced Nursing on its website which was mentioned in the part about citation metrics in JCR and Scopus.

After searching on the internet the method of calculating IBI factor can be found (see a sample below). However, we find this metric controversial.

The method of calculating IBI factor is as follows: first the journal is awarded points for criteria divided into sections A to E. Then the resulting IBI Factor is calculated according to the equation:

IBI Factor can be considered a misleading metric for several reasons:

One question is whether it is possible to evaluate almost 5,000 journals on a yearly basis in the way described above, especially due to personal issues.
The metric is not transparent. For example, already in the first section A, it is not clear why the sum of journals should be multiplied by 10 and why the number of this year’s articles should be divided by the number of articles published the last year. In the other sections, it is not clear whether a journal receives 10 points only if it complies with all criteria, or if the points in the given section are divided by the number of criteria and journals then receive the respective number of points for the given criterion (e.g. if a journal complies with only one criterion out of four in the section B, does it receive 0 points or 2.5 points?).
The criteria are controversial. For example, because there is no law or norm ordering that a journal should be a weekly, monthly, or yearly periodical, the criterion in section B evaluating the number of issues published last year is irrelevant. Similarly, journals are not obliged to have a website, therefore the criteria 2, 3, 5, 6, and 8 are also irrelevant.

In this way other criteria in the following sections could also be proven wrong or useless. Nevertheless, for the purpose of this material, the notes provided above suffice to demonstrate that it is vital to check metrics other than those from JCR and Scopus thoroughly.

2.2 Evaluating the professional quality of a journal

The second step during journal evaluation must be an analysis of the journal’s content, one focused mainly on its professional quality rather than on bad grammar or spelling as commonly associated with untrustworthy journals. Many authors have published in untrustworthy journals because they did not assess its professional quality, despite the fact that they can use one of the following tools for examining the quality of text processing besides their professional knowledge.

For example, the Joanna Briggs Institute created critical appraisal tools for various types of studies with questions checking comprehensibility, attention to detail, objectivity, and verifiability of the research results. Each of these questions is also complemented with an explanation of what specifically the question is aiming at in the article.

Similar aid is provided by checklists from the Critical Appraisal Skills Programme containing questions about the clarity of aims and results of the research, the suitability of the chosen research method and the gathering of results, as well as compliance with ethical principles of research. These checklists help authors to assess the quality of the content of randomly chosen articles not only based on their specialization but also with the help of a standardized method. In this way authors can get an idea about the quality of editorial work and the journal’s requirements for the professional quality of the published articles.

2.3 Learning about the journal’s operating procedures

The third step when evaluating a journal is to try learning about the way the journal operates. In the case of journals with open peer review, it is necessary to read some peer reviews and the communication between the reviewers and the editor, as this provides the most accurate information about the review process and the editor’s reasons for accepting an article.

Since such open peer review is rarely employed by journals, we depend on information from secondary sources in this step. These sources may include platforms such as ResearchGate, Academia.edu, Retractionwatch.com and Retractiondatabase.org, where researchers share their experience with publishing. Naturally, the information obtained on these platforms needs to be assessed critically. For example, one cannot conclude that the journal as a whole or its publisher are untrustworthy after seeing one article with forged or otherwise manipulated results. Instead, one should check whether, for example, the editorial board of the journal properly retracted the article afterwards. Or in the case that one uses platforms such as ResearchGate, one must pay attention to whether the discussants support their claims with evidence.

Indexation of a journal in Journal Citation Reports (JCR) and Scopus may also indicate how reliable the journal is. This is because if a journal fails to meet JCR’s and Scopus’s evaluation criteria or exhibits non-standard citation practices, the journal is excluded from their interface accessible to users. Therefore, users should be interested in the reasons why the indexation of a journal was interrupted or terminated. JCR provides these reasons in a brief form in its title suppression list and Scopus in its discontinued titles list. If necessary, one can try to reconstruct their evaluation approach. When evaluating a journal indexed in JCR, one needs to focus on possible non-standard citation practices of the journal (a significant increase or fall in the number of citations, self-citations, and articles, majority of citations from a small group of journals) as well as on compliance with 28 criteria from JCR. In a journal indexed in Scopus, the following data are checked within the journal’s field: the self-citation rate, the total citation rate, CiteScore citation metrics, number of articles, number of full-text clicks and abstract usage.

For example, in JCR’s title suppression list from 2019, the International Journal of Civic Engineering is listed with the note "Self", which indicates that the value of its impact factor is influenced by a high number of self-citations.

If you look at the specific data about this journal in JCR, you will learn that while its impact factor value ranges from 0.372 to 0.695 (average 0.497), its impact factor value without self-citation varies between 0.150 and 0.382 (with an average of 0.254). With regard to the method of calculating the impact factor, this means that approximately half of the citations of articles published in the International Journal of Civil Engineering were self-citations. The administrators of JCR considered such a high number of self-citations too significant an influence on the impact factor and therefore excluded the journal from their list.

On the other hand, Scopus has the criterion "Number of articles". Here it checks whether a journal published half the number of articles or less than other journals from the same field. However, it does not provide reasons for why the bar was set to half and not a different percentage. Moreover, this criterion ignores the fact that due to the varying publication schedules of journals, the number of articles published may differ as well.

4 Conclusion

As it follows from this material, today it is an absolute must to carefully evaluate the quality of publications by the respective publisher. The ability to identify untrustworthy journals is becoming a natural part of the publication process. Before authors submit their article, they should always consider whether the publisher and its work display characteristics of untrustworthy publishing. When selecting a journal, authors must pay attention to whether all crucial information is provided and in a transparent way (e.g. contact details, clearly set financial policy of the publisher, verifiability of the professional qualification of editorial-board members, clearly set course of peer-review process, provision of only relevant citation metrics, etc.). Moreover, one cannot rely only on checking the formal criteria of a journal, rather one must also focus on its professional quality as well as the way the journal operates.

3 Author mills - dubious practices with monographs

Unfair publishing practices have affected even the field of monograph publication. A typical example is the so-called author mill or academic author mill. This term refers to publishers’ practices where the business model consists of producing a large number of titles in very small editions - the very opposite of well-established publishing houses which focus on a limited number of good-quality authors and publish their works in thousands of copies. The target group of these publishers are usually postdoctoral researchers whom such publishers actively try to persuade to publish their doctoral theses with them.

Recognizing bogus publishers is more difficult compared to identification of untrustworthy journals, because publishers of books do not commonly provide detailed information on their websites regarding the description of the review process, editorial board, etc. Therefore, one should pay attention to various details and, above all, consult one’s colleagues for what experiences they might have had with the publisher (e.g. whether there was a review process, what care was devoted to final language editing, etc.).