Introduction

An integral part of researchers’ duties in a research institution is not only to publish their results but also to report the results for the purpose of evaluating their research. For this purpose mathematical and statistical methods are applied to gain a quantitative evaluation of research results. These methods process data provided by authors about their publications and the citation rates of their works from the Journal Citation Reports (JRC) and Scopus databases.

In the JCR database, impact factor serves for this purpose, and since 2017, another key metric, Article Influence Score (AIS), has been used by the Research, Development and Innovation Council.

Both databases count the Hirsch index, also called the h-index, to evaluate the quality of a given author’s publication activity.

Impact factor – definition

Impact factor is one of the tools for quantitatively evaluating research results. This factor measures the average citation rate of articles in a given journal for a specific period of time. It is defined as follows: the ratio of citations recorded in the given (evaluated) year to all articles published in a given journal in the preceding two years to the total number of all those articles (Fig. 1). Impact factor values may be found in the JCR database.

Limitations and Biases of the Impact Factor

• While the impact factor is a time-tested indicator evaluating the quality of journals, it is completely unsuitable for assessing the quality of individual articles or researchers themselves.

• Journals from different fields cannot be compared based on their values of impact factor, because every field has different publication conventions and citation potential.

• The language of the publications may also influence the value of the impact factor. A journal publishing in another language than English is commonly less cited despite the fact that it may be a periodical of high quality.

• The type of documents the journal commonly publishes, e.g. reviews, research articles or case reports, is another variable that affects the value of the impact factor.

• The type of access to full texts also affects the impact factor. Open-access journals have a larger citation potential, because their articles reach a readership more quickly and without any barriers, whether financial or technical.

• A two-year period for calculating the impact factor may be too short, because to prepare publications citing the respective article may take longer in some fields. For that reason a five-year impact factor was added to help the user gain a more precise picture about the journal’s citation rate.

• The value of the impact factor is very often influenced by a small number of highly-cited papers in a journal where the majority of articles does not reach the value of the impact factor calculated (see Figures 2 and 3).

Article Influence Score (AIS)

AIS is a metric which is subject to a bibliometric analysis performed by the Research Development and Innovation Council (RVVI). Compared to impact factor, AIS is a more sophisticated metric, because it calculates the value of the average influence of every article in a journal for a five-year period (roughly similar to five-year impact factor). A score greater than 1 indicates that each article in a specific journal has an above-average influence, while a score smaller than 1 means that articles in a journal have below-average influence. AIS values may be found in the JCR database.

SCImago Journal Rank (SJR)

SJR is a sophisticated metric whose main principle is similar to Google Page Rank (a number assigned by Google to every URL that expresses the credibility and importance of the given website). SJR takes into consideration the importance of citations (i.e. not all citations are equally important). The main principle for calculating SJR is an algorithm that measures how many times the journal was cited by other journals and what importance these individual citations have. The importance of citations is derived from the prestige of the journal from which the citations come. The aim of SJR is to facilitate a relevant comparison of journals’ quality from different fields. In the bibliometric analysis carried out by the Research Development and Innovation Council, this metric is a key tool for evaluating research organisations in the Czech Republic.

CiteScore

CiteScore is a metric calculated in the same way as impact factor with the difference that the number of articles and their citations are taken from the last three years and not two as in the case of impact factor.

Source Normalized Impact per Paper (SNIP)

SNIP designed by Henk Moed from the Centre for Science and Technology Studies (CTWS) is another indicator in Scopus. It expresses the ratio of an average citation of an article in a given journal to the citation potential of the field (the probability that a document will be cited in the given field). The SNIP metric helps authors determine which journals in their field are of high quality. It may also serve a key indicator when deciding where to publish.

SNIP is calculated with a simple formula:

The IPP is a value computed in a similar way like impact factor, but IPP works with data from the last three years (not just two as in the case of impact factor).

The result of IPP provides information on how many times every article published in the XY journal in the years 2013–2015 was cited in 2016.

DCP is an abbreviation for the citation potential of a journal in its respective field. The value of DCP is calculated as the ratio of the citation potential of a journal in the database to the median value of a journal in the same field. Obtaining these details is very demanding, and it must be carried out with the help of computer technology.

The median of the journal’s field is the median value of citation potential in the respective field.

H-index

Hirsch index (h-index) is an indicator assessing the publication activity of individual authors or groups of authors (e.g. within an institution).

H-index is calculated for journals as well, because it may be determined for any set of records. However, this metric is primarily connected with assessing the publication performance of an author as an individual.

Note, that the value of h-index may differ based on which database you use for the computation, because not every database records the same journals and their volumes.

H-index indicates how many articles received a higher number of citations than their order number (publications are in a descending order according to the number of citations they received). H-index equals the order number of the article which received the same or a higher number of citations than its order number in the list – see Fig. 5.

Reference list

1. Callaway E. Beat it, impact factor! Publishing elite turns against controversial metric. Nature. 2016;535(7611):210-211. doi:10.1038/nature.2016.20224. Accessed April 16, 2019.

2. Colledge L, de Moya‐Anegón F, Guerrero‐Bote V, López‐Illescas C, El Aisati M, Moed H. SJR and SNIP: two new journal metrics in Elsevier’s Scopus. Serials: The Journal for the Serials Community. 2010;23(3):215-221. doi:10.1629/23215. Accessed February 6, 2019.

3. Grzybowski A. Impact factor - benefits and limitations. Acta Ophthalmologica. 2015;93(3):201-202. doi:10.1111/aos.12579. Accessed March 3, 2019.

4. Kreiner G. The Slavery of the h-index—Measuring the Unmeasurable. Frontiers in Human Neuroscience. 2016;10. doi:10.3389/fnhum.2016.00556. Accessed March 20, 2019

5. Kurmis AP. Understanding the limitations of the journal impact factor. J Bone Joint Surg Am. 2003;85-A(12):2449-2454. Accessed March 20, 2019.

6. Schreiber M. An empirical investigation of theg-index for 26 physicists in comparison with theh-index, theA-index, and theR-index. Journal of the American Society for Information Science and Technology. 2008;59(9):1513-1522. doi:10.1002/asi.20856. Accessed April 11, 2019.

7. Tregoning J. How will you judge me if not by impact factor?Nature. 2018;558(7710):345-345. doi:10.1038/d41586-018-05467-5. Accessed April 11, 2019.

8. Van Noorden R. Controversial impact factor gets a heavyweight rival. Nature. 2016;540(7633):325-326. doi:10.1038/nature.2016.21131. Accessed March 19, 2019.

9. Vanclay JK. Bias in the journal impact factor. Scientometrics. 2009;78(1):3-12. doi:10.1007/s11192-008-1778-4. Accessed February 12, 2019.

10. Description of Scimago Journal Rank Indicator. https://clarivate.com/essays/impact-factor/. Published 2007. Accessed March 17, 2019.

11. Clarivate Analytics. The Clarivate Analytics Impact Factor. https://clarivate.com/essays/impact-factor/. Accessed February 27, 2019

12. CWTS Journal Indicators - Methodology. http://www.journalindicators.com/methodology. Accessed February 6, 2019.

13. How are CiteScore metrics used in Scopus? - Scopus: Access and use Support Center. https://service.elsevier.com/app/answers/detail/a_id/14880/supporthub/scopus. Accessed March 18, 2019.

14. InCites Help. Journal Impact Factor. http://help.incites.clarivate.com/inCites2Live/indicatorsGroup/aboutHandbook/usingCitationIndicatorsWisely/jif.html. Accessed March 3, 2019.

15. InCites Journal Citation Reports Help. Article Influence Score. http://help.incites.clarivate.com/incitesLivejcrglossaryAZgroup/g4/7790-TRS.html. Accessed April 3, 2019.

16. SNIP. Elsevier. https://journalinsights.elsevier.com/journals/0301-0104/snip. Accessed April 16, 2019.