An integral part of the duties of researchers in scientific and research institutions is not only to publish their research results but to report the results for the purpose of evaluating their research. The evaluation process uses mathematical and statistical methods to gain a quantitative evaluation of research results. These methods process data provided by authors about their publications and the citations of their works in the Journal Citation Reports (JRC) and Scopus databases.
In the JCR database, the impact factor and the Article Influence Score (AIS) are the key metrics used by the Research, Development and Innovation Council for their bibliometric analysis since 2017.
Both databases count the Hirsch index, also called the h-index, to evaluate the quality of a given author’s publication activity.
Journal Citation Reports | Subject of metrics | Scopus | Subject of metrics |
|---|---|---|---|
Impact factor | Journal | CiteScore | Journal |
Article Influence Score | Article | SNIP | Journal |
H-index | Author | SJR | Journal |
h-index | Author |
The impact factor is one of the tools for quantitative evaluation of research results. This factor measures the average number of citations of articles published in a given journal during the previous two years by journals indexed in the JCR database (Fig. 1).
While the impact factor is a time-tested indicator evaluating the quality of journals, it is completely unsuitable for assessing the quality of individual articles or researchers themselves.
Journals from different fields cannot be compared based on their values of impact factor, because every field has different publication conventions and citation potential.
The language of the publications may also influence the value of the impact factor. A journal publishing in another language than English is commonly less cited despite the fact that it may be a periodical of high quality.
The type of documents the journal commonly publishes, e.g. reviews, research articles or case reports, is another variable that affects the value of the impact factor.
The type of access to full texts also affects the impact factor. Open-access journals have a larger citation potential, because their articles reach a readership more quickly and without any barriers, whether financial or technical.
A two-year period for calculating the impact factor may be too short, because to prepare publications citing the respective article may take longer in some fields. For that reason a five-year impact factor was added to help the user gain a more precise picture about the journal’s citation rate.
The value of the impact factor is very often influenced by a small number of highly-cited papers in a journal where the majority of articles does not reach the value of the impact factor calculated (see Figures 2 and 3).
AIS is a metric which is subject to a bibliometric analysis performed by the Research Development and Innovation Council (RVVI). Compared to impact factor, AIS is a more sophisticated metric, because it calculates the value of the average influence of every article in a journal for a five-year period (roughly similar to five-year impact factor). A score greater than 1 indicates that each article in a specific journal has an above-average influence, while a score smaller than 1 means that articles in a journal have below-average influence. AIS values may be found in the JCR database.
SJR is a sophisticated metric whose main principle is similar to Google Page Rank (a number assigned by Google to every URL that expresses the credibility and importance of the given website). SJR takes into consideration the importance of citations (i.e. not all citations are equally important). The main principle for calculating SJR is an algorithm that measures how many times the journal was cited by other journals and what importance these individual citations have. The importance of citations is derived from the prestige of the journal from which the citations come. The aim of SJR is to facilitate a relevant comparison of journals’ quality from different fields. In the bibliometric analysis carried out by the Research Development and Innovation Council, this metric is a key tool for evaluating research organisations in the Czech Republic.
CiteScore is a metric calculated similar to the impact factor, with the difference that the number of articles and their citations are taken from the previous three years and not two as in the case of the impact factor.
SNIP is a citation metric designed by the late Henk F. Moed from the Centre for Science and Technology Studies (CTWS) in Leiden in the Netherlands. It expresses the ratio of an average citation of an article in a given journal to the citation potential of the field (the probability that a document will be cited in the given field). The SNIP metric helps authors determine which journals in their field are of high quality. It may also serve a key indicator when deciding where to publish.
SNIP is calculated with a simple formula:
SNIP
=
IPP(journal's impact per publication)
DCP(Database Citation Potential)
The IPP is a value computed in a similar way like impact factor, but IPP works with data from the last three years (not just two as in the case of impact factor).
IPP2016
=
number of citations in 2016 to articles between 2013-2015 in journal XY
number of all articles published between 2013-2015 in journal XY
The result of IPP provides information on how many times every article published in the XY journal in the years 2013–2015 was cited in 2016.
DCP is an abbreviation for the citation potential of a journal in its respective field. The value of DCP is calculated as the ratio of the citation potential of a journal in the database to the median value of a journal in the same field. Obtaining these details is very demanding, and it must be carried out with the help of computer technology.
DCP2016
=
citation potential of journal XY in database
median value of journal XY in its field in database
The median of the journal’s field is the median value of citation potential in the respective field.
The Hirsch index (h-index) was designed by the physicist Jorge E. Hirsch from the University of California. This indicator focuses primarily on the publication activity of individual authors but can also be calculated for a journal. The value of the h-index for a single author can differ in different databases due to differences in their content.
H-index indicates how many articles received a higher number of citations than their order number (publications are in a descending order according to the number of citations they received). H-index equals the order number of the article which received the same or a higher number of citations than its order number in the list – see Fig. 5.
Callaway E. Beat it, impact factor! Publishing elite turns against controversial metric. Nature. 2016;535(7611):210-211. doi:10.1038/nature.2016.20224. Accessed April 16, 2019.
Colledge L, de Moya‐Anegón F, Guerrero‐Bote V, López‐Illescas C, El Aisati M, Moed H. SJR and SNIP: two new journal metrics in Elsevier’s Scopus. Serials: The Journal for the Serials Community. 2010;23(3):215-221. doi:10.1629/23215. Accessed February 6, 2019.
Grzybowski A. Impact factor – benefits and limitations. Acta Ophthalmologica. 2015;93(3):201-202. doi:10.1111/aos.12579. Accessed March 3, 2019.
Kreiner G. The Slavery of the h-index—Measuring the Unmeasurable. Frontiers in Human Neuroscience. 2016;10. doi:10.3389/fnhum.2016.00556. Accessed March 20, 2019
Kurmis AP. Understanding the limitations of the journal impact factor. J Bone Joint Surg Am. 2003;85-A(12):2449-2454. Accessed March 20, 2019.
Schreiber M. An empirical investigation of theg-index for 26 physicists in comparison with theh-index, theA-index, and theR-index. Journal of the American Society for Information Science and Technology. 2008;59(9):1513-1522. doi:10.1002/asi.20856. Accessed April 11, 2019.
Tregoning J. How will you judge me if not by impact factor?Nature. 2018;558(7710):345-345. doi:10.1038/d41586-018-05467-5. Accessed April 11, 2019.
Van Noorden R. Controversial impact factor gets a heavyweight rival. Nature. 2016;540(7633):325-326. doi:10.1038/nature.2016.21131. Accessed March 19, 2019.
Vanclay JK. Bias in the journal impact factor. Scientometrics. 2009;78(1):3-12. doi:10.1007/s11192-008-1778-4. Accessed February 12, 2019.
Description of Scimago Journal Rank Indicator. https://clarivate.com/essays/impact-factor/. Published 2007. Accessed March 17, 2019.
Clarivate Analytics. The Clarivate Analytics Impact Factor. https://clarivate.com/essays/impact-factor/. Accessed February 27, 2019
CWTS Journal Indicators – Methodology. http://www.journalindicators.com/methodology. Accessed February 6, 2019.
How are CiteScore metrics used in Scopus? – Scopus: Access and use Support Center. https://service.elsevier.com/app/answers/detail/a_id/14880/supporthub/scopus/. Accessed March 18, 2019.
InCites Help. Journal Impact Factor. http://help.incites.clarivate.com/inCites2Live/indicatorsGroup/aboutHandbook/usingCitationIndicatorsWisely/jif.html. Accessed March 3, 2019.
InCites Journal Citation Reports Help. Article Influence Score. http://help.incites.clarivate.com/incitesLiveJCR/glossaryAZgroup/g4/7790-TRS.html. Accessed April 3, 2019.
SNIP. Elsevier. https://journalinsights.elsevier.com/journals/0301-0104/snip. Accessed April 16, 2019.