Publication Crimes and How to Fight Them ŠTĚPÁN ČADA What are the signs of quality science? uWhat makes a publication trustworthy enough for you to cite it? uWhat makes a scientist reliable enough for you to trust his opinions on the subject? uWhat makes a scientific project solid enough for you to collaborate? uWhat makes a sufficient publication basis to start your own research? uWhat determines whether a project gets funded? uWhat determines whether a paper gets published? u What are the signs of quality science? uWhat makes a publication trustworthy enough for you to cite it? uWhat makes a scientist reliable enough for you to trust his opinions on the subject? uWhat makes a scientific project solid enough for you to collaborate? uWhat makes a sufficient publication basis to start your own research? uWhat determines whether a project gets funded? uWhat determines whether a paper gets published? u uScience as we know it is largely based on trust (among scientists themselves but also between scientists and society). The trust however needs to be protected. u What are the signs of quality science? uScientific awards? uFunding? uPublications in high-impact journals? uMedia coverage? uAuthor’s career? uAffiliation with prestigious university/institute? uNumber of publications? uNumber of citations? uTransparency (availability of raw data, open peer reviews…)? uIndexing on PubMed/WoS/Scopus? uPeer review? What are the signs of quality science? uGregg Semenza uJohns Hopkins’ Institute for Cell Engineering uNobel prize in physiology, 2019 uMore than 50 publications questioned online based on problematic image data u9/2022: 4 papers in PNAS retracted (together cited more than 750x) uCurrently (10/2024) total of 13 retractions. uhttps://www.nature.com/articles/d41586-022-03032-9 uhttps://retractionwatch.com/2023/10/02/nobel-prize-winner-gregg-semenza-tallies-tenth-retraction/ u A person with a beard and glasses Description automatically generated uScientific awards? uFunding? uPublications in high-impact journals? uMedia coverage? uAuthor’s career? uAffiliation with prestigious university/institute? uNumber of publications? uNumber of citations? uTransparency (availability of raw data)? uIndexing on PubMed/WoS/Scopus? uPeer review? What are the signs of quality science? uAlzheimer disease research uSimufilam u8/2021: NIH received a report about potential data manipulation in the clinical testing of new drug Simufilam, which lead to federal investigation of Cassava Sciences company which owns it. u10/2023: Co-developer of the drug, Hoau-Yan Wang, was found guilty of research misconduct, based on image data manipulation. uhttps://www.science.org/content/article/co-developer-cassava-s-potential-alzheimer-s-drug-cited-eg regious-misconduct uSylvain Lesné u7/2022: Signs of image data manipulation exposed also in one of the most-cited works in the AD field (Lesné et al., 2006). uIt is estimated that tens of millions USD from NIH grants and 16 years of research have been wasted. uhttps://www.science.org/content/article/potential-fabrication-research-images-threatens-key-theory -alzheimers-disease u u u u u What are the signs of quality science? What are the signs of quality science? https://retractionwatch.com/2023/07/24/meet-the-author-who-has-published-more-than-500-letters-to-t he-editor-in-a-year/ •“Viroj Wiwanitkit has published 543 items indexed in PubMed in the last 12 months, the vast majority of them letters to the editor. Most of Wiwanitkit’s letters with colleagues appear to be only a single paragraph. Many concern COVID-19 and vaccinations, but the catalog includes letters about monkeypox, knee replacement surgery, bipolar disorder, even ChatGPT.” A person with black hair and a yellow shirt Description automatically generated Viroj Wiwanitkit Joseph Ayobabalola University in Nigeria What are the signs of quality science? https://www.nature.com/articles/d41586-018-06185-8 •“We searched Scopus for authors who had published more than 72 papers (the equivalent of one paper every 5 days) in any one calendar year between 2000 and 2016, a figure that many would consider implausibly prolific. We found more than 9,000 individuals, and made every effort to count only ‘full papers’ — articles, conference papers, substantive comments and reviews — not editorials, letters to the editor and the like.” • •“The number of hyperprolific authors (after our exclusions) grew about 20-fold between 2001 and 2014, and then levelled off (…). Over the same period, the total number of authors increased by 2.5-fold.” A diagram of a number of authors Description automatically generated with medium confidence Traditional publishing system is failing uInsufficient motivation of reviewers uPeer-reviewing is time-consuming and traditionally not honored by journals. uFinancial conflict of interest of publishing houses uMore papers in shorter time → more money. uDisproportionate open-access publishing fees uhttps://www.spectrumnews.org/news/imaging-journal-editors-resign-over-extreme-open-access-fees/ uInsufficient motivation of publishers and institutions to investigate and retract problematic publications. uInvestigation of research ethics is fully in hands of the affiliated institutions, which often leads to conflicts of interest (reputation loss, prominent position of involved researchers). Investigation rarely leads to punishment of involved researchers. u“Publish or perish” vs. replication crisis (replication studies and negative data are rarely published) uReporting problematic data and ethical misconducts carries high personal and career risks but at the same time is not rewarded by publishers or institutions. u u u u u Problematic publications are not effectively eliminated from the scientific record. uRelative number of retracted publications does not reflect the yearly increase in number of all published articles. “(…)we estimate — on the basis of evidence from surveys, studies and reports from sleuths — that one in 50 papers would meet at least one of the criteria for retraction from the Committee on Publication Ethics, a non-profit collective in Eastleigh, UK. These include “clear evidence that the findings are unreliable”, whether because of falsified data, plagiarism, faked peer review or just ‘major error’, which might involve contaminated cell lines or another non-fraudulent problem. Yet the rate of retraction is still under 0.1%.” Ivan Oransky, founder of Retraction Watch, 2022. https://www.nature.com/articles/d41586-022-02071-6 uInsufficient motivation of publishers/editors uInsufficient communication/unwillingness to verify critique by readers but also to publish corrections suggested by authors(!). uToo long periods of investigation (years). Problematic publications are not effectively eliminated from the scientific record. uEven when false publications are retracted, they can still be read and cited. u https://retractionwatch.com/the-retraction-watch-leaderboard/top-10-most-highly-cited-retracted-pap ers/ Before/after retraction Problematic publications are not effectively eliminated from the scientific record. 2023 2024 We identified 478 retracted articles, 220 (46%) of which were cited at least once. We contacted 1297 corresponding authors of the papers that cited these articles, 417 (30%) of whom responded to our survey and were included in the final analysis. The median number of authors in the analyzed articles was five, and the median elapsed time from retraction to citation was 3 yr. Most of the corresponding authors (89%) were unaware of the retracted status of the cited article, mainly because of inadequate notification of the retraction status in journals and/or databases and the use of stored copies. uEven when false publications are retracted, they can still be read and cited. u Problematic publications are not effectively eliminated from the scientific record. uPaolo Macchiarini uKarolinska Institutet (KI), Sweden uExperimental transplantation of artificial tracheas in combination with stem cells (2011-2012) u7 out of 8 operated patients died u“One patient died suddenly when the implant caused massive bleeding just 4 months after it was implanted; the two others survived for 2.5 and nearly 5 years, respectively, but suffered painful and debilitating complications before their deaths.”[1] u2016 – guilty of research misconduct (KI), and then few more times: u“The articles contain fabricated and distorted descriptions of the patients’ conditions before and after the operations. Justification is lacking for treatment of the patients on the grounds of so-called vital indication (when a given treatment is the last resort for survival), and one misses reference to relevant animal experiments which must precede human studies that involve unproven methods. Furthermore, ethical approvals are lacking, as are appropriate informed consents.”[2] u2023 – sentenced in Sweden to prison for 2,5 years for causing bodily harm to his patients u u [1] https://www.science.org/content/article/transplant-surgeon-gets-prison-sentence-failed-stem-cell-tr eatments [2] https://news.ki.se/seven-researchers-responsible-for-scientific-misconduct-in-macchiarini-case A person in a suit and tie Description automatically generated When publication system fails, lives are at stake uDespite the long-standing evidence of data falsification and research misconduct ruling (2016), the articles in Lancet were only marked with an Expression of concern in February 2023 and retracted after public backlash in October 2023. uhttps://retractionwatch.com/2023/10/27/lancet-retracts-two-more-papers-by-convicted-surgeon-paolo- macchiarini/ u Antivax movement as a symptom of publishing system dysfunction A book cover with a person's face Description automatically generated uAndrew J. Wakefield u Royal Free Hospital and UCL (London) u1998 – published an influential study in Lancet connecting vaccination against measles to new form of enterocolitis linked to autism. Attempts at replication by other researchers failed. u2004 – Sunday Times revealed undisclosed financial conflict of interest on Wakefield part. By selling test-kits for “autistic enterocolitis” he made up to 43 million GBP. u2006 - 2010 - General Medical Council (GMC) investigation. uFound evidence of research misconduct, falsification of data and ethical approvals. u2010 – article retracted, Wakefield struck off the UK medical register uMore: https://www.bmj.com/content/342/bmj.c5347 u Antivax movement as a symptom of publishing system dysfunction ScienceDirect homepage November 2021 https://retractionwatch.com/2015/02/03/frauds-long-tail-measles-outbreak-shows-important-look-downs tream-retractions/ uPublishing industry works for profit and has no motivation to reform. uJournals are encouraged to publish increasing number of articles over shorter span of time. uInstitutions and publishers are too passive to effectively investigate research misconduct. uResearchers are forced to publish as often as possible, but not honored for reviewing the literature, publishing replication and negative data. uWhistleblowers face high personal risks, but at the same time are not rewarded for their work. u uThe result is an environment which encourages production of poor-quality and falsified research publications. uFalsified publications are usually hard to uncover, but even when it happens, it can still undermine the trust of the public in scientific research in general. uConsequences of public distrust towards research are severe. Traditional publishing system is failing How to recognize problematic publication in biomedicine field? uImage data are currently one of the main sources for revealing problems in presented data. uAt least 4 % of primary biomedicine publications (1 in 25) contains problematic image data.[1] uThis number likely represents only the tip of the iceberg: uNot all problematic publications contain image data uNot all problematic publications contain easily identifiable image problems. uSometimes, the problem is hidden in the publication process itself (manipulation/absence of peer review, “hijacking” of special issues and whole journals). uThe real number of problematic publications is much higher. u u [1] https://journals.asm.org/doi/10.1128/mbio.00809-16 Case study uSplicing uCompiling bands from different membranes/gels next to each other. uConsidered bad practice now but tolerated in older papers. A close-up of a test results Description automatically generated A blue and yellow background with black dots Description automatically generated A close-up of a test results Description automatically generated Case study uImage duplication type I (simple duplication of image panel). Case study uDuplication type I (western blot) Case study uDuplication type II - overlapping panels Case study uDuplication type I and II – rotation of panel A collage of different types of petri dishes Description automatically generated Case study uDuplication type I and II – rotation of panel u A collage of different types of petri dishes Description automatically generated A collage of different types of petri dishes Description automatically generated uImage reuse for “illustration” of other experiments→ “self-plagiarism” Case study uUnauthorized reuse of images from other sources (→ plagiarism) Case study uDuplication type III uParts of the image repeat inside one panel or among panels. uWhat could have happened here? A close-up of a test Description automatically generated file Case study uDuplication type III – parts of the image are identical between panels uWhat could have happened here? Case study uWhat could have happened here? Sometimes the image data are not needed A graph of different colored rectangular objects Description automatically generated https://retractionwatch.com/2022/12/05/a-paper-used-capital-ts-instead-of-error-bars-but-wait-there s-more/ file file Paper Mills uOrganized groups (companies) producing and selling fabricated research articles prepared for publication (or including the publication). uThey have templates and image libraries, which they use to generate content fitting the customer. uTypical scenario: The MDs in Chinese hospitals are required to publish in international journals for carrier advancement. This is not always realistic (lack of time, funding, research equipment, language barrier). The MD therefore pays the publication as a service in a specialized company. The company designs the publication based on the customer’s requests and target journal requirements. uPaper Mill products are very hard to expose for certain. You need to search through publications across topics, to find similar templates. u u Paper Mills uThe probability of encountering a paper-mill product depends on a journal and publisher. uIn 2022 Committee on Publication Ethics (COPE) report estimates, that up to 2 % of all curently submitted manuscripts are Paper Mill products. (https://publicationethics.org/node/55256) u“Data on over 53,000 papers were analysed. This was shared by six publishers and spans a wide range of subject areas; overall the percentage of suspect papers being submitted to journals ranges from 2-46%. The analysis shows that most journals will see 2% suspected fake papers submitted and then for journals where paper mills have been successful in getting papers accepted, they see a sharp increase in suspect submissions.” uAnother study trying to detect such publications concluded it could be up to 1% of articles in PubMed (https://www.nature.com/articles/d41586-022-02997-x) u u u u u Some signs of paper mill products u Template-like names u“Long non-coding RNA [enter name][enter effect] in [enter disease].” uRepeating and overlapping images in articles attributed to different authors from different institutions. uVery similar visual style of western blot and flow cytometry data. uNo innovative approaches, unique methodology. uBinary character of conclusions (yes or no), or very shallow description. uGeneric, non-institutional email addresses of authors (142838@168.com) uNo responses from authors following inquiries. uIf they do respond, they usually blame a different anonymous laboratory which made the experiments for them, but which is not acknowledged in the paper. u A screenshot of a graph Description automatically generated A close-up of a test results Description automatically generated A diagram of a test Description automatically generated with medium confidence https://forbetterscience.com/2020/01/24/the-full-service-paper-mill-and-its-chinese-customers/ https://forbetterscience.com/2021/05/26/the-chinese-paper-mill-industry-interview-with-smut-clyde-a nd-tiger-bb8/ The Case of Travelling Ruler David Bimler aka “Smut Clyde” https://forbetterscience.com/2021/05/17/the-ruler-of-the-aging-papermill/ uPublication of authors from at least 4 different institutions contain images of the same ruler. Fabrication using computer programs uCurrent revolution in AI connected to text and image generation represents acute danger for the future of scientific publishing because the falsified data will be harder and harder to distinguish. https://doi.org/10.3389/fcell.2023.1339390 Problematic Paper Screener uhttps://www.irit.fr/~Guillaume.Cabanac/problematic-paper-screener Guillaume Cabanac Tortured phrases uOne way how to bypass the plagiarism detection is by using paraphrasing software, which uses thesaurus to replace phrases in text with synonyms. uIt has a catch though: uSoftware is paraphrasing also the established scientific terms. uResult: Tortured phrases, which make the overall text unintelligible and can be automatically detected. u Abuse of ChatGPT for research fabrication Abuse of ChatGPT for research fabrication What can we do? 1.Be informed. 2.The best defense is offense. Report problematic data in scientific papers, when you encounter them. Information resources uRetraction Watch uhttps://retractionwatch.com/ uGood source of information about ongoing investigations and retracted papers. uAlso manages the database of all retractions: uhttp://retractiondatabase.org u u Ivan Oransky Adam Marcus Information sources uFor Better Science uhttps://forbetterscience.com/ uPersonal blog of Leonid Schneider. uWrites about ongoing investigations, but also about cases which have not been investigated yet. uWritten in a humorous way. uContains detailed description of Paper Mills and how they were exposed: uhttps://forbetterscience.com/2021/05/26/the-chinese-paper-mill-industry-interview-with-smut-clyde- and-tiger-bb8/ u u Leonid Schneider Information sources uScience Integrity Digest uhttps://scienceintegritydigest.com/ uBlog of Elisabeth Bik, one of the leading sleuths in biomedical research. uAlso very active on Twitter/X: u Elisabeth Bik Information sources uPubPeer uhttps://pubpeer.com/ uDiscussion forum for individual articles (must have DOI or PMID identifier). uCan be used also for publishing of long (post-publication) peer-reviews of articles. uOffers extension to browsers and Zotero, which warns about the presence of papers with PubPeer comments. Informační zdroje uPubPeer uhttps://pubpeer.com/ uDiskuzní forum k individuálním článkům majícím DOI nebo PMID číslo uNabízí extension k prohlížeči a Zotero, které upozorňují na přítomnost článků s PubPeer komentáři 23 potentially problematic articles out of 200 displayed! uAllows to post about the concerns connected to individual articles. uDiscussion is moderated, which prevents baseless personal attacks. uPosts can be anonymous or under your own name. uBy posting your question or concern, you can notify authors about the discovered problems (PubPeer sends email notifications to the corresponding author). uEven if the authors decide to not react, your post can warn other researchers about potential problems with the article. uPubPeer posts have been recently used as a base for institutional investigation. Using PubPeer to notify others about problematic data Summary uScientific publishing system is by large based on trust in the honest conduct of authors. uCurrently, this system fails due to multiple factors which endangers public trust in research. uIn all phases of working with literature, it is important to be aware of the existence of problematic publications and to avoid spreading their influence. uPart of the problematic publications can be identified based on image data. uOther types of problematic publications are hard to identify, it is therefore important to be informed about the latest developments in publishing ethics. uProtecting human knowledge from misinformation is the main duty of a researcher. It is therefore necessary to actively address and critique problematic publications and demand their correction or retraction. Thank you for your attention! Bonus: Are we doing it wrong? Publication fees to expensive? Sell the spots on your paper! A black screen with white text Description automatically generated A screenshot of a computer Description automatically generated Bonus: Are we doing it wrong? Hijacked journals •Use cloned websites and ISSN identifiers of established journals. •Articles do not undergo peer review and publication fee goes to hijackers. •Using fake identifiers, the articles can be indexed in Scopus anyway and cited. •Articles citing papers from hijacked articles are called “citejacked”. • •Retraction Watch Hijacked Journal Checker: •https://retractionwatch.com/the-retraction-watch-hijacked-journal-checker/ • • • https://www.irit.fr/~Guillaume.Cabanac/problematic-paper-screener/ACLM_Citejacked.pdf Bonus: Are we doing it wrong? Sneaked references •Citation manipulation is one of the most common “publication crimes”. •Self-citations. •Citing of irrelevant articles. •Omitting important sources. •It is however not so hard to detect. •Sneaked citations are citations only present in article metadata. •Not visible to readers in the article itself. •Not visible even to authors/editors. https://retractionwatch.com/2023/10/09/how-thousands-of-invisible-citations-sneak-into-papers-and-m ake-for-fake-metrics/