The importance of high-quality content: curation and reevaluation in Scopus Scopus uniquely combines a comprehensive, curated abstract and citation database with enriched data and linked scholarly content. Users can quickly find relevant and trusted research, identify experts, and access reliable data, metrics and analytical tools to support confident decisions around research strategy — all from one database. New content is added to Scopus after a rigorous evaluation process by the subject experts of the Content Selection and Advisory Board (CSAB). The experts of the CSAB determine whether a title is eligible for indexing in Scopus using the Scopus title selection criteria. They also continuously re-evaluate titles indexed in Scopus to ensure the quality of the existing content is maintained. In this article we discuss the importance of a rigorous content curation mechanism to exclude poor-quality and predatory publications from Scopus. We use the terms “poor-quality” and “predatory” journals throughout this piece as they are not the same. Poor-quality journals may not meet certain quality standards based on their publication performance or bibliometric indicators. However, these journals do not necessarily engage in unethical publication practice.
The importance of using research information that you can trust Researchers need curated, high-quality content. Poor-quality journals may be easy to identify from bibliometric indicators’ perspective, however, particularly journals in niche areas may have limited readership or low citations, in which case bibliometrics might not necessarily be an indicator of low quality. Predatory journals are a threat to the integrity of science. Although usage of the term predatory publishing is widespread, at the practical level it remains ill-defined and subject to personal interpretation and judgement. Specifically: • Labeling a journal predatory may be controversial if there is no evidence. Publishers that have, in someone’s subjective opinion, some predatory publications may also publish high-quality journals. • It is not always straightforward to identify predatory journals. Journals change their editorial policies over time; a journal that did not start off as predatory may become so over the years or vice versa. There have been several attempts to define predatory publishing. Scopus has been using the following guidelines and definitions. • Organizations including the Committee on Publication Ethics (COPE) and the World Association of Medical Editors (WAME) define global best editorial and publication practices —predatory journals do not meet them. • In 2019 a group of authors reached a consensus definition of predatory publications which was subsequently published as Defining predatory journals and responding to the threat they pose: a modified Delphi consensus process. In summary, this definition reads: “Predatory journals and publishers are entities that prioritize self-interest at the expense of scholarship and are characterized by false or misleading information, deviation from best editorial and publication practices, a lack of transparency, and/or the use of aggressive and indiscriminate solicitation practices.” In recent years, the growth of global research output and the expansion of various publishing business models have led to a rise in the number of newly launched journals, including titles that could be considered predatory. Despite forming a small proportion of all journals published globally, predatory journals are a significant challenge that concerns all research publishing stakeholders: authors, editors, researchers, research institutions, publishers, funding bodies and governments. The rise of new predatory journals is partly driven by the increasing pressure on researchers to publish, in order to secure funding and to advance their careers. In some instances, these forces may encourage authors to be less scrupulous about where their work appears than might previously have been the case. Many governments and institutions have sought to establish expectations for their country’s research by issuing mandates for their researchers to publish in Scopus or Web of Science (WoS) indexed titles; this has inevitably induced many predatory journals and publishers to target these databases for their business practices. Scopus and the CSAB seek to minimize the impact of poor-quality or predatory journals and ensure that Scopus users get the best possible evidence for their work.
Evaluation and re-evaluation of content covered by Scopus Only the most reliable journals and their content are available in Scopus as they are carefully curated and ultimately selected by the independent CSAB, an international group of subject experts across all fields of research. Year round, the CSAB are reviewing titles that are submitted to Scopus. In addition, a process of continuous monitoring and re-evaluation was developed ensuring that the quality of the existing content is also maintained. Out of the full corpus of journals covered by Scopus, journals considered predatory or that perform poorly are identified for comprehensive re-evaluation by the CSAB. Four ways a journal can be flagged for re-evaluation include:
1. The journal is underperforming as it does not meet any of the three metrics and benchmarks for journals in the same subject area. The indicators have been developed and agreed by the CSAB in partnership with the Scopus team.
2. Concerns about the publication standards of the journal or publisher have been raised by formal complaints.
3. The journal shows outlier behavior based on its publishing performance in Scopus.
4. Continuous curation based on CSAB feedback. Underperforming titles are detected using the three metrics shown in Table 1.
The CSAB re-evaluates journals that fail to meet any of the three metrics and benchmarks for two consecutive years. Metric Benchmarks and Explanation Self-citation rate The journal has a substantially higher self-citation rate, when compared to peer journals in its subject field. Total citation rate The journal received a substantially lower number of citations, when compared to peer journals in its subject field. CiteScore The journal has a substantially lower CiteScore, when compared to peer journals in its subject field. Table 1. The three metrics and benchmarks to identify underperforming journals for re-evaluation.
A journal can also be identified for re-evaluation if concerns about its publication standards are raised, such as the quality or editorial practices of specific titles, or other issues that impact their suitability for continued coverage. Outlier behavior identified through statistical analysis is particularly effective in flagging potential predatory journals. Scopus runs an algorithm that flags journals based on a number of predictors, including sudden change in output volume, sudden change in publishing country and/or affiliations and high journal/author self-citation rates. Finally, the Scopus team collects and analyzes previous CSAB title evaluation feedback and continuously flags titles for re-evaluation. The CSAB re-evaluates all titles identified for underperformance, publication standard concerns, outlier behavior, or continuous content curation. A CSAB subject expert re-evaluates these titles with the same criteria used for the selection of new journals. Because poor-performing journals may still be relevant for the communities they serve and because predatory publishing is subject to personal interpretation, independent review of individual journals by academic subject experts in each field is essential. Not all journals that are initially identified as poor-quality or predatory are confirmed to be below standards after re-evaluation by the CSAB, and they may decide to continue coverage. If the CSAB determines that a publication is no longer meeting the standards for inclusion in Scopus anymore, indexing of new content from that journal is discontinued. To make sure that content from journals undergoing re-evaluation is not included in Scopus during the re-evaluation process, the content flow is put on hold pending the review. For journals that are discontinued or put on hold, the content already indexed in Scopus remains. This as a matter of scientific record and to ensure the stability and consistency of research trend analytics which is a core value of Scopus. In exceptional cases of proven severe unethical publication practice, content already indexed in Scopus may be removed. The Scopus team informs publishers of the titles about the re-evaluation procedure as well as its outcome. All journals discontinued in Scopus are publicly available via the Scopus discontinued sources list.
Since 2016, the CSAB has re-evaluated 990 titles published by 539 different publishers leading to 536 titles discontinued for indexing. In Figure 1 below, the results are broken down by the original reason why the journal was identified for re-evaluation.
“Journal hijacking” refers to cybercrimes that involve building fake websites that mimic the names, editorial boards, ISSNs, similar URLs, or journal indexing information of reputable journals. [1,2] This, for example, occurs in but is not exclusive to print-only journals or regional-language journals, for which authentic websites are not always easily detectable. In The guardians of Scopus, Prof. Sack mentions “Whether or not a journal is predatory is not a binary decision; there is a broad spectrum of predatory journal behaviors.” Journal hijacking similarly falls within this spectrum and stands on the deceptive side. We have stringent procedures in place to detect hijacked journals as early as possible and to ensure that the content indexed in Scopus is from the legitimate source. When a journal is hijacked, it does not necessarily mean that the hijacked content is also indexed in Scopus. Despite our continuous efforts to remain vigilant, organizations behind this activity are clever and can be extremely effective in their endeavors, such that Scopus is sometimes misled and content from a hijacked journal gets in. This can occur when journals update URLs or content feeds. We have strict validations in place to manage these changes and continue to improve our procedures. In those cases where there is evidence that the articles in Scopus are not authentic, that is, are not sourced from the genuine journal, the content will be removed from Scopus. We realize this may have huge implications for the authors involved, therefore we only do this by exception and as soon as possible after the content has been loaded into Scopus.
[1] Butler, D. Sham journals scam authors. Nature 495, 421–422 (2013). https://doi.org/10.1038/495421a
[2] COPE Council. COPE Discussion Document: Predatory Publishing. November 2019. https://doi.org/10.24318/ cope.2019.3.6.
Scopus responds to community concerns about predatory journals It is not surprising that Scopus, as a leading abstract and citation database, receives questions about possible predatory content. All journals within Scopus face re-evaluation based on the criteria outlined above, and the Scopus team also responds to community input. In February 2021, Nature published a news item on the publication of a research article by authors from Charles University and the Economics Institute of the Czech Academy of Sciences in the journal Scientometrics (now retracted). The article is based on a study from 2017 looking into predatory publishing using Beall’s list as a definition for predatory journals and analyzing the geographical origin of authors publishing in these journals. Please note that since January 2017, Beall’s List is no longer being maintained. All of the 137 suspicious titles mentioned in the paper have gone through the re-evaluation process and as a result for 97 titles (71%), the decision was made to discontinue coverage in Scopus. Also, all other journals listed by Beall that are mentioned in the paper have gone through the re-evaluation process and as a result 65% of these titles were also discontinued. Meanwhile, the Scientometrics article has been retracted because some of the findings are found to be unreliable. It also emphasizes the risks of the continued use of blacklists and its criteria for characterizing predatory journals or publishers. Throughout 2020, the Scopus team was in continuous contact with national organizations such as the Russian Academy of Sciences (RAS) and the Indian University Grants Commission (UGC), both of which shared their investigation results on publication malpractices for over 100 journals of concern, some of which were included in both lists. All journals affected were actively investigated and those journals for which publication concerns were validated were sent for re-evaluation by the CSAB. As a result, approximately 45 journals were discontinued or were no longer active and publicly announced in the Scopus discontinued sources list. The UGC Consortium for Academic and Research Ethics (UGC-CARE) approved list of journals includes all journals covered by Scopus. In January 2019 the paper Is Scopus polluting its own database by indexing junk articles? was posted on the institutional repository of the Ludwig Maximilian University of Munich. The unpublished paper is a case study of five journals that are claimed to pollute the Scopus database quality. Eventually, for all five journals mentioned, the CSAB made the decision to discontinue coverage in Scopus.
Conclusion Scopus and the CSAB recognize they have a shared responsibility for the ongoing curation and re-evaluation of the database to ensure that only journals containing high quality content remain actively indexed and those journals that do not adhere to ethical publication standards are kept out. With its independent expert title selection, re-evaluation and publication discontinuation practices, Scopus puts forth a state-of-the-art defense against predatory publishing. The decision to re-evaluate and potentially exclude a journal takes time and is complex, but if done properly this process minimizes the risk of discontinuing legitimate sources. It is an ongoing challenge to detect journals that begin to show signs of practices associated with predatory journals. Along with their current rigor in curation and re-evaluation, Scopus and the CSAB are committed to developing new approaches to ensure that the combined quality and breadth of Scopus content continue to be unrivalled and highly trusted.
Useful sources
• Scopus Content Selection and Advisory Board
• Title selection criteria • Committee on Publication Ethics (COPE)
• World Association of Medical Editors (WAME) • Principles of Transparency and Best Practice in Scholarly Publishing • Nature. 2019 Dec; 576(7786): 210–212. DOI: 10.1038/d41586-019-03759-y • BMJ Open 2020 10(2) 10:e035561 DOI: 10.1136/bmjopen-2019-035561 • Scopus’ discontinued sources list • Scopus Re-evaluation Process • Scopus curation and re-evaluation webinar • The importance of high-quality content in Scopus (Scopus blog) Written by: Karen Holland, CSAB Chair—Nursing; Health Professions Prof. Peter Brimblecombe, CSAB Chair—Environmental Science Dr. Wim Meester, Director of Product Management, Scopus Tracy Chen, Product Manager, Scopus
No comments:
Post a Comment