-
Among the 26 medical journals in EWL, four were at the 'high-risk level', 18 were at the 'medium-risk level', and four were at the 'low-risk level'. Five were from the publisher SPANDIDOS PUBL LTD ('medium-risk'), four were from the publisher E-CENTURY PUBLISHING CORP (two were 'high risk' and two were 'medium risk'), two were from the publisher SAGE PUBLICATIONS ('medium risk'), and two were from the publisher MDPI ('low risk'). Each of the remaining 13 publishers had one early warning SCI-indexed journal. Of the 26 journals, 23 were also included in PubMed, including three 'high-risk' journals, 17 'medium-risk' journals, and three 'low-risk' journals. In terms of the JCR quartile in 2020, four were in discipline Q1, 10 were in Q2, and six were in Q3 and Q4 according to their highest quartile. In terms of the CAS quartile in 2020, no journal was in discipline Q1, three were in Q2, nine were in Q3, and 14 were in Q4 according to the highest quartile. In terms of OA journals, 17 were OA journals, one was a non-OA journal, and eight were optional. All journals collected an article processing charge (the charges are published on their websites). In terms of publication cycle, two journals were semimonthly (24 editions a year), 17 were monthly (12 editions a year), one released 52 editions a year, one released nine editions a year, two were bimonthly (six editions a year), one released two editions a year, and two were annual. See Supplemental Table S1.
Representative indicators
Impact factor
-
Our statistics show that the 2019 IF of the 26 journals in EWL was 0.166−5.117 [2.58 (1.727−3.422)], among which 12 had an IF higher than 3, accounting for 46.1%. Except for Journal of Clinical Medicine, which was indexed by SCI for only 4 years, the 5-year (2014–2018) IF of the other 25 journals was 0.253–5.397 [2.42 (1.91−3.31)], among which 11 had an IF higher than 3, accounting for 44.0%. See Supplemental Table S2.
Self-citation rates
-
Except for Journal of Clinical Medicine, which was indexed by SCI for only 4 years, the 5-year (2014–2018) self-citation rates of the 25 medical journals in EWL were 0.8%–22.4%, among which 22 had a self-citation rate lower than 10%, accounting for 88%; two had a self-citation rate between 10% and 20%, accounting for 8%; and only one was higher than 20%, accounting for 4%. The 2019 self-citation rates of the 26 medical journals in EWL were 1.1%–31.4%, among which only two had a rate higher than 20%, accounting for 7.7%; three had a rate between 10% and 20%, accounting for 11.5%; 17 had a rate lower than 5%, accounting for 65.4%; and four had a rate between 5% and 10%, accounting for 15.4%. See Supplemental Table S2.
Global total number of articles published and proportion of articles authored by Chinese scholars
-
Between 2016 and 2020, nine journals each published more than 1000 articles, which was the average value of the annual global total number of articles published by the 26 medical journals in EWL, accounting for 34.6%. Two journals exceeded the average value of 2,000 articles as the annual total number of articles published (with mean values of 4,109.6 and 4,310 articles, respectively). Between 2016 and 2020, the proportion of articles authored by Chinese scholars remained at a high level (18 journals) and witnessed a gradual increasing trend (six journals) among the 26 medical journals in EWL, except for the International Journal of Environmental Research and Public Health and the Journal of Clinical Medicine at lower levels. Nineteen out of 26 medical journals in EWL were characterized by an average 5-year proportion of articles authored by Chinese scholars over 50%, among which three journals had an average 5-year proportion higher than 90%. See Supplemental Table S2.
Retraction of published papers
-
Between 2016 and 2020, only three journals had no retractions, 22 journals each had cumulative retractions of fewer than 50 articles (1–43 articles) and one journal, the European Review for Medical and Pharmacological Sciences, had 174 retractions in total (with a peak of 96 articles in 2020), among the 26 early warning SCI indexed medical journals. In addition, journals with more than 20 cumulative retractions included the International Journal of Clinical and Experimental Pathology (31 articles), the International Journal of Clinical and Experimental Medicine (21 articles), MEDICINE (43 articles, among which the peak was 13 articles retracted in 2020), Oncology Letters (41 articles, among which the peak was 15 articles retracted in 2020), Experimental and Therapeutic Medicine (25 articles), OncoTargets and Therapy (31 articles, among which the peak was 20 articles retracted in 2020), and LIFE SCIENCES (38 articles, among which the peak was 19 articles retracted in 2020). See Supplemental Table S2.
Follow-up analysis was conducted with the high proportions of articles authored by Chinese scholars, and the high prevalence of retractions was characteristic of the medical journals in EWL.
Characteristics of early warning SCI indexed medical journals at different risk levels
-
There was no significant difference in the JCR quartile, CAS quartile, OA or not, 2019 IF, 5-year (2014–2018) IF, 2019 self-citation rate, 5-year (2014–2018) self-citation rate, or 2016–2020 combined retraction rates of the journals among the high-risk, medium-risk, and low-risk levels (p > 0.05). There was a significant difference in the publication cycle, the 2016–2020 global total number of articles published, and the proportion of articles authored by Chinese scholars (p < 0.05). See Table 1. The high proportion of articles authored by Chinese scholars was a major characteristic.
Table 1. Characteristics of the 26 SCI indexed medical journals at different risk levels.
Category Option Total High risk (n = 4) Medium risk (n = 18) Low risk (n = 4) X2/H P value JCR quartile Q1 4 0 3 1 3.888 0.692 Q2 10 1 7 2 Q3 6 1 5 0 Q4 6 2 3 1 CAS quartile Q2 3 0 2 1 5.096 0.278 Q3 9 0 7 2 Q4 14 4 9 1 Whether OA No 1 0 1 0 3.160 0.531 Optional 8 0 7 1 Yes 17 4 10 3 Publication cycle <12 editions/year 6 0 5 1 10.138 0.038 = 12 editions/year 17 3 13 1 >12 editions/year 3 1 0 2 2019 IF Med (P25−P75) 2.58(1.727−3.422) 0.902(1.88−2.656) 2.705 (1.997−3.713) 3.076(0.899−3.561) 4.343 0.114 5-year (2014−2018) IF Med (P25−P75) 2.42 (1.91−3.31) 1.59 (0.76−2.34) 2.71 (2.01−3.65) 2.84(0.83−3.41) 3.169 0.205 Global total number of articles published between 2016 and 2020 Med (P25−P75) 3 603.5
(1 134.5−7 006.25)7 760
(4 939.5−18 604.75)3 086.5
(858−5 088.5)5 438.5
(2 469−17 201.75)7.078 0.029 Proportion of articles authored by Chinese scholars between 2016 and 2020 Med (P25−P75) 74.3 (54.95−86.7) 83.97 (68.07−94.33) 74.94 (61.79−86.7) 37.53 (12.27−54.06) 9.004 0.011 2016-2020 Combined retraction rate Med (P25−P75) 0.33(0.08−0.71) 0.44(0.2−2.44) 0.33(0.06−0.61) 0.46(0.05−0.97) 1.152 0.562 5-year (2014−2018) self-citation rate Med (P25−P75) 0.03 (0.01−0.43) 0.03 (0.01−0.15) 0.03 (0.01−0.04) 0.03 (0.01−0.11) 0.446 0.800 2019 self-citation rate Med (P25−P75) 0.04 (0.02−0.07) 0.06 (0.04−0.21) 0.03 (0.02−0.05) 0.08 (0.04−0.16) 4.832 0.089 CAS is Chinese Academy of Sciences; The sample size of 2019 IF and self-citation rate are 26. The sample size of the 5-year (2014–2018) IF and the 5-year (2014–2018) self-citation rate are 25 during 2016 to 2020 M (P25, P75). 'Early warning journals' questionnaire for the burn specialty group
Questionnaire recovery
-
In total, 424 valid questionnaires were recovered.
Basic profiles of the respondents
-
The majority of respondents were male, aged 42.0 years on average (22.0–64.0), with the age group of 40 years or younger accounting for the highest proportion. The majority were of Han ethnicity. The highest education levels were a master's degree and a doctorate degree, the majority had a senior professional title, and the employers were mainly colleges and universities, research institutes, and teaching hospitals. The mean value of the respondents' working years was 17.00 years (1.0–40.0 years), with the majority having either no more than 10 years or over 20 years of experience. The most common occupation type was clinical doctor. See Supplemental Table S3.
Factors considered by the 424 respondents for article publication
-
The main factors considered for article publication were, according to ranking, the academic influence of the journal (impact factor), the type of journal (such as statistical source journal, core journal, or SCI indexed journal), journal reputation, and the relevant guiding policies of the state and the employer (such as reward and evaluation requirements). See Fig. 1.
Respondents' awareness of the policy of early warning journals
-
In most cases, the respondents' awareness level was 'know and understand a little bit'. For the list of early warning journals, it was 'know the EWL but don't know the specific journals' in most cases; for the impact of the list on journal selection, it was 'have impact to some extent' in most cases; for the effect of EWL on boosting research integrity, it was 'have some effect' in most cases; for the effect of EWL on improving journal quality management, it was 'have some effect' in most cases; for the impact of EWL on the desire to publish research outcomes, it was 'have some impact' in most cases; for the impact of EWL on choosing Chinese journals as the first choice, it was 'yes, will consider' in most cases. See Table 2.
Table 2. Understanding of the policy of early warning journals by the 424 respondents.
Category Option Number (persons) Ratio (%) Awareness of the policy of early warning journals Don't know 86 20.3 Know but don’t understand 101 23.8 Know and understand a little bit 171 40.3 Know and understand a lot 66 15.6 Whether know the EWL or not Don’t know the EWL 98 23.1 Know the EWL but don’t know the specific journals 213 50.2 Know the EWL and know the journals on the list 113 26.7 Will EWL affect the article submission choice? No 54 12.7 Won’t consider the journals on the EWL at all 167 39.4 Yes, to some extent 203 47.9 The effect of EWL on boosting research integrity No effect 10 2.4 Obvious effect 166 39.2 Some effect 248 58.5 The effect of EWL on improving journal quality management No effect 4 0.9 Some effect 241 56.8 Obvious effect 179 42.2 The impact of EWL on the desire to publish research outcome No impact 114 26.9 Some impact 240 56.6 Obvious impact 70 16.5 The impact of EWL on selecting Chinese journals as the first choice No 63 14.9 Yes, will consider 317 74.8 Yes, first choice 44 10.4 EWL is Early Warning List. The main factors for early warning of journals were, according to ranking, low academic quality of journals, a high self-citation rate, the relationship with retractions, and a high proportion of articles authored by Chinese scholars. See Fig. 2.
-
Among the 26 early warning SCI indexed medical journals released by CAS at the end of 2020, the characteristics are high proportions of articles authored by Chinese scholars and widespread retractions. Table 1 shows that journals at the 'high-risk level' had the largest total number of articles published and the highest proportion of articles authored by Chinese scholars, which is in line with the risk level. The total number of articles published in journals at the 'low-risk level' was higher than that of journals at the 'medium-risk level', whereas the proportion of articles authored by Chinese scholars was lower than that of 'medium-risk' journals, in line with the risk level. Therefore, a high proportion of articles authored by Chinese scholars was the main characteristic of warning journals. Retraction is essentially self-purification aimed at defending academic truth and integrity[18]. However, it is treated cautiously by publishers because it causes ethical problems related to publication[19]. With regard to cases with a large number of retractions, the journal European Review for Medical and Pharmacological Sciences retracted nearly 200 articles in 5 years and 96 articles in 2020 alone, and MEDICINE, Oncology Letters, OncoTargets and Therapy, and LIFE SCIENCES had retractions that reached a peak in 2020. There is no doubt that problems of research integrity and publication ethics may have emerged. These constitute the reasons that some journals are put on the EWL.
Applicability of the early warning journal policy
-
Figure 1 and Table 2 show that the main factors that the respondents considered for article publication included journal reputation; more than four-fifths (87.3%) of the respondents stated that EWL had an impact on their article submission choices, and almost all respondents believed that early warning of journals had an effect on boosting research integrity (97.7%) and improving journal quality management (99.1%). All of these findings suggest that scholars accept the mechanism of early warning journals, and this mechanism has good prospects for promotion.
As Fig. 2 shows, according to the respondents, the main reasons for some journals being put on the EWL included a high proportion of articles authored by Chinese scholars, retractions, and high self-citation rates. The high proportions of articles authored by Chinese scholars and retractions were in line with the characteristic indicators of medical journals in EWL stated in 4.1, which means there is common ground between the perceptions of the respondents and the early warning analysis of the evaluation institutions. High self-citation rates were chosen as one of the main reasons why journals were included in the EML in Fig. 2, which indicates that scholars are aware of normative academic communication.
As Table 3 shows, 1) two high-risk journals on the 2020 warning list, namely, the International Journal of Clinical and Experimental Pathology and the International Journal of Clinical and Experimental Medicine, with 90% Chinese authors in 2019 and 2020, were delisted from the SCI database in 2021; 2) two medium-risk journals on the 2020 warning list, namely, the American Journal of Translational Research and the Journal of Biomaterials and Tissue Engineering, with 90% Chinese authors during 2019–2022, were still on the warning list issued in 2023; 3) the average Chinese author rate is relatively high for the 2020 low-risk journal Acta Medica Mediterranea, with an annual average of over 80%, and shows an upward trend during 2019–2022; it is still listed in the EWL issued in 2023, and the early warning level has been upgraded to medium-risk; and 4) the rate of Chinese authors of other journals on the 2020 warning list showed a general downward trend during 2019–2022.
Table 3. Data of Published articles in 26 warning medical journals from 2019 to 2022.
Journal name Warning level Number of articles published 2019 2020 2021 2022 Total Chinese scholars n(%) Total Chinese scholars n(%) Total Chinese scholars n(%) Total Chinese scholars n(%) European Review for Medical and Pharmacological Sciences High 1375 1129(82%) 1614 1221(76%) 1001 380(38%) 1186 313(26%) International Journal of Clinical and Experimental Pathology High 514 468(91%) 384 336(88%) – – – – Medicine High 4689 2897(62%) 5292 3537(67%) 4583 2689(59%) 4079 2132(52%) International Journal of Clinical and Experimental Medicine High 1703 1643(96%) 1222 1185(97%) – – – – Biomedicine & Pharmacotherapy Medium 1431 987(69%) 1338 995(74%) 1603 700(44%) 1422 701(49%) Experimental and Molecular Pathology Medium 122 60(49%) 174 88(51%) 114 51(45%) 74 31(42%) Brazilian Journal of Medical and Biological Research Medium 150 79(53%) 148 90(61%) 184 82(45%) 108 23(21%) Cancer Biomarkers Medium 149 113(76%) 163 119(73%) 130 82(63%) 147 70(48%) International Journal of Immunopathology and Pharmacology Medium 64 27(42%) 69 37(54%) 114 38(33%) 90 35(39%) Oncology Research Medium 117 111(95%) 95 74(78%) 36 22(61%) 16 8(50%) American Journal of Cancer Research Medium 206 152(74%) 320 222(69%) 400 220(55%) 380 225(59%) Medical Science Monitor Medium 1126 996(88%) 982 895(91%) 535 412(77%) 352 219(62%) Oncology Letters Medium 1509 1178(78%) 1196 924(77%) 845 573(68%) 466 252(54%) Experimental and Therapeutic Medicine Medium 1271 1069(84%) 1261 989(78%) 1485 1058(71%) 757 479(63%) OncoTargets and Therapy Medium 1075 960(89%) 1170 1057(90%) 521 391(75%) 191 61(32%) Oncology Reports Medium 608 440(72%) 436 332(76%) 378 229(61%) 219 74(34%) Molecular Medicine Reports Medium 1172 1066(91%) 846 740(87%) 876 721(82%) 370 248(67%) International Journal of Molecular Medicine Medium 455 351(77%) 379 303(80%) 288 231(80%) 147 80(54%) Journal of International Medical ReSearch Medium 888 685(77%) 1271 1026(81%) 995 759(76%) 596 333(56%) American Journal of Translational Research Medium 636 576(91%) 644 589(91%) 1479 1391(94%) 752 672(89%) Journal of Biomaterials and Tissue Engineering Medium 259 236(91%) 280 270(96%) 346 334(97%) 326 310(95%) Aging-US Medium 889 546(61%) 1620 1252(77%) 1580 1272(81%) 658 441(67%) LIFE SCIENCES Low 908 484(53%) 1300 713(55%) 1320 576(44%) 817 208(25%) Journal of Clinical Medicine Low 2230 31(1%) 4133 39(1%) 5986 67(1%) 7535 517(7%) International Journal of Environmental Research and Public Health Low 5164 1376(27%) 9594 1728(18%) 13429 1857(14%) 17089 4099(24%) Acta Medica Mediterranea Low 555 409(74%) 607 485(80%) 629 505(80%) 623 555(89%) In addition, six journals appear repeatedly on the 2021 and 2020 lists, three are repeated on the 2020 and 2023 lists, and most repeated journals have a high rate of Chinese authors. Furthermore, "'he high rate of Chinese authors' is the main feature of early warning medical journals. After being warned, the journal is either delisted from the SCI index or continues to be monitored, or it improves on the basis of the 'high Chinese author rate', indicating that the EWL is applicable, recognized by the academic community, and has achieved significant results.
Limitations
-
Limited by study time and resources, this study could not investigate all burn specialty professionals across China[20]. Nevertheless, we collected samples from approximately 30 provinces, autonomous regions, and direct municipalities of China. The participants were experts, ordinary doctors, and nurses of the burn specialty, and their attitude was representative of the attitude of the entire burn specialty field toward the policy of early warning journals. We hope that in the future, sci-tech workers and science editors from more disciplines will join this study series so that we can provide comprehensive practical statistics on the Chinese policy of early warning journals worldwide.
Future perspectives
-
Many similar blacklists or whitelists have been identified, namely, Beall's List, Cabells' Journal Blacklist, and DOAJ inclusion, but each has its limitations. The annual EWL released by CAS has aroused wide public concern since 2020. The influence of these lists clearly illustrates the importance of the role of blacklists in the academic universe. Based on the mentioned works, we propose to establish a global warning alliance to fulfill the pressing need for information on predatory and questionable quality publishing, integrate and unify the warning monitoring indicators, and regularly update and release the EWL with peer experts' comments for free to provide a reference when choosing journals and quality control standards for journal management.
-
The EWL released by the CAS has attracted widespread attention in the academic world. From our analysis, the medical journals in EWL released by CAS are characterized by a high proportion of articles from China as well as a high rate of retraction. The questionnaire results showed that most burn specialty scholars believe that the establishment of such a list is beneficial to enhancing research integrity and improving journal quality management as well as for selecting journals for publication. The EWL can provide important practical data for the 'global alliance of early warning journals' in the future.
-
About this article
Cite this article
Mo Y, Liang G, Zhen N, Lin H, Xiang H, et al. 2023. Establishment of an early warning list of SCI-indexed international journals with dual benefits for both scientists and publishers. Publishing Research 2:3 doi: 10.48130/PR-2023-0003
Establishment of an early warning list of SCI-indexed international journals with dual benefits for both scientists and publishers
- Received: 17 January 2023
- Accepted: 13 June 2023
- Published online: 19 July 2023
Abstract: At the end of 2020, the National Science Library of the Chinese Academy of Sciences (CAS) released the Early Warning List (EWL) of International Journals as guidance for scholars to choose appropriate journals in which to publish. The aims of the study were to analyze the characteristics of medical-related journals in the list and to survey scholars in the field of burn specialty regarding their understanding of and attitude toward the journals in EWL to determine their attitude toward the list and the potential use of this list as a reference for international journal quality control. All 26 early warning medical journals in the EWL are characterized by a high proportion of articles from China, as well as a high rate of retraction. The average 5-year proportion of articles authored by Chinese scholars of 19 journals was over 50%, and three were over 90%. Eight journals each retracted more than 20 articles from 2016 to 2020. The questionnaire survey showed that most burn specialty scholars believed that the establishment of such a list was beneficial for enhancing research integrity and improving journal quality management, as well as selecting journals for publication. Based on our study, we recommend establishing a global list of early warning journals to allow the EWL to be updated regularly and released for free. Peer review could be introduced to boost credibility. The EWL released by CAS can provide important practical data for the 'global alliance of early warning journals' in the future.