'Tortured phrases' in post-publication peer review of materials, computer and engineering sciences reveal linguistic-related editing problems

Basic editorial functions, such as editing, lie at the heart of reliable and quality science publishing. The accuracy of English in the context of scientific publishing has become central to ranked and indexed journals^[1]. The fine-scale refinement of language, grammar or punctuation, and attention to detail, are aspects that can define a journal that places emphasis on accuracy and perfection^[2]. Consequently, inaccuracies that arise from editing errors and/or failure in editorial quality control can harm the image of a journal and its editors, if such issues are extensive or repetitive, following traditional peer review^[3,4]. The integrity of the publication process has been placing, in recent times, a greater level of scrutiny on the ethical integrity of the published literature, although attention to editing has – until recent times – generally considered to be a minor aspect of that scrutiny, even though such errors reflect – to different extents – a lack of editorial integrity^[5]. Not all tools that exist to fortify research and publishing integrity fully achieve this goal, and some in fact have limitations or flaws, such as now-defunct Publons^[6], Open Research and Contributor Identifier (ORCID)^[7], or text similarity or plagiarism detection software^[8].

Editors' tasks are not few, nor are they simple. They may range widely from overseeing peer review, or in the case of the editor-in-chief, macro-management of the editorial team, interaction with the publisher while also micro-managing tasks during the revision and processing of tasks related to style, language, or settings during or after the peer review of a manuscript^[4,9]. Their functions also include the need to pay attention to ethics-and integrity-related issues such as the detection of textual similarity or plagiarism, or use of image-related forensics to detect manipulated images^[4,10,11]. Increasingly, editors are also tasked with dealing with post-publication challenges on them, their work, and the papers that they have approved for publication^[4,12]. Such tasks are accompanied by the weight of editorial responsibility, in which failure in any of these aspects, whether minor or major, reflects a failure in editorial micro- and macro-management.

The post-publication movement has been abuzz in recent years, and has focused its core attention on ethics and integrity-related issues such as fraud and misconduct. Within those broad themes, abuse of the peer review process, the illicit (i.e., paid-for) or undisclosed (i.e., lack of due attribution or non-acknowledged) use of third party services and consultancies, such as paper production or paper mills^[13], analytical services, language and editing-related services^[14], false authorship (guest or ghost), plagiarism, data abuses and/or statistical errors^[15], are just a fraction of the issues currently plaguing the integrity of the published literature, and challenging the borders of responsibility between authors, editors and publishers. There is a formidable market of writing and editing services, including those provided by several mainstream publishers^[16]. To the author's knowledge, limited attention or scrutiny has been to these for-profit services.

The culmination of post-publication scrutiny of multiple issues related to academic journals, editors and publishers is resulting in a marked increase in errata and retractions^[17]. In all of these cases, at least in legitimate scholarly journals, all papers were ultimately approved for publication by editors, supposedly with pre-publication proof-reading by publishers' technical editors and/or copyeditors. Some academics whose native language is not English may use autocorrect software^[18], and the undeclared use of such software is not easy – or almost impossible – for copyeditors to detect. Even if authors circumvented established rules or guidelines, accidentally or purposefully, finally, editors provided their stamp of approval for the publication of a study, following claimed peer review and editorial screening, that was insufficiently scrutinized. For this reason, even though there is de facto shared responsibility for failure of the peer review process and editorial handling that is often implicit in an erratum or retraction, it is not always explicitly conveyed to the academic community or the public because communication, such as through retraction notices, can be skewed, opaque and/or incomplete^{[19, 20]}.

Editors and publishers are continually adjusting their managerial tasks to accommodate new skills, techniques and support structures such as online submission systems, reference managers, statistical software, or text similarity detection software. The efficiency of editorial processing often requires an amalgamation of such skills and the responsibilities described above. The lack of any one of these may result in the escape of details and the loss of quality of the final product, i.e., the published paper.

It is within this wider discussion of editorial responsibilities that recent attention has been focused on the existence of 'tortured phrases'^[21], which are non-standard or bizarre terms and phrases that seemingly replace standard scientific jargon or conventional English expressions, possibly as an attempt to avoid, during the peer review process, the detection of textual similarity or plagiarism, and thus foil automated artificial intelligence (AI)- or machine-based software and human detection^{[22, 23]}. Conventional AI-based plagiarism-detection software is not always able to detect text that has been derived from reverse translation or the use of paraphrasing tools, although some tools are now able to detect machine-paraphrased text^[24]. These errors clearly reflect editorial failures at macro- and micro-levels, but it is unclear to what extent authors, editors or publishers' proof departments fail to notice such errors during the peer review and proofing processes, and subsequent publication.

For example, a common English expression such as 'upsetting the apple cart' might be described, after the use of machine- or AI-aided thesauruses or synonymizing software, as 'disturbing the fruit chariot', thereby reducing textual similarity in three of the four original words, or by 75%, thereby avoiding detection by text similarity software. Also hypothetically, a term such as 'skin cancer' might be retextualized as 'epidermal growths', distorting standard medical jargon^[25], while the use of incorrect or ambiguous terms may result in novel and non-existent attributes, such as the heterosexual charge of a lithium-ion battery^[26] or the carbon structure^[27], or the misrepresentation of conventional medical terms such as Alzheimer's disease with 'tortured phrases' such as 'Alzheimer's affliction', 'Alzheimer's infection', 'Alzheimer's problem', or 'Alzheimer's sickness'^[28]. Such adjustments would introduce fatal linguistic errors, ultimately reducing comprehension by the reader and the paper's scientific value. They might also reveal an increasing over-reliance by editors on software that discovers textual similarity but that automatically interprets it as plagiarism, i.e., the potentially erroneous synonymization of textual similarity and plagiarism^[8,23]. If the purpose of authors that engage in textual synonymization through the use of thesauruses or other AI-assisted techniques, such as reverse-translation software^[23,29], is to avoid the detection of plagiarism, then perhaps the culture of plagiarism and plagiarism detection in academic publishing needs to be rethought and retaught, especially to academics for whom English is not their native language^[30].

What makes known cases (see select examples in Table 1) particularly troubling is that they are being detected in journals that are indexed (PubMed, Web of Science, Scopus, DOAJ, Crossref, etc.), have been assigned a digital object identifier (DOI) and a metric (Clarivate Analytics Journal Impact Factor (JIF), Elsevier/Scopus CiteScore, Scimago Journal Ranking, etc.), carry an industry brand of editorial quality (OASPA, COPE, ICMJE, etc. membership or compliance)^[9], or are otherwise traditionally classified as safe-to-publish-in white-listed journals. If basic editorial failure is being discovered in such status quo peer-reviewed journals and publishers^[31], then the moral impact and weighting of voices of those who criticize the lack of editorial quality or failed peer review in so-called unscholarly, 'predatory' or academically suspect journals or publishers^[32] will be reduced.

The greater risk to status quo journals or publishers is that their reputations will suffer, trust in the efficiency of the editorial process may be lost, and if such errors are not corrected at the post-publication stage, as they should be^[33], then they might begin to encroach on the publishing research territory that was once exclusively assigned to the latter ('predatory') group of journals or publishers, creating a gray zone of publishing quality^[34]. More importantly, in such cases, if status quo journals and publishers continue to draw benefit (ranking, metrics, indexing, branding), despite the existence of wide-spread errors, there may be a worrying perception that they are unfairly receiving benefit from the publication of erroneous science^[35]. For example, using the stated price on the publishers' websites to access the papers listed in Table 1 (except evidently for open access papers), the total price to access these papers with errors (in this case 'tortured phrases' and lack of clarity) is US$ 1,040. While some may argue that the sale of papers with minor errors is a trivial issue, when papers carry unethical or fraudulent elements, the issue of the sale of those papers will increasingly become a reputational problem.

It is unclear at this time if the potentially thousands or more cases that have been detected thus far of linguistically-compromised papers, possibly using techniques to avoid the detection of textual similarity or plagiarism, are merely instances of non-native English speakers making frivolous, inadvertent and/or unintentional errors, or if there is a more sinister unethical component^[21,29], such as the intentional 'targeting' of lower-ranked journals, proceedings, or book chapters, to avoid the detection of such manipulation. Independent of the reason, or the intention of culprits, one aspect is clear: there has been failure in quality control by the authors, editors, publishers and all associated parties related to copyediting and textual verification. Consequently, there needs to be heightened accountability and transparency. Moreover, so that the same errors are not repeated by the same or other parties, reasons for their existence need to be transparently explained by authors, editors and publishers. Absent public transparency, it will be difficult, if not impossible, to reform publishing culture, procedures, technologies and tools. Therefore, novel ways to correct affected literature^[36], and to reform the culture of error-prone scientific publication, rewarding good behavior^[37], without leaving academia stigmatized and traumatized^[38], are needed.

Incidentally, two of the papers in Table 1, both published by IOP Publishing, have been retracted, with one of the reasons for their retraction being indicated in the retraction notices as the existence of 'tortured phrases'.

This paper and its interpretations have several limitations. Table 1 is only a small non-exhaustive sample of 35 papers, limited exclusively to the materials, computer and engineering sciences, which may or may not be representative of the total sample size, or range, of the issue of 'tortured phrases' as detected by Cabanac et al.^[21] and others. These cases, drawn primarily from the discovery by named and anonymous individuals at PubPeer, do not serve as a trend indicator of the frequency or the presence of other cases of 'tortured phrases' in the text, so it is still premature to draw conclusions regarding patterns or trends related to authors, journals or publishers. Readers and academics that conduct such analyses must be careful not to automatically associate cases of 'tortured phrases' with academic misconduct, even though some are hinting at this possibility^[21,23], and they must be careful about drawing potentially erroneous conclusions based on authors' country of origin or institutional affiliation to avoid potential racial, cultural or other profiling. The open access nature of the papers listed in Table 1 has been indicated, as has the JIF of the journals in which they have been published, but a detailed analysis of the open access versus subscription (pay to access) status of such papers is warranted. Finally, it is important to study the risk to the integrity of literature that cites these papers, and the way in which these erroneous texts have been interpreted and/or cloned into literature that have cited these papers.