Slay the Jabberwock
“Twas brillig, and the slithy toves / Did gyre and gimble in the wabe: / All mimsy were the borogoves, / And the mome raths outgrabe.”
Peer review is widely viewed as an essential step for ensuring scientific quality of a work and represents a cornerstone of scholarly publishing. On the other hand, those involved in the publishing process are often driven by incentives which may undermine the quality of published work.
Influence over the peer review process is real power, and not one that should be used lightly or for reasons of financial gain. But there are hundreds of “predatory publishers” that hardly review the papers they charge to publish, and sometimes undertake no review at all. But the veneer of credibility that an author gets by being published in a peer-reviewed journal starts to disappear as soon as the peer review process stops being trustworthy.
A text corpus is a large and structured collection of scientific texts used to do statistical analysis and hypothesis testing, checking occurrences or proving linguistic rules within a specific language.
Text corpora are the main knowledge base in several linguistics skills, including speech recognition and machine translation. Corpora and frequency lists derived from them are also useful for language teaching.
About a decade ago, some MIT students wrote a programme that randomly picked out clauses from a scientific text corpus. It then arranged the words in a syntactically correct way, although actual meaning was eliminated, making the text read like a nonsense piece of writing by Lewis Carroll. They used the text for submissions to a conference at which they suspected low submission standards to exist.
The article was accepted for presentation and thus was launched the SCIgen project.
Researchers have now taken another shot at the peer reviewing process by making randomly generated nonsense and fake reviews of actual published articles. The nonsense reviews were also accepted for publication.
While the algorithm “cannot possibly deceive any rigorous editorial procedure,” according to the researchers, it definitely works if the editor skim-reads the text (or doesn’t read it at all). The need for real writers, real editors and real reviewers in literature, research and scientific writing cannot be stressed enough. This is why the jabberwock must be slain.
https://pdos.csail.mit.edu/archive/scigen/#examples
(Jabberwocky illustration by John Tenniel)