More fake papers - Computer science this time

Get your science fix here: research, quackery, activism and all the rest
Post Reply
User avatar
Bird on a Fire
Princess POW
Posts: 6861
Joined: Fri Oct 11, 2019 5:05 pm
Location: Portugal

More fake papers - Computer science this time

Post by Bird on a Fire » Mon Aug 09, 2021 2:35 pm

In April 2021, a series of strange phrases in journal articles piqued the interest of a group of computer scientists. The group, led by Guillaume Cabanac at the University of Toulouse in France, could not understand why researchers would use the terms ‘counterfeit consciousness’, ‘profound neural organization’ and ‘colossal information’ in place of the more widely recognized terms ‘artificial intelligence’, ‘deep neural network’ and ‘big data’.

Further investigation revealed that these strange terms — which they dub “tortured phrases” — are probably the result of automated translation or software that attempts to disguise plagiarism. And they seem to be rife in computer-science papers.
To get a sense of how many papers are affected, the researchers ran a search for 30 tortured phrases in journal articles indexed in the citation database Dimensions. They found more than 860 publications that included at least one of the phrases, 500 of which were published in a single journal: Microprocessors and Microsystems.
https://www.nature.com/articles/d41586-021-02134-0

Mostly Chinese authors, as usual.
He has the grace of a swan, the wisdom of an owl, and the eye of an eagle—ladies and gentlemen, this man is for the birds!

User avatar
Martin Y
After Pie
Posts: 2185
Joined: Mon Nov 11, 2019 1:08 pm

Re: More fake papers - Computer science this time

Post by Martin Y » Mon Aug 09, 2021 2:51 pm

Automatic translation of portions that weren't originally written in English is the obvious non-sinister explanation. Of course it could be word substitution to conceal plagiarism but unless they start finding examples of that it's not much of a smoking gun.

User avatar
basementer
Dorkwood
Posts: 1026
Joined: Mon Nov 11, 2019 1:03 pm
Location: Aotearoa
Contact:

Re: More fake papers - Computer science this time

Post by basementer » Mon Aug 09, 2021 4:04 pm

I rather like the phrase "flag commotion" as a replacement for Spoiler:
Money is just a substitute for luck anyway. - Tom Siddell

User avatar
Bird on a Fire
Princess POW
Posts: 6861
Joined: Fri Oct 11, 2019 5:05 pm
Location: Portugal

Re: More fake papers - Computer science this time

Post by Bird on a Fire » Mon Aug 09, 2021 4:06 pm

Martin Y wrote:
Mon Aug 09, 2021 2:51 pm
Automatic translation of portions that weren't originally written in English is the obvious non-sinister explanation. Of course it could be word substitution to conceal plagiarism but unless they start finding examples of that it's not much of a smoking gun.
Still, you'd expect reviewers and editors to be sufficiently familiar with field-specific buzzwords like "big data" to spot and correct the errors.

I suppose the best case scenario would be that the entire writing, reviewing and editing process occurred adequately but in Chinese, followed by Google translate that nobody checked?

It's pretty f.cking fishy.
He has the grace of a swan, the wisdom of an owl, and the eye of an eagle—ladies and gentlemen, this man is for the birds!

Holylol
Buzzberry
Posts: 45
Joined: Tue Nov 19, 2019 10:14 am

Re: More fake papers - Computer science this time

Post by Holylol » Mon Aug 09, 2021 6:20 pm

Some reporting in RetractionWatch on this journal:
https://retractionwatch.com/2021/07/12/ ... -expected/

There is a comment by G. Cabanac.

User avatar
Woodchopper
Stummy Beige
Posts: 3989
Joined: Sat Oct 12, 2019 9:05 am

Re: More fake papers - Computer science this time

Post by Woodchopper » Mon Aug 09, 2021 9:24 pm

Bird on a Fire wrote:
Mon Aug 09, 2021 4:06 pm
Martin Y wrote:
Mon Aug 09, 2021 2:51 pm
Automatic translation of portions that weren't originally written in English is the obvious non-sinister explanation. Of course it could be word substitution to conceal plagiarism but unless they start finding examples of that it's not much of a smoking gun.
Still, you'd expect reviewers and editors to be sufficiently familiar with field-specific buzzwords like "big data" to spot and correct the errors.

I suppose the best case scenario would be that the entire writing, reviewing and editing process occurred adequately but in Chinese, followed by Google translate that nobody checked?

It's pretty f.cking fishy.
Academia's dirty secret* is that there are many journals which will publish pretty much anything and exist solely to pad the CVs of the authors and editors [ETA and make money]. When they charge for access they are known as predatory journals, but as we've seen in the past with alternative medicine, journals can also be run as a career advancement club for all involved, and by financed via library subscriptions.

Active researchers in a field can avoid the b.llsh.t as they will know which are the dodgy journals. But it is a much bigger problem when outsiders aren't aware.

*Alright, its not much of a secret.

IvanV
Snowbonk
Posts: 462
Joined: Mon May 17, 2021 11:12 am

Re: More fake papers - Computer science this time

Post by IvanV » Tue Aug 10, 2021 12:48 pm

Woodchopper wrote:
Mon Aug 09, 2021 9:24 pm
Academia's dirty secret* is that there are many journals which will publish pretty much anything...

*Alright, its not much of a secret.
Such as satirical papers exposing the entire subject area as b.llsh.t.. Or at least pretentious vacuity if one considers that a different category from b.llsh.t.

User avatar
Woodchopper
Stummy Beige
Posts: 3989
Joined: Sat Oct 12, 2019 9:05 am

Re: More fake papers - Computer science this time

Post by Woodchopper » Tue Aug 10, 2021 4:01 pm

IvanV wrote:
Tue Aug 10, 2021 12:48 pm
Woodchopper wrote:
Mon Aug 09, 2021 9:24 pm
Academia's dirty secret* is that there are many journals which will publish pretty much anything...

*Alright, its not much of a secret.
Such as satirical papers exposing the entire subject area as b.llsh.t.. Or at least pretentious vacuity if one considers that a different category from b.llsh.t.
I think that’s different in that the journal editors appeared to be sincere. I’m thinking more of cases where the editors and reviewers don’t bother to read the submissions.

IvanV
Snowbonk
Posts: 462
Joined: Mon May 17, 2021 11:12 am

Re: More fake papers - Computer science this time

Post by IvanV » Tue Aug 10, 2021 4:51 pm

Woodchopper wrote:
Tue Aug 10, 2021 4:01 pm
IvanV wrote:
Tue Aug 10, 2021 12:48 pm
Woodchopper wrote:
Mon Aug 09, 2021 9:24 pm
Academia's dirty secret* is that there are many journals which will publish pretty much anything...

*Alright, its not much of a secret.
Such as satirical papers exposing the entire subject area as b.llsh.t.. Or at least pretentious vacuity if one considers that a different category from b.llsh.t.
I think that’s different in that the journal editors appeared to be sincere.
Sincere in running a journal to support an entire branch of study that was, in my opinion, mainly a career advancement club for academics, and providing very little benefit to society.

The journals play their part by enforcing this opaque language. A language so opaque, and at the same time imprecise, that even the practitioners could not easily see complete rubbish when it was written in it. This, I think, was what Sokal was trying to demonstrate in constructing his clever satire. If the journals made people write papers in plain English, as is often said they should do, and use precise terminology rather pretentious jargon, it would soon be clear how little many of these papers actually said.

So in my view it is just a different tactic to the same end.

Allo V Psycho
Snowbonk
Posts: 460
Joined: Sat Nov 16, 2019 8:18 am

Re: More fake papers - Computer science this time

Post by Allo V Psycho » Wed Aug 11, 2021 9:35 am

Martin Y wrote:
Mon Aug 09, 2021 2:51 pm
Automatic translation of portions that weren't originally written in English is the obvious non-sinister explanation. Of course it could be word substitution to conceal plagiarism but unless they start finding examples of that it's not much of a smoking gun.
What's interesting, though, is that the 'tortured phrases' are 'Rogetisations' of English words. While it is certainly not conclusive evidence of plagiarism with Rogetisation, and evidence of the plagiarism still needs to be found, but at the moment, I tilt in the direction of being concerned.

User avatar
Martin Y
After Pie
Posts: 2185
Joined: Mon Nov 11, 2019 1:08 pm

Re: More fake papers - Computer science this time

Post by Martin Y » Wed Aug 11, 2021 9:47 am

Allo V Psycho wrote:
Wed Aug 11, 2021 9:35 am
Martin Y wrote:
Mon Aug 09, 2021 2:51 pm
Automatic translation of portions that weren't originally written in English is the obvious non-sinister explanation. Of course it could be word substitution to conceal plagiarism but unless they start finding examples of that it's not much of a smoking gun.
What's interesting, though, is that the 'tortured phrases' are 'Rogetisations' of English words. While it is certainly not conclusive evidence of plagiarism with Rogetisation, and evidence of the plagiarism still needs to be found, but at the moment, I tilt in the direction of being concerned.
Yes, good point and I hadn't thought of that. It's not transliteration of another language's idioms, it's word-for-word transliteration of English terms.

secret squirrel
Snowbonk
Posts: 495
Joined: Wed Nov 13, 2019 12:42 pm

Re: More fake papers - Computer science this time

Post by secret squirrel » Wed Aug 11, 2021 10:08 am

IvanV wrote:
Tue Aug 10, 2021 4:51 pm
Sincere in running a journal to support an entire branch of study that was, in my opinion, mainly a career advancement club for academics, and providing very little benefit to society.

The journals play their part by enforcing this opaque language. A language so opaque, and at the same time imprecise, that even the practitioners could not easily see complete rubbish when it was written in it. This, I think, was what Sokal was trying to demonstrate in constructing his clever satire. If the journals made people write papers in plain English, as is often said they should do, and use precise terminology rather pretentious jargon, it would soon be clear how little many of these papers actually said.

So in my view it is just a different tactic to the same end.
It's interesting that while as an experiment Socal's hoax was more or less as badly designed as it could possibly be, it's still taken to be strong evidence of something by a lot of very objective people who just happened to believe that thing anyway (which is not to say that that thing is false).

I can point to at least one occasion when absolute nonsense was published in one of the world's top three pure maths journals, despite extremely rigorous peer review and filtering by the almost painfully high profile editorial board. Having the right name and saying something the editors want to hear can go a long way, even in the hardest sciences.

User avatar
sTeamTraen
After Pie
Posts: 2107
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: More fake papers - Computer science this time

Post by sTeamTraen » Wed Aug 11, 2021 12:51 pm

Woodchopper wrote:
Mon Aug 09, 2021 9:24 pm
Academia's dirty secret* is that there are many journals which will publish pretty much anything and exist solely to pad the CVs of the authors and editors [ETA and make money]. When they charge for access they are known as predatory journals, but as we've seen in the past with alternative medicine, journals can also be run as a career advancement club for all involved, and by financed via library subscriptions.

Active researchers in a field can avoid the b.llsh.t as they will know which are the dodgy journals. But it is a much bigger problem when outsiders aren't aware.

*Alright, its not much of a secret.
I think a bigger problem is when journals from notionally reputable publishers are doing much the same thing. Every academic gets weirdly-worded spam ("Greeting for the Day, we are Eagerly Awaiting Your Tremendous Research!!") soliciting duff papers for money, but there is a reasonable belief that if a journal is published by Taylor & Francis or Elsevier that there will be a non-asleep action editor and reviewers reading the manuscript. Not always true, sadly.
Something something hammer something something nail

User avatar
Bird on a Fire
Princess POW
Posts: 6861
Joined: Fri Oct 11, 2019 5:05 pm
Location: Portugal

Re: More fake papers - Computer science this time

Post by Bird on a Fire » Thu Aug 12, 2021 3:39 am

For-profit publishing is garbage and it needs to die immediately. It prevents most people from accessing research (legally) and creates all sorts of perverse incentives.

Since I entered academia (if undergrad even counts) about 11 years ago, preprint servers have taken off massively. They are generally run on a non-profit basis and are free to access, so the academic community can clearly set up dependable systems to host pdfs as well as doing all the writing, reviewing and most of the editing for free. So why we're diverting billions of (mostly taxpayers' and donors') money to a handful of corporations who in most cases add basically zero value is beyond me.
He has the grace of a swan, the wisdom of an owl, and the eye of an eagle—ladies and gentlemen, this man is for the birds!

secret squirrel
Snowbonk
Posts: 495
Joined: Wed Nov 13, 2019 12:42 pm

Re: More fake papers - Computer science this time

Post by secret squirrel » Thu Aug 12, 2021 11:00 am

Bird on a Fire wrote:
Thu Aug 12, 2021 3:39 am
For-profit publishing is garbage and it needs to die immediately. It prevents most people from accessing research (legally) and creates all sorts of perverse incentives.

Since I entered academia (if undergrad even counts) about 11 years ago, preprint servers have taken off massively. They are generally run on a non-profit basis and are free to access, so the academic community can clearly set up dependable systems to host pdfs as well as doing all the writing, reviewing and most of the editing for free. So why we're diverting billions of (mostly taxpayers' and donors') money to a handful of corporations who in most cases add basically zero value is beyond me.
It's because name journals are prestige brokers, and academia both runs on prestige and lacks the leadership to coordinate an alternative system. In an ideal world, professional societies and funding bodies would establish new diamond open access journals or equivalents using the funds currently used to pay for journal subscriptions and gold open access publishing fees. These would be large enough to handle the required volume, and diverse enough in prestige levels to play the filtering role that sustains the current journal system. But you'd have to persuade thousands of academics to give up high profile editing jobs to take on the same responsibilities at less prestigious new journals. You'd have to persuade hiring committees that papers in some of these new journals should carry the same weight as papers in Nature etc. And the people doing this persuading would be people who have benefited the most from the current system.

tom p
Dorkwood
Posts: 1211
Joined: Mon Nov 11, 2019 1:14 pm
Location: the low countries

Re: More fake papers - Computer science this time

Post by tom p » Thu Sep 02, 2021 2:39 pm

secret squirrel wrote:
Wed Aug 11, 2021 10:08 am
IvanV wrote:
Tue Aug 10, 2021 4:51 pm
Sincere in running a journal to support an entire branch of study that was, in my opinion, mainly a career advancement club for academics, and providing very little benefit to society.

The journals play their part by enforcing this opaque language. A language so opaque, and at the same time imprecise, that even the practitioners could not easily see complete rubbish when it was written in it. This, I think, was what Sokal was trying to demonstrate in constructing his clever satire. If the journals made people write papers in plain English, as is often said they should do, and use precise terminology rather pretentious jargon, it would soon be clear how little many of these papers actually said.

So in my view it is just a different tactic to the same end.
It's interesting that while as an experiment Socal's hoax was more or less as badly designed as it could possibly be
Was it not satire, rather than science? Pretty sure they are different things.
There's a reason Armando Iannucci has baftas, rather than Nobels

secret squirrel
Snowbonk
Posts: 495
Joined: Wed Nov 13, 2019 12:42 pm

Re: More fake papers - Computer science this time

Post by secret squirrel » Thu Sep 02, 2021 2:44 pm

tom p wrote:
Thu Sep 02, 2021 2:39 pm
secret squirrel wrote:
Wed Aug 11, 2021 10:08 am
It's interesting that while as an experiment Socal's hoax was more or less as badly designed as it could possibly be
Was it not satire, rather than science? Pretty sure they are different things.
There's a reason Armando Iannucci has baftas, rather than Nobels
He described it as an experiment:
So, to test the prevailing intellectual standards, I decided to try a modest (though admittedly uncontrolled) experiment
Certainly many people have drawn general conclusions from it.

Post Reply