Fake science playing into the hands of deniers

Get your science fix here: research, quackery, activism and all the rest
kerrya1
Sindis Poop
Posts: 88
Joined: Tue Nov 12, 2019 11:13 am

Re: Fake science playing into the hands of deniers

Post by kerrya1 » Wed May 12, 2021 1:09 pm

I really really want the researchers caught doing this to be punished severely. Whether it is the result of malfeasence or incompentence is irrelevant, people like me need to have case studies that we can use to persuade researchers to do the following routinely:

1) Properly organise their files and folders, using easily understood or clearly documented naming conventions and structures as appropriate;
2) Use institution provided networked storage to improve the back-up and security of data, when this isn't possible then properly organised and documented back-up processes need to be in place;
3) Create relevant documentation and metadata for all data produced/collected at the time of data collection rather than making it up at the end;
4) Ensure that all relevant protocols, code, algorithims, grimoires, etc are made available alongside the data (IP permitting)
5) Have a plan for the long-term preservation and sharing of the data from the outset, and follow it even if there are no publications or other outputs from the projects - even negative data can have value or maybe useful for other research purposes.

Following the FAIR principles, I would hope that all the active researchers on here are already familiar with these.

If we can make adherance to good Open Science practices the norm rather than the exception then sloppy or fraudulent work should be much quicker and easier to identify.

User avatar
sTeamTraen
After Pie
Posts: 1900
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: Fake science playing into the hands of deniers

Post by sTeamTraen » Wed May 12, 2021 9:34 pm

We also need what I call the airport security model of verification. When you're at the airport, and the security gate beeps (whether because you forgot to take your belt off or simply because it beeps one time in 20 randomly), you don't get to say "But I am Professor Important, how dare you accuse me of being a terrorist". You take off your belt, assume the position, etc, and you don't complain.

Imagine if submitting a paper to a journal meant there was a 1 in N chance that the code and data (which you *have* to submit; workarounds can be made for sensitive stuff, but that's a small minority and none of it is the nucular codes) would be audited in public. If you're trying to smuggle fake data on board, you will be publicly exposed. It wouldn't be long before fraud dropped to near zero. At the moment, I wouldn't be surprised if 20% or more of papers in some fields are basically either completely fabricated, or so "enhanced" as to be indistinguishable from having been fabricated.
Something something hammer something something nail

User avatar
Bird on a Fire
Light of Blast
Posts: 5996
Joined: Fri Oct 11, 2019 5:05 pm
Location: nadir of brie

Re: Fake science playing into the hands of deniers

Post by Bird on a Fire » Wed May 12, 2021 10:32 pm

Plus, automated checks. People are spotting duplicated images with AI, for instance, and there's the GRIM test and others for finding statistical inconsistencies.

Journals get billions of our money every year for PDF hosting. At the very least they could start earning their keep by investing 0.01% of it in some open-source tools.

Yes, they'd be gameable by industrial paper mills (cough China cough), but it would be better than nothing.
He has the grace of a swan, the wisdom of an owl, and the eye of an eagle—ladies and gentlemen, this man is for the birds!

User avatar
sTeamTraen
After Pie
Posts: 1900
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: Fake science playing into the hands of deniers

Post by sTeamTraen » Thu May 13, 2021 11:41 pm

Bird on a Fire wrote:
Wed May 12, 2021 10:32 pm
and there's the GRIM test and others for finding statistical inconsistencies.
I am familiar with that one. :lol: Our paper that introduced it has over 100 citations now, which is bonkers.

But it's only of use for integer data.
Something something hammer something something nail

User avatar
dyqik
Stummy Beige
Posts: 3847
Joined: Wed Sep 25, 2019 4:19 pm
Location: Masshole
Contact:

Re: Fake science playing into the hands of deniers

Post by dyqik » Fri May 14, 2021 1:22 am

kerrya1 wrote:
Wed May 12, 2021 1:09 pm
I really really want the researchers caught doing this to be punished severely. Whether it is the result of malfeasence or incompentence is irrelevant, people like me need to have case studies that we can use to persuade researchers to do the following routinely:

2) Use institution provided networked storage to improve the back-up and security of data, when this isn't possible then properly organised and documented back-up processes need to be in place;
It would be great if my organization, the largest government museum institution in the richest country in the world would fund backup storage for my government funded and FOIAble work. But our IT people won't do it.

kerrya1
Sindis Poop
Posts: 88
Joined: Tue Nov 12, 2019 11:13 am

Re: Fake science playing into the hands of deniers

Post by kerrya1 » Fri May 14, 2021 1:43 pm

dyqik wrote:
Fri May 14, 2021 1:22 am
kerrya1 wrote:
Wed May 12, 2021 1:09 pm
I really really want the researchers caught doing this to be punished severely. Whether it is the result of malfeasence or incompentence is irrelevant, people like me need to have case studies that we can use to persuade researchers to do the following routinely:

2) Use institution provided networked storage to improve the back-up and security of data, when this isn't possible then properly organised and documented back-up processes need to be in place;
It would be great if my organization, the largest government museum institution in the richest country in the world would fund backup storage for my government funded and FOIAble work. But our IT people won't do it.
For f.cks sake that is ludicrious, obviously a decision made by short-sighted morons with no cost of recreating lost data :evil:

User avatar
dyqik
Stummy Beige
Posts: 3847
Joined: Wed Sep 25, 2019 4:19 pm
Location: Masshole
Contact:

Re: Fake science playing into the hands of deniers

Post by dyqik » Fri May 14, 2021 2:22 pm

kerrya1 wrote:
Fri May 14, 2021 1:43 pm
dyqik wrote:
Fri May 14, 2021 1:22 am
kerrya1 wrote:
Wed May 12, 2021 1:09 pm
I really really want the researchers caught doing this to be punished severely. Whether it is the result of malfeasence or incompentence is irrelevant, people like me need to have case studies that we can use to persuade researchers to do the following routinely:

2) Use institution provided networked storage to improve the back-up and security of data, when this isn't possible then properly organised and documented back-up processes need to be in place;
It would be great if my organization, the largest government museum institution in the richest country in the world would fund backup storage for my government funded and FOIAble work. But our IT people won't do it.
For f.cks sake that is ludicrious, obviously a decision made by short-sighted morons with no cost of recreating lost data :evil:
They have a fully backed up multiuser linux system, where users can do stuff. But I'm using Windows specific software on computers that I have to admin myself as I need to install and remove hardware and software on them, and our division's Computing Facility won't provide a networked backup system to things other than their linux server cluster.

User avatar
bolo
Catbabel
Posts: 764
Joined: Mon Nov 11, 2019 1:17 pm
Location: Washington DC

Re: Fake science playing into the hands of deniers

Post by bolo » Fri May 14, 2021 2:53 pm

Will they pay for a subscription to a cloud backup service like Carbonite?

User avatar
dyqik
Stummy Beige
Posts: 3847
Joined: Wed Sep 25, 2019 4:19 pm
Location: Masshole
Contact:

Re: Fake science playing into the hands of deniers

Post by dyqik » Sat May 15, 2021 12:04 pm

bolo wrote:
Fri May 14, 2021 2:53 pm
Will they pay for a subscription to a cloud backup service like Carbonite?
They do give us Dropbox, One Drive and Google Drive. But these aren't exactly archiving and backup solutions.

I can easily generate half a terabyte a day in simulation outputs. I don't need to keep all of them, but I do need to keep some day's outputs.

User avatar
shpalman
Light of Blast
Posts: 4509
Joined: Mon Nov 11, 2019 12:53 pm
Location: One step beyond

Re: Fake science playing into the hands of deniers

Post by shpalman » Sat May 15, 2021 12:28 pm

I pay for my own 2TB of Dropbox, and the Polimi gives us 1TB of OneDrive (a pain in the arse to use via Linux as file storage but handy to be able to work on shared MS Office documents in the browser) so it's kind of adorable that our group has been granted 100 Gb of space on a NAS in Milan. My Linux box, which basically runs a big smb share for everyone in the group here in Como, has 8TB of space in a RAID5 array.

At least I'm able to actually access it now from Como, we used to be on a completely different network, that's why I never bothered asking for more space there (and resisted calls that I should manage it). I think I maxed it out by backing up some of our data and growth logs to it just because if you don't use the space you won't get more.

I was also once running an experiment at the ESRF in a period in which data collection kept failing because their shared disk space kept running out of inodes.
molto tricky

Post Reply