Scrutable

... about dishonest science, in PLOS Biology.

Good rant, well done.

Nero wrote: ↑

Wed Sep 11, 2024 7:17 am

Good rant, well done.

Yes, very good, but also frustrating, as is the way with stuff that deserves ranting I guess.

Thank you so much for this. There's a few bits I didn't quite follow, implications rather than written out in full, that doubtless the practitioners get but I don't. The most cryptic was this one, there's no additional info on the Xwitter link.

In another case, a field-leading journal rejected a completed Stage 2 Registered Report on the basis of the results obtained after pre-registration, defeating the purpose of this article type.

Is the implication that the data showed a null result, so the journal rejected the paper because that's too boring? Whereas actually it is important to know that experiment failed to demonstrate the hypothesis.

This is so incredibly depressing, but also utterly unsurprising. My job is to promote and support Open Research in a Russell Group Uni but it's like banging my head off a brick wall much of the time. Funders set requirements for grant holders but there are no consequences for just ignoring them, publisher requirements are mostly ignored, REF pays the barest lip service to OR, reproducibility, and transparency, and the reward and recognition systems in academic are still almost entirely based on citations / H-index and grant income.

Early career researchers are normal keen and can see the benefits of OR, but without the active support and encouragement from PIs and dept/school leaders they quickly realise that they need to keep playing the same old games or leave academia.

Anyway, excellent rant.

IvanV wrote: ↑

Wed Sep 11, 2024 10:03 pm

Thank you so much for this. There's a few bits I didn't quite follow, implications rather than written out in full, that doubtless the practitioners get but I don't. The most cryptic was this one, there's no additional info on the Xwitter link.

In another case, a field-leading journal rejected a completed Stage 2 Registered Report on the basis of the results obtained after pre-registration, defeating the purpose of this article type.

Is the implication that the data showed a null result, so the journal rejected the paper because that's too boring? Whereas actually it is important to know that experiment failed to demonstrate the hypothesis.

Yes. Of course this applies to any study, within the limits of what we can learn given its statistical power, but with a Registered Report in particular the whole point is that you get the reviewing of the method, etc, done first and then the article is guaranteed to be accepted even if it doesn't find anything. But many journals just pay lip service to that as a general practice, while continuing to cherry-pick positive results for themselves.

shpalman wrote: ↑

Thu Sep 26, 2024 10:27 am

Scientific rigor proponents retract paper on benefits of scientific rigor

I think this is a case where the senior people should have had (except that never happens) a "kill your darlings" moment on the author list. Apparently they have been working on this paper in various ways since 2012, and the last author really believes in a Heisenberg-style explanation for the decline effect (which would be totally Upminster). I suspect the whole project had become a sort of living zombie, and it didn't need a lot of external critical insight after 11 years of groupthink and/or "Please let's get this over with" to spot the accumulated problems.