Peer review sucks?

[bjn]

Article from a working scientist on why peer review sucks and should be done away with. Any comments?

https://www.experimental-history.com/p/ ... eer-review

[dyqik]

Peer review is worth exactly what the journals pay for it.

[KAJ]

I see that was published DEC 13, 2022. Has there been much change since? Not to my knowledge, but I'm out of the loop these days.

[Allo V Psycho]

What a witty and wise article! Unfortunately, it doesn't match my experience as author, reviewer and Editor-in-Chief, over nearly fifty years. As a researcher I usually found peer review of my work helpful, even if it is just because it represents the first couple of external readers. As an author It's really easy to read your draft as what you're thinking, rather than what you've actually written. Peer reviewers often raised a question which made me think, 'yes, I didn't put that very clearly'. Kind of like a fancy and very informed proof reader. And I used to recommend to my PhD students and post docs to actively engage in peer review - it makes you think about the scientific writing process rather than just the science (as well as sometimes giving you an advance heads up on something in very competitive and quickly moving fields)! Since I felt I benefitted from peer reviews, I also felt I had to pay that back by thoughtful reviews, sometimes spending a day or several days on the review, and being fair. Of course, our major subject journal also paid reviewers, which helped: £15 in 1980 would be about £80 now, so worth having. I know of one case where the authors of a paper found a review so helpful that they asked if the reviewer could be an author on the revised version (the reviewer declined).

But that was then. Now I rarely agree to review papers, nor recommend others to do so. I still do it as conscientiously as ever, if I do agree, but it is an exception. The process is not as useful as it once was. There are a number of factors behind this, not mentioned in the (excellent) link. One is open access (which I think is the real disaster here). It seemed so sensible. But it has also brought about a marked increase in the number of journals, because now there is a new market. The may not ALL be predatory. But they all have a financial interest in accepting work. And since researchers who have the money can now pretty much guarantee getting published no matter what the standard of their work is, they don't have to try so hard to make them good in advance. Anther factor is the rise of managerialism in science/Universities, where output was counted rather than rated on quality. If your promotion (or even your job) depends on the number of papers you publish, you will have to publish lots of papers to feed the kids and pay the mortgage. In my view the average standard of papers has dropped sharply over the years. And finally electronic journals don't have the page costs of print so can publish many more articles.

I know I will sound like a traditionalist/old fogey. But when I was EiC of a print journal (paid by library subscriptions), we received three times as many papers as we could publish. We were in competition with another (American) journal to be the lead journal in the field, so we wanted to choose the best possible papers (and improve them through review if we possibly could). Then we would get cited more often and beat the rivals - and we would be a desirable place to publish so we would get the best papers, in what felt like a virtuous circle. Honestly, it was rare to even receive a terrible paper - it would become an office joke. It wasn't generally worth sending us dross, so people didn't.

Now I reckon the majority of papers published in my field are dross. I don't always even trust them for honesty (and Steamy here hasn't encouraged me)! And there are too many being published to keep track of anyway. i don't write as many papers as I used to (because my career position/age means I can get away with not publishing) even though I have some interesting results - it's not worth the effort of submitting, and getting really poor quality reviews . Instead I share results informally through fora and social media. So I've ended up in something of the same place as the article author, but I assign different reasons for it.

[wilsontown]

I tend to agree with a lot of what Allo V Psycho writes above.

What do we actually want peer review to achieve? If we want it to spot fraudulent results, it doesn't really work for that. And if we want it to be quality assurance, then it doesn't really do that either. Perhaps the best thing I could say is that a peer reviewed paper is almost always going to be better than one that was published without being peer reviewed. Certainly in my experience reviews can be useless or unhelpful, but my papers have always been improved by the overall peer review process. Still, I'm no longer convinced that's worth the time and resources spent on it, mostly on a voluntary basis.

I'm currently a deputy editor in a reasonably well-known journal in my field, and I definitely recognise the bit about poor-quality submissions. I have recently received some absolute garbage and I don't hesitate to send these back without reviews as it would just be a waste of everyone's time. No doubt not all of my contributions have been outstanding additions to the literature, but I would have been embarrassed to submit some of the dross we are receiving.

[kerrya1]

I am part of a loose coalition of researchers and open research advocates at our institution trying to encourage the adoption of Open Peer Review. In theory it should make peer review a more discursive and collaborative process, and remove many of the common criticism of the current peer review process. But if it does in practice remains to be seen.