There is too much science for science to work properly

Get your science fix here: research, quackery, activism and all the rest
Post Reply
User avatar
Bird on a Fire
Princess POW
Posts: 7177
Joined: Fri Oct 11, 2019 5:05 pm
Location: Portugal

There is too much science for science to work properly

Post by Bird on a Fire » Fri Oct 08, 2021 7:06 pm

According to this paper in PNAS:
In many academic fields, the number of papers published each year has increased significantly over time. Policy measures aim to increase the quantity of scientists, research funding, and scientific output, which is measured by the number of papers produced. These quantitative metrics determine the career trajectories of scholars and evaluations of academic departments, institutions, and nations. Whether and how these increases in the numbers of scientists and papers translate into advances in knowledge is unclear, however. Here, we first lay out a theoretical argument for why too many papers published each year in a field can lead to stagnation rather than advance. The deluge of new papers may deprive reviewers and readers the cognitive slack required to fully recognize and understand novel ideas. Competition among many new ideas may prevent the gradual accumulation of focused attention on a promising new idea. Then, we show data supporting the predictions of this theory. When the number of papers published per year in a scientific field grows large, citations flow disproportionately to already well-cited papers; the list of most-cited papers ossifies; new papers are unlikely to ever become highly cited, and when they do, it is not through a gradual, cumulative process of attention gathering; and newly published papers become unlikely to disrupt existing work. These findings suggest that the progress of large scientific fields may be slowed, trapped in existing canon. Policy measures shifting how scientific work is produced, disseminated, consumed, and rewarded may be called for to push fields into new, more fertile areas of study.
Perhaps turning researchers into content providers for large publishing corporations wasn't the best economic model to incentivise scientific progress after all.
He has the grace of a swan, the wisdom of an owl, and the eye of an eagle—ladies and gentlemen, this man is for the birds!

User avatar
Fishnut
Dorkwood
Posts: 1587
Joined: Mon Nov 11, 2019 1:15 pm
Location: UK

Re: There is too much science for science to work properly

Post by Fishnut » Fri Oct 08, 2021 7:42 pm

That sounds like a really interesting paper.

I remember reading this Guardian Long Read a few years ago (I can't remember if it was discussed in previous iterations of this forum or not) and being blown away. I had no idea that Rupert f.cking Murdoch was responsible for the proliferation of journals, but it does make the insane model of researchers do to the work, researchers peer-review others' work for free, researchers pay to publish their work and researchers pay to access the work of others while the publishers make a fortune make much more sense.
it's okay to say "I don't know"

jeremy
Pop Chop
Posts: 1
Joined: Sun Nov 17, 2019 4:15 pm

Re: There is too much science for science to work properly

Post by jeremy » Sat Oct 09, 2021 2:56 pm

Fishnut wrote:
Fri Oct 08, 2021 7:42 pm
That sounds like a really interesting paper.

I remember reading this Guardian Long Read a few years ago (I can't remember if it was discussed in previous iterations of this forum or not) and being blown away. I had no idea that Rupert f.cking Murdoch was responsible for the proliferation of journals, but it does make the insane model of researchers do to the work, researchers peer-review others' work for free, researchers pay to publish their work and researchers pay to access the work of others while the publishers make a fortune make much more sense.
I think you mean Robert f.cking Maxwell.

User avatar
Fishnut
Dorkwood
Posts: 1587
Joined: Mon Nov 11, 2019 1:15 pm
Location: UK

Re: There is too much science for science to work properly

Post by Fishnut » Sat Oct 09, 2021 9:46 pm

jeremy wrote:
Sat Oct 09, 2021 2:56 pm
Fishnut wrote:
Fri Oct 08, 2021 7:42 pm
That sounds like a really interesting paper.

I remember reading this Guardian Long Read a few years ago (I can't remember if it was discussed in previous iterations of this forum or not) and being blown away. I had no idea that Rupert f.cking Murdoch was responsible for the proliferation of journals, but it does make the insane model of researchers do to the work, researchers peer-review others' work for free, researchers pay to publish their work and researchers pay to access the work of others while the publishers make a fortune make much more sense.
I think you mean Robert f.cking Maxwell.
You're right, I did. I even checked the article to make sure I had the right f.cker and still managed to mix them up.
it's okay to say "I don't know"

User avatar
Bird on a Fire
Princess POW
Posts: 7177
Joined: Fri Oct 11, 2019 5:05 pm
Location: Portugal

Re: There is too much science for science to work properly

Post by Bird on a Fire » Sat Oct 09, 2021 10:22 pm

A couple of times in my PhD I've tried to stay on top of the literature by getting table of contents and alerts for all the journals that regularly publish my kind of stuff.

It was hours a week just to sift through titles, ending with a million tabs for abstracts I'd skim, and end up wanting to read about 10% of them, of which I could actually access less than half.

These days I just rely on tweets, emails and chatting to people I know. And I think everyone else does too.
He has the grace of a swan, the wisdom of an owl, and the eye of an eagle—ladies and gentlemen, this man is for the birds!

User avatar
sTeamTraen
After Pie
Posts: 2167
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: There is too much science for science to work properly

Post by sTeamTraen » Sat Oct 09, 2021 10:35 pm

One of my hobbies at conferences is suggesting to my fellow psychologists that we would all be better off with 90% fewer researchers all getting 10x as much grant money so we could run decent sample sizes. Individually they are surprisingly receptive to this idea, presumably because they all imagine that other people would make up the 90%.
Something something hammer something something nail

User avatar
dyqik
Light of Blast
Posts: 4323
Joined: Wed Sep 25, 2019 4:19 pm
Location: Masshole
Contact:

Re: There is too much science for science to work properly

Post by dyqik » Sun Oct 10, 2021 12:27 am

I can't remember that last time I read a journal or astro-ph.

User avatar
Woodchopper
Stummy Beige
Posts: 4120
Joined: Sat Oct 12, 2019 9:05 am

Re: There is too much science for science to work properly

Post by Woodchopper » Sun Oct 10, 2021 11:15 am

Bird on a Fire wrote:
Sat Oct 09, 2021 10:22 pm
A couple of times in my PhD I've tried to stay on top of the literature by getting table of contents and alerts for all the journals that regularly publish my kind of stuff.

It was hours a week just to sift through titles, ending with a million tabs for abstracts I'd skim, and end up wanting to read about 10% of them, of which I could actually access less than half.

These days I just rely on tweets, emails and chatting to people I know. And I think everyone else does too.
Once you get your PhD it'll get a lot easier. Just set up a Google Scholar alert for anything that cites one of your publications. Ignore the rest as irrelevant.

User avatar
Stephanie
After Pie
Posts: 2306
Joined: Wed Sep 25, 2019 4:38 pm
Location: clinging tenaciously to your buttocks

Re: There is too much science for science to work properly

Post by Stephanie » Sun Oct 10, 2021 3:04 pm

I remember reading a commentary about a psychological theory that included this. I think about it quite a lot.
As a doctoral student, I read all of psychology whether it be biological, social, developmental or cognitive. I read philosophy, sociology and psychiatry and I even read novels related in all sorts of ways to the human condition. Now my students drown in the literature. We have published so many studies, so many theories in so many journals and books that they cannot possibly grasp it all and an academic learned helplessness develops. So the solution is known as ‘being focused’ or ‘specialising’ and similarly academic careers are based on ‘having a reputation in a field’ or making a contribution ‘to an area’. Yet I have examined students whose subject is cancer who have never read the literature on heart disease, students who study smoking but have ‘focused only’ on the qualitative research and those who only cite research from the UK as this is where their study is. And likewise many researchers may know their ‘field’ and are known ‘for an area’ but remain blissfully unaware of all the other literature that could enrich their work. And then these areas become inbred, stunted and stale as there is no chance of cross-fertilisation as they know no other area to cross-fertilise with. The vastness of the literature therefore means that it remains unread causing a narrowing vision of what psychology is and our research becomes increasingly obvious.
"I got a flu virus named after me 'cause I kissed a bat on a dare."

User avatar
Woodchopper
Stummy Beige
Posts: 4120
Joined: Sat Oct 12, 2019 9:05 am

Re: There is too much science for science to work properly

Post by Woodchopper » Wed Oct 13, 2021 6:28 am

Stephanie wrote:
Sun Oct 10, 2021 3:04 pm
I remember reading a commentary about a psychological theory that included this. I think about it quite a lot.
As a doctoral student, I read all of psychology whether it be biological, social, developmental or cognitive. I read philosophy, sociology and psychiatry and I even read novels related in all sorts of ways to the human condition. Now my students drown in the literature. We have published so many studies, so many theories in so many journals and books that they cannot possibly grasp it all and an academic learned helplessness develops. So the solution is known as ‘being focused’ or ‘specialising’ and similarly academic careers are based on ‘having a reputation in a field’ or making a contribution ‘to an area’. Yet I have examined students whose subject is cancer who have never read the literature on heart disease, students who study smoking but have ‘focused only’ on the qualitative research and those who only cite research from the UK as this is where their study is. And likewise many researchers may know their ‘field’ and are known ‘for an area’ but remain blissfully unaware of all the other literature that could enrich their work. And then these areas become inbred, stunted and stale as there is no chance of cross-fertilisation as they know no other area to cross-fertilise with. The vastness of the literature therefore means that it remains unread causing a narrowing vision of what psychology is and our research becomes increasingly obvious.
Well yes and no.

I agree that so much is published that would take a huge amount of time for someone to master a subject beyond the field in which they work, and those fields tend to be defined pretty narrowly.

But its not difficult to use someone else's mastery. Scientists regularly publish summaries of the literature in their field, and so systematic reviews, meta analysis etc have a vital role to play in research. As I advised a PhD student a couple of days ago, its not necessary to read a huge number of articles from another field, just go to Google Scholar and find someone else's literature review. Its also possible to get a basic understanding of the key findings in a field from reading textbooks or publications designed for a lay audience (I'm thinking of reports written by academics more than popular science bestsellers). Paywalls used to be a problem, but vast amounts of material are now available for free or for very little money.

Lots of methods are also sufficiently similar that people can appraise a paper and understand some of its basic strengths and weaknesses even if they are not experts in that field - for example similar statistical methods are used in fields as diverse as economics, medicine and ecology. So an expert on cancer should be able to read a paper on heart disease (though of course that doesn't apply elsewhere, a biologist might struggle to understand a physics paper).

Assuming that the students mentioned are doing graduate degrees I think that the problem is more a culture of intellectual Fordism in which students are treated as items on a production line. In that environment reading widely can be seen as a distraction from the purpose of getting their degree certificate and necessary number of publications on time.

User avatar
dyqik
Light of Blast
Posts: 4323
Joined: Wed Sep 25, 2019 4:19 pm
Location: Masshole
Contact:

Re: There is too much science for science to work properly

Post by dyqik » Wed Oct 13, 2021 3:51 pm

Stephanie wrote:
Sun Oct 10, 2021 3:04 pm
I remember reading a commentary about a psychological theory that included this. I think about it quite a lot.
As a doctoral student, I read all of psychology whether it be biological, social, developmental or cognitive. I read philosophy, sociology and psychiatry and I even read novels related in all sorts of ways to the human condition. Now my students drown in the literature. We have published so many studies, so many theories in so many journals and books that they cannot possibly grasp it all and an academic learned helplessness develops. So the solution is known as ‘being focused’ or ‘specialising’ and similarly academic careers are based on ‘having a reputation in a field’ or making a contribution ‘to an area’. Yet I have examined students whose subject is cancer who have never read the literature on heart disease, students who study smoking but have ‘focused only’ on the qualitative research and those who only cite research from the UK as this is where their study is. And likewise many researchers may know their ‘field’ and are known ‘for an area’ but remain blissfully unaware of all the other literature that could enrich their work. And then these areas become inbred, stunted and stale as there is no chance of cross-fertilisation as they know no other area to cross-fertilise with. The vastness of the literature therefore means that it remains unread causing a narrowing vision of what psychology is and our research becomes increasingly obvious.
This is why review articles and graduate level textbooks exist.

User avatar
Woodchopper
Stummy Beige
Posts: 4120
Joined: Sat Oct 12, 2019 9:05 am

Re: There is too much science for science to work properly

Post by Woodchopper » Wed Oct 13, 2021 3:53 pm

dyqik wrote:
Wed Oct 13, 2021 3:51 pm
Stephanie wrote:
Sun Oct 10, 2021 3:04 pm
I remember reading a commentary about a psychological theory that included this. I think about it quite a lot.
As a doctoral student, I read all of psychology whether it be biological, social, developmental or cognitive. I read philosophy, sociology and psychiatry and I even read novels related in all sorts of ways to the human condition. Now my students drown in the literature. We have published so many studies, so many theories in so many journals and books that they cannot possibly grasp it all and an academic learned helplessness develops. So the solution is known as ‘being focused’ or ‘specialising’ and similarly academic careers are based on ‘having a reputation in a field’ or making a contribution ‘to an area’. Yet I have examined students whose subject is cancer who have never read the literature on heart disease, students who study smoking but have ‘focused only’ on the qualitative research and those who only cite research from the UK as this is where their study is. And likewise many researchers may know their ‘field’ and are known ‘for an area’ but remain blissfully unaware of all the other literature that could enrich their work. And then these areas become inbred, stunted and stale as there is no chance of cross-fertilisation as they know no other area to cross-fertilise with. The vastness of the literature therefore means that it remains unread causing a narrowing vision of what psychology is and our research becomes increasingly obvious.
This is why review articles and graduate level textbooks exist.
Yes, you just summed up my 300 word post in a sentence.

User avatar
Bird on a Fire
Princess POW
Posts: 7177
Joined: Fri Oct 11, 2019 5:05 pm
Location: Portugal

Re: There is too much science for science to work properly

Post by Bird on a Fire » Wed Oct 13, 2021 9:02 pm

dyqik wrote:
Wed Oct 13, 2021 3:51 pm
Stephanie wrote:
Sun Oct 10, 2021 3:04 pm
I remember reading a commentary about a psychological theory that included this. I think about it quite a lot.
As a doctoral student, I read all of psychology whether it be biological, social, developmental or cognitive. I read philosophy, sociology and psychiatry and I even read novels related in all sorts of ways to the human condition. Now my students drown in the literature. We have published so many studies, so many theories in so many journals and books that they cannot possibly grasp it all and an academic learned helplessness develops. So the solution is known as ‘being focused’ or ‘specialising’ and similarly academic careers are based on ‘having a reputation in a field’ or making a contribution ‘to an area’. Yet I have examined students whose subject is cancer who have never read the literature on heart disease, students who study smoking but have ‘focused only’ on the qualitative research and those who only cite research from the UK as this is where their study is. And likewise many researchers may know their ‘field’ and are known ‘for an area’ but remain blissfully unaware of all the other literature that could enrich their work. And then these areas become inbred, stunted and stale as there is no chance of cross-fertilisation as they know no other area to cross-fertilise with. The vastness of the literature therefore means that it remains unread causing a narrowing vision of what psychology is and our research becomes increasingly obvious.
This is why review articles and graduate level textbooks exist.
Well sort of, but do many students (or other researchers) actually tend to read reviews and textbooks from outside their immediate field?
He has the grace of a swan, the wisdom of an owl, and the eye of an eagle—ladies and gentlemen, this man is for the birds!

User avatar
dyqik
Light of Blast
Posts: 4323
Joined: Wed Sep 25, 2019 4:19 pm
Location: Masshole
Contact:

Re: There is too much science for science to work properly

Post by dyqik » Thu Oct 14, 2021 12:12 am

No, but if you decided that you wanted to know something about something outside your field, it's where you would start.

Post Reply