Page 1 of 1

There is too much science for science to work properly

Posted: Fri Oct 08, 2021 7:06 pm
by Bird on a Fire
According to this paper in PNAS:
In many academic fields, the number of papers published each year has increased significantly over time. Policy measures aim to increase the quantity of scientists, research funding, and scientific output, which is measured by the number of papers produced. These quantitative metrics determine the career trajectories of scholars and evaluations of academic departments, institutions, and nations. Whether and how these increases in the numbers of scientists and papers translate into advances in knowledge is unclear, however. Here, we first lay out a theoretical argument for why too many papers published each year in a field can lead to stagnation rather than advance. The deluge of new papers may deprive reviewers and readers the cognitive slack required to fully recognize and understand novel ideas. Competition among many new ideas may prevent the gradual accumulation of focused attention on a promising new idea. Then, we show data supporting the predictions of this theory. When the number of papers published per year in a scientific field grows large, citations flow disproportionately to already well-cited papers; the list of most-cited papers ossifies; new papers are unlikely to ever become highly cited, and when they do, it is not through a gradual, cumulative process of attention gathering; and newly published papers become unlikely to disrupt existing work. These findings suggest that the progress of large scientific fields may be slowed, trapped in existing canon. Policy measures shifting how scientific work is produced, disseminated, consumed, and rewarded may be called for to push fields into new, more fertile areas of study.
Perhaps turning researchers into content providers for large publishing corporations wasn't the best economic model to incentivise scientific progress after all.

Re: There is too much science for science to work properly

Posted: Fri Oct 08, 2021 7:42 pm
by Fishnut
That sounds like a really interesting paper.

I remember reading this Guardian Long Read a few years ago (I can't remember if it was discussed in previous iterations of this forum or not) and being blown away. I had no idea that Rupert f.cking Murdoch was responsible for the proliferation of journals, but it does make the insane model of researchers do to the work, researchers peer-review others' work for free, researchers pay to publish their work and researchers pay to access the work of others while the publishers make a fortune make much more sense.

Re: There is too much science for science to work properly

Posted: Sat Oct 09, 2021 2:56 pm
by jeremy
Fishnut wrote:
Fri Oct 08, 2021 7:42 pm
That sounds like a really interesting paper.

I remember reading this Guardian Long Read a few years ago (I can't remember if it was discussed in previous iterations of this forum or not) and being blown away. I had no idea that Rupert f.cking Murdoch was responsible for the proliferation of journals, but it does make the insane model of researchers do to the work, researchers peer-review others' work for free, researchers pay to publish their work and researchers pay to access the work of others while the publishers make a fortune make much more sense.
I think you mean Robert f.cking Maxwell.

Re: There is too much science for science to work properly

Posted: Sat Oct 09, 2021 9:46 pm
by Fishnut
jeremy wrote:
Sat Oct 09, 2021 2:56 pm
Fishnut wrote:
Fri Oct 08, 2021 7:42 pm
That sounds like a really interesting paper.

I remember reading this Guardian Long Read a few years ago (I can't remember if it was discussed in previous iterations of this forum or not) and being blown away. I had no idea that Rupert f.cking Murdoch was responsible for the proliferation of journals, but it does make the insane model of researchers do to the work, researchers peer-review others' work for free, researchers pay to publish their work and researchers pay to access the work of others while the publishers make a fortune make much more sense.
I think you mean Robert f.cking Maxwell.
You're right, I did. I even checked the article to make sure I had the right f.cker and still managed to mix them up.

Re: There is too much science for science to work properly

Posted: Sat Oct 09, 2021 10:22 pm
by Bird on a Fire
A couple of times in my PhD I've tried to stay on top of the literature by getting table of contents and alerts for all the journals that regularly publish my kind of stuff.

It was hours a week just to sift through titles, ending with a million tabs for abstracts I'd skim, and end up wanting to read about 10% of them, of which I could actually access less than half.

These days I just rely on tweets, emails and chatting to people I know. And I think everyone else does too.

Re: There is too much science for science to work properly

Posted: Sat Oct 09, 2021 10:35 pm
by sTeamTraen
One of my hobbies at conferences is suggesting to my fellow psychologists that we would all be better off with 90% fewer researchers all getting 10x as much grant money so we could run decent sample sizes. Individually they are surprisingly receptive to this idea, presumably because they all imagine that other people would make up the 90%.

Re: There is too much science for science to work properly

Posted: Sun Oct 10, 2021 12:27 am
by dyqik
I can't remember that last time I read a journal or astro-ph.

Re: There is too much science for science to work properly

Posted: Sun Oct 10, 2021 11:15 am
by Woodchopper
Bird on a Fire wrote:
Sat Oct 09, 2021 10:22 pm
A couple of times in my PhD I've tried to stay on top of the literature by getting table of contents and alerts for all the journals that regularly publish my kind of stuff.

It was hours a week just to sift through titles, ending with a million tabs for abstracts I'd skim, and end up wanting to read about 10% of them, of which I could actually access less than half.

These days I just rely on tweets, emails and chatting to people I know. And I think everyone else does too.
Once you get your PhD it'll get a lot easier. Just set up a Google Scholar alert for anything that cites one of your publications. Ignore the rest as irrelevant.

Re: There is too much science for science to work properly

Posted: Sun Oct 10, 2021 3:04 pm
by Stephanie
I remember reading a commentary about a psychological theory that included this. I think about it quite a lot.
As a doctoral student, I read all of psychology whether it be biological, social, developmental or cognitive. I read philosophy, sociology and psychiatry and I even read novels related in all sorts of ways to the human condition. Now my students drown in the literature. We have published so many studies, so many theories in so many journals and books that they cannot possibly grasp it all and an academic learned helplessness develops. So the solution is known as ‘being focused’ or ‘specialising’ and similarly academic careers are based on ‘having a reputation in a field’ or making a contribution ‘to an area’. Yet I have examined students whose subject is cancer who have never read the literature on heart disease, students who study smoking but have ‘focused only’ on the qualitative research and those who only cite research from the UK as this is where their study is. And likewise many researchers may know their ‘field’ and are known ‘for an area’ but remain blissfully unaware of all the other literature that could enrich their work. And then these areas become inbred, stunted and stale as there is no chance of cross-fertilisation as they know no other area to cross-fertilise with. The vastness of the literature therefore means that it remains unread causing a narrowing vision of what psychology is and our research becomes increasingly obvious.

Re: There is too much science for science to work properly

Posted: Wed Oct 13, 2021 6:28 am
by Woodchopper
Stephanie wrote:
Sun Oct 10, 2021 3:04 pm
I remember reading a commentary about a psychological theory that included this. I think about it quite a lot.
As a doctoral student, I read all of psychology whether it be biological, social, developmental or cognitive. I read philosophy, sociology and psychiatry and I even read novels related in all sorts of ways to the human condition. Now my students drown in the literature. We have published so many studies, so many theories in so many journals and books that they cannot possibly grasp it all and an academic learned helplessness develops. So the solution is known as ‘being focused’ or ‘specialising’ and similarly academic careers are based on ‘having a reputation in a field’ or making a contribution ‘to an area’. Yet I have examined students whose subject is cancer who have never read the literature on heart disease, students who study smoking but have ‘focused only’ on the qualitative research and those who only cite research from the UK as this is where their study is. And likewise many researchers may know their ‘field’ and are known ‘for an area’ but remain blissfully unaware of all the other literature that could enrich their work. And then these areas become inbred, stunted and stale as there is no chance of cross-fertilisation as they know no other area to cross-fertilise with. The vastness of the literature therefore means that it remains unread causing a narrowing vision of what psychology is and our research becomes increasingly obvious.
Well yes and no.

I agree that so much is published that would take a huge amount of time for someone to master a subject beyond the field in which they work, and those fields tend to be defined pretty narrowly.

But its not difficult to use someone else's mastery. Scientists regularly publish summaries of the literature in their field, and so systematic reviews, meta analysis etc have a vital role to play in research. As I advised a PhD student a couple of days ago, its not necessary to read a huge number of articles from another field, just go to Google Scholar and find someone else's literature review. Its also possible to get a basic understanding of the key findings in a field from reading textbooks or publications designed for a lay audience (I'm thinking of reports written by academics more than popular science bestsellers). Paywalls used to be a problem, but vast amounts of material are now available for free or for very little money.

Lots of methods are also sufficiently similar that people can appraise a paper and understand some of its basic strengths and weaknesses even if they are not experts in that field - for example similar statistical methods are used in fields as diverse as economics, medicine and ecology. So an expert on cancer should be able to read a paper on heart disease (though of course that doesn't apply elsewhere, a biologist might struggle to understand a physics paper).

Assuming that the students mentioned are doing graduate degrees I think that the problem is more a culture of intellectual Fordism in which students are treated as items on a production line. In that environment reading widely can be seen as a distraction from the purpose of getting their degree certificate and necessary number of publications on time.

Re: There is too much science for science to work properly

Posted: Wed Oct 13, 2021 3:51 pm
by dyqik
Stephanie wrote:
Sun Oct 10, 2021 3:04 pm
I remember reading a commentary about a psychological theory that included this. I think about it quite a lot.
As a doctoral student, I read all of psychology whether it be biological, social, developmental or cognitive. I read philosophy, sociology and psychiatry and I even read novels related in all sorts of ways to the human condition. Now my students drown in the literature. We have published so many studies, so many theories in so many journals and books that they cannot possibly grasp it all and an academic learned helplessness develops. So the solution is known as ‘being focused’ or ‘specialising’ and similarly academic careers are based on ‘having a reputation in a field’ or making a contribution ‘to an area’. Yet I have examined students whose subject is cancer who have never read the literature on heart disease, students who study smoking but have ‘focused only’ on the qualitative research and those who only cite research from the UK as this is where their study is. And likewise many researchers may know their ‘field’ and are known ‘for an area’ but remain blissfully unaware of all the other literature that could enrich their work. And then these areas become inbred, stunted and stale as there is no chance of cross-fertilisation as they know no other area to cross-fertilise with. The vastness of the literature therefore means that it remains unread causing a narrowing vision of what psychology is and our research becomes increasingly obvious.
This is why review articles and graduate level textbooks exist.

Re: There is too much science for science to work properly

Posted: Wed Oct 13, 2021 3:53 pm
by Woodchopper
dyqik wrote:
Wed Oct 13, 2021 3:51 pm
Stephanie wrote:
Sun Oct 10, 2021 3:04 pm
I remember reading a commentary about a psychological theory that included this. I think about it quite a lot.
As a doctoral student, I read all of psychology whether it be biological, social, developmental or cognitive. I read philosophy, sociology and psychiatry and I even read novels related in all sorts of ways to the human condition. Now my students drown in the literature. We have published so many studies, so many theories in so many journals and books that they cannot possibly grasp it all and an academic learned helplessness develops. So the solution is known as ‘being focused’ or ‘specialising’ and similarly academic careers are based on ‘having a reputation in a field’ or making a contribution ‘to an area’. Yet I have examined students whose subject is cancer who have never read the literature on heart disease, students who study smoking but have ‘focused only’ on the qualitative research and those who only cite research from the UK as this is where their study is. And likewise many researchers may know their ‘field’ and are known ‘for an area’ but remain blissfully unaware of all the other literature that could enrich their work. And then these areas become inbred, stunted and stale as there is no chance of cross-fertilisation as they know no other area to cross-fertilise with. The vastness of the literature therefore means that it remains unread causing a narrowing vision of what psychology is and our research becomes increasingly obvious.
This is why review articles and graduate level textbooks exist.
Yes, you just summed up my 300 word post in a sentence.

Re: There is too much science for science to work properly

Posted: Wed Oct 13, 2021 9:02 pm
by Bird on a Fire
dyqik wrote:
Wed Oct 13, 2021 3:51 pm
Stephanie wrote:
Sun Oct 10, 2021 3:04 pm
I remember reading a commentary about a psychological theory that included this. I think about it quite a lot.
As a doctoral student, I read all of psychology whether it be biological, social, developmental or cognitive. I read philosophy, sociology and psychiatry and I even read novels related in all sorts of ways to the human condition. Now my students drown in the literature. We have published so many studies, so many theories in so many journals and books that they cannot possibly grasp it all and an academic learned helplessness develops. So the solution is known as ‘being focused’ or ‘specialising’ and similarly academic careers are based on ‘having a reputation in a field’ or making a contribution ‘to an area’. Yet I have examined students whose subject is cancer who have never read the literature on heart disease, students who study smoking but have ‘focused only’ on the qualitative research and those who only cite research from the UK as this is where their study is. And likewise many researchers may know their ‘field’ and are known ‘for an area’ but remain blissfully unaware of all the other literature that could enrich their work. And then these areas become inbred, stunted and stale as there is no chance of cross-fertilisation as they know no other area to cross-fertilise with. The vastness of the literature therefore means that it remains unread causing a narrowing vision of what psychology is and our research becomes increasingly obvious.
This is why review articles and graduate level textbooks exist.
Well sort of, but do many students (or other researchers) actually tend to read reviews and textbooks from outside their immediate field?

Re: There is too much science for science to work properly

Posted: Thu Oct 14, 2021 12:12 am
by dyqik
No, but if you decided that you wanted to know something about something outside your field, it's where you would start.