https://medium.com/the-spike/seventy-te ... d96c23dbf4
The implications probably stretch beyond analysis of fMRI:
FWIW I recently teamed up with a friend and contributed an analysis to a similar project for ecology & evolutionary biology. I'm interested (if a bit anxious!) to see what the results are. On the one hand, analytical pipelines are generally less complex, though this is rapidly changing with the advent of high-dimensional data from e.g. biologgers (not to mention the -omics cans of worms). But on the other, field data is absolutely rife with confounding effects, many of which are difficult to measure, and it often seems to me a bit of a personal choice whether or not certain things get included in the analysis.he NARPS paper ends with the warning that “although the present investigation was limited to the analysis of a single fMRI dataset, it seems highly likely that similar variability will be present for other fields of research in which the data are high-dimensional and the analysis workflows are complex and varied”.
The Rest of Science: “Do they mean us?”
Yes, they mean you. These crises should give any of us working on data from complex pipelines pause for serious thought. There is nothing unique to fMRI about the issues they raise.
I think this kind of thing has interesting implications for open data and data publication - if 70 can analysts get 70 different results from the same dataset, the importance of enabling independent verification of results is clear.