Sports Science - is it really science?

Get your science fix here: research, quackery, activism and all the rest
Post Reply
MartinDurkin
Clardic Fug
Posts: 150
Joined: Mon Nov 11, 2019 5:00 pm

Sports Science - is it really science?

Post by MartinDurkin » Fri Nov 15, 2019 9:46 am

Just saw this on twitter (some bloke called Goldacre, seems to be interested in this kind of stuff)
Cold water poured on scientific studies based on 'statistical cult'
https://amp.smh.com.au/national/cold-wa ... sion=true

Is this MBI method as dodgy as SMH article says?

User avatar
FredM
Clardic Fug
Posts: 160
Joined: Mon Nov 11, 2019 4:36 pm
Location: Norwich
Contact:

Re: Sports Science - is it really science?

Post by FredM » Fri Nov 15, 2019 10:04 am

Detailed stats paper critique of MBI here. My italics.

Conclusions
“ We have given a precise description of “magnitude-based inference” for the problem of comparing two means and discussed its interpretation in detail. “Magnitude-based inference” begins with the computation of a confidence interval (that has the usual frequentist interpretation in terms of repeated samples). We show that the calculations can be interpreted either as P values for particular tests or as approximate Bayesian calculations, which lead to a type of test. In the former case, this means that 1) the “magnitude-based inference” calculations are not derived directly from the confidence interval but from P values for particular tests and 2) “magnitude-based inference” is less conservative than standard inference because it changes the null hypothesis and uses one-sided instead of two-sided P values. The inflated level of the test means that it should not be used. Finally, the sample size calculations should not be used. Rather than use “magnitude-based inference,” a better solution is to be realistic about the limitations of the data and use either confidence intervals or a fully Bayesian analysis.”

User avatar
spinybear
Fleury White
Posts: 9
Joined: Thu Nov 14, 2019 4:00 pm

Re: Sports Science - is it really science?

Post by spinybear » Fri Nov 15, 2019 10:30 am

FiveThirtyEight has a good readable explanation.

Small sample sizes have their place. I regularly work with small sample sizes for qualitative research, but there's all kinds of wrong with such small numbers for statistical inference. There are quant methods that can deal well with relatively low sample sizes (though all way higher than "just 11"!), but these would only be methodologically sound on really straightforward questions with clearly defined boundaries.

greyspoke
Fuzzable
Posts: 252
Joined: Mon Nov 11, 2019 4:36 pm

Re: Sports Science - is it really science?

Post by greyspoke » Fri Nov 15, 2019 5:31 pm

It isn't clear how widespread it is in sports science generally.

In competitive sport, there is a big difference between "may improve performance or may have no effect" and "may either improve or reduce performance". I'd take a long shot on the former, but would be wary of the latter.

Post Reply