Fishnut wrote: ↑Sun Dec 08, 2019 10:18 pm
The idea that the numerical value doesn't have significance is b.llsh.t. Of course it does. But no figure is gospel, that's just how science works. I work in marine ecology and my data are messy as f.ck. They are still meaningful but if someone was to go and do the same study using different fish they'll get slightly different data.
This is the nub of it, put far more elegantly than I could have put it.
Whenever we get data from social research we have the same discussion about the data - 'illustrative or indicative?' This is essentially the distinction between saying '62% of respondents say that Lancashire County Cricket Club are the world's finest sporting outfit' and 'A lot of respondents say that Lancashire County Cricket Club are the world's finest sporting outfit'.
If your aim is to work out what proportion of the sample venerate Lancs CCC and so are prone to misery and despair between April and September of a given summer, then you want the data to be indicative and have a proportion, but often if you want to explore an issue, merely being able to say with a degree of authority 'a lot of people are sadly infatuated with Lancs' can be very valuable. The difficulty of social research is that even if you get a census sample that meets the strict mathematical criteria required to evade the need to weight (very difficult to do indeed) the next time you take the survey,
the results will be different and so in a way an excessive focus on the exact figure is not always helpful.
This is a change in mindset that, in my experience, many scientists can struggle with (and many others thrive with to be fair). It does not mean a lack of rigour or seriousness in the field. It means you are no longer dealing with a discipline where there is a 'correct' answer that, if you work hard enough and design the experiments right, you can eventually solve. There is never an immutably correct answer in social research that you can measure and remain confident that it will stay measured. There isn't an equation that will describe it.
Your language and approach has to be different. It is, to be honest, a fascinating challenge that I think will appeal to a lot of scientists and we could do with more good quants in the field to be honest.