So, you think you're smarter than Monty Hall?

Get your science fix here: research, quackery, activism and all the rest
Post Reply
User avatar
Sciolus
Dorkwood
Posts: 1313
Joined: Mon Nov 11, 2019 6:42 pm

So, you think you're smarter than Monty Hall?

Post by Sciolus » Fri Oct 08, 2021 8:34 pm

Want to bet $5000 on that?
New York Times, July 21, 1991 wrote:Sitting at the dining room table, Mr. Hall quickly conducted 10 rounds of the game as this contestant tried the non-switching strategy. The result was four cars and six goats. Then for the next 10 rounds the contestant tried switching doors, and there was a dramatic improvement: eight cars and two goats. A pattern was emerging.
"So her answer's right: you should switch," Mr. Hall said, reaching the same conclusion as the tens of thousands of students who conducted similar experiments at Ms. vos Savant's suggestion...

After the 20 trials at the dining room table, the problem also captured Mr. Hall's imagination. He picked up a copy of Ms. vos Savant's original column, read it carefully, saw a loophole and then suggested more trials.
On the first, the contestant picked Door 1.
"That's too bad," Mr. Hall said, opening Door 1. "You've won a goat." "But you didn't open another door yet or give me a chance to switch." "Where does it say I have to let you switch every time? I'm the master of the show. Here, try it again."
On the second trial, the contestant again picked Door 1. Mr. Hall opened Door 3, revealing a goat. The contestant was about to switch to Door 2 when Mr. Hall pulled out a roll of bills.
"You're sure you want Door No. 2?" he asked. "Before I show you what's behind that door, I will give you $3,000 in cash not to switch to it."
"I'll switch to it."
"Three thousand dollars," Mr. Hall repeated, shifting into his famous cadence. "Cash. Cash money. It could be a car, but it could be a goat. Four thousand."
"I'll try the door."
"Forty-five hundred. Forty-seven. Forty-eight. My last offer: Five thousand dollars."
"Let's open the door."
"You just ended up with a goat," he said, opening the door...

Was Mr. Hall cheating? Not according to the rules of the show, because he did have the option of not offering the switch, and he usually did not offer it.
And although Mr. Hall might have been violating the spirit of Ms. vos Savant's problem, he was not violating its letter. Dr. Diaconis and Mr. Gardner both noticed the same loophole when they compared Ms. vos Savant's wording of the problem with the versions they had analyzed in their articles.
"The problem is not well-formed," Mr. Gardner said, "unless it makes clear that the host must always open an empty door and offer the switch. Otherwise, if the host is malevolent, he may open another door only when it's to his advantage to let the player switch, and the probability of being right by switching could be as low as zero." Mr. Gardner said the ambiguity could be eliminated if the host promised ahead of time to open another door and then offer a switch.
(How have I not heard of that in the last 30 years?)

User avatar
Grumble
Light of Blast
Posts: 4746
Joined: Mon Nov 11, 2019 1:03 pm

Re: So, you think you're smarter than Monty Hall?

Post by Grumble » Fri Oct 08, 2021 11:05 pm

Sciolus wrote:
Fri Oct 08, 2021 8:34 pm
Want to bet $5000 on that?
New York Times, July 21, 1991 wrote:Sitting at the dining room table, Mr. Hall quickly conducted 10 rounds of the game as this contestant tried the non-switching strategy. The result was four cars and six goats. Then for the next 10 rounds the contestant tried switching doors, and there was a dramatic improvement: eight cars and two goats. A pattern was emerging.
"So her answer's right: you should switch," Mr. Hall said, reaching the same conclusion as the tens of thousands of students who conducted similar experiments at Ms. vos Savant's suggestion...

After the 20 trials at the dining room table, the problem also captured Mr. Hall's imagination. He picked up a copy of Ms. vos Savant's original column, read it carefully, saw a loophole and then suggested more trials.
On the first, the contestant picked Door 1.
"That's too bad," Mr. Hall said, opening Door 1. "You've won a goat." "But you didn't open another door yet or give me a chance to switch." "Where does it say I have to let you switch every time? I'm the master of the show. Here, try it again."
On the second trial, the contestant again picked Door 1. Mr. Hall opened Door 3, revealing a goat. The contestant was about to switch to Door 2 when Mr. Hall pulled out a roll of bills.
"You're sure you want Door No. 2?" he asked. "Before I show you what's behind that door, I will give you $3,000 in cash not to switch to it."
"I'll switch to it."
"Three thousand dollars," Mr. Hall repeated, shifting into his famous cadence. "Cash. Cash money. It could be a car, but it could be a goat. Four thousand."
"I'll try the door."
"Forty-five hundred. Forty-seven. Forty-eight. My last offer: Five thousand dollars."
"Let's open the door."
"You just ended up with a goat," he said, opening the door...

Was Mr. Hall cheating? Not according to the rules of the show, because he did have the option of not offering the switch, and he usually did not offer it.
And although Mr. Hall might have been violating the spirit of Ms. vos Savant's problem, he was not violating its letter. Dr. Diaconis and Mr. Gardner both noticed the same loophole when they compared Ms. vos Savant's wording of the problem with the versions they had analyzed in their articles.
"The problem is not well-formed," Mr. Gardner said, "unless it makes clear that the host must always open an empty door and offer the switch. Otherwise, if the host is malevolent, he may open another door only when it's to his advantage to let the player switch, and the probability of being right by switching could be as low as zero." Mr. Gardner said the ambiguity could be eliminated if the host promised ahead of time to open another door and then offer a switch.
(How have I not heard of that in the last 30 years?)
I have heard that there wasn’t always a switch offered when I read about it, maybe I read a more nuanced telling of it.

In a similar vein: there’s a famous social science phenomenon called the Hawthorne Effect. In 1930’s Chicago there was an experiment run on factory workers to see what would improve productivity. They tried improving lighting, reducing lighting, all sorts of things. Every change they made improved productivity. The conclusion was that the workers like the attention and the change in environment and that was what improved productivity. However a few years ago someone reanalysed the original data and found that the effect all but disappeared once you corrected for confounding factors - one of the biggest was days of the week. If the change was implemented on a Monday that day was more productive than the preceding Friday! The change was always implemented on a Monday. Hawthorne did not demonstrate the Hawthorne effect (although a more subtle version of it is better demonstrated and well established).

https://www.nber.org/system/files/worki ... w15016.pdf
where once I used to scintillate
now I sin till ten past three

Allo V Psycho
Catbabel
Posts: 734
Joined: Sat Nov 16, 2019 8:18 am

Re: So, you think you're smarter than Monty Hall?

Post by Allo V Psycho » Sat Oct 09, 2021 7:33 am

Sciolus wrote:
Fri Oct 08, 2021 8:34 pm
Want to bet $5000 on that?

(Monty Hall problem)

(How have I not heard of that in the last 30 years?)
I think the key quote is
But I do know that my first reaction has been wrong time after time on similar problems. Our brains are just not wired to do probability
problems very well, so I'm not surprised there were mistakes."
I heard about quite a long time ago (at least 20 years, I think).
Despite 'understanding' the statistical explanation, I still had to write a computer programme to 'show' myself it was true.

User avatar
shpalman
Princess POW
Posts: 8241
Joined: Mon Nov 11, 2019 12:53 pm
Location: One step beyond
Contact:

Re: So, you think you're smarter than Monty Hall?

Post by shpalman » Sat Oct 09, 2021 8:08 am

Allo V Psycho wrote:
Sat Oct 09, 2021 7:33 am
Sciolus wrote:
Fri Oct 08, 2021 8:34 pm
Want to bet $5000 on that?

(Monty Hall problem)

(How have I not heard of that in the last 30 years?)
I think the key quote is
But I do know that my first reaction has been wrong time after time on similar problems. Our brains are just not wired to do probability
problems very well, so I'm not surprised there were mistakes."
I heard about quite a long time ago (at least 20 years, I think).
Despite 'understanding' the statistical explanation, I still had to write a computer programme to 'show' myself it was true.
Especially as these problems are often deliberately designed to break our intuition in some way.
having that swing is a necessary but not sufficient condition for it meaning a thing
@shpalman@mastodon.me.uk

Allo V Psycho
Catbabel
Posts: 734
Joined: Sat Nov 16, 2019 8:18 am

Re: So, you think you're smarter than Monty Hall?

Post by Allo V Psycho » Sat Oct 09, 2021 9:50 am

shpalman wrote:
Sat Oct 09, 2021 8:08 am
Allo V Psycho wrote:
Sat Oct 09, 2021 7:33 am
Sciolus wrote:
Fri Oct 08, 2021 8:34 pm
Want to bet $5000 on that?

(Monty Hall problem)

(How have I not heard of that in the last 30 years?)
I think the key quote is
But I do know that my first reaction has been wrong time after time on similar problems. Our brains are just not wired to do probability
problems very well, so I'm not surprised there were mistakes."
I heard about quite a long time ago (at least 20 years, I think).
Despite 'understanding' the statistical explanation, I still had to write a computer programme to 'show' myself it was true.
Especially as these problems are often deliberately designed to break our intuition in some way.
I used to lecture on assessing evidence, and touched on human biases, e.g. from 'thinking fast and slow'

"OK, class, How many animals of each kind did Moses lead into the Ark?"

While 'two' was the common answer, a rather more biblical student once answered '7 for some animals', but of course was still wrong.

And then I would explain that knowing about these biases was no great help in avoiding them.

"So, class, quick: what do cows drink?"

And then we did Simpson's paradox (still not even got to probability yet)

At a small U.S. college with only two faculties, for the larger Science Faculty, 825 applicants were white and 108 were non-white. Of the whites, 512 were admitted, (62%), whilst 89 (82%) non-white applicants were admitted. In the Arts Faculty, 272 whites and 241 non-whites applied. 65 whites were admitted (24%) and 83 non-whites(34%).

Plainly, this suggests favouritism towards non-whites, and should be stamped out! MAGA!
It was a student exercise to work out the ratio of successful white and non-white applicants to the College as a whole.

User avatar
shpalman
Princess POW
Posts: 8241
Joined: Mon Nov 11, 2019 12:53 pm
Location: One step beyond
Contact:

Re: So, you think you're smarter than Monty Hall?

Post by shpalman » Sat Oct 09, 2021 10:03 am

Allo V Psycho wrote:
Sat Oct 09, 2021 9:50 am
shpalman wrote:
Sat Oct 09, 2021 8:08 am
Allo V Psycho wrote:
Sat Oct 09, 2021 7:33 am


I think the key quote is


I heard about quite a long time ago (at least 20 years, I think).
Despite 'understanding' the statistical explanation, I still had to write a computer programme to 'show' myself it was true.
Especially as these problems are often deliberately designed to break our intuition in some way.
I used to lecture on assessing evidence, and touched on human biases, e.g. from 'thinking fast and slow'

"OK, class, How many animals of each kind did Moses lead into the Ark?"

While 'two' was the common answer, a rather more biblical student once answered '7 for some animals', but of course was still wrong.
The correct answer is of course none because that never happened.

Spoiler:


"So a man tells me he has at least one son but I can see in the garden that there are three doors and a goat what is the probability that it's the same Tuesday as last week on his birthday?"
Allo V Psycho wrote:
Sat Oct 09, 2021 9:50 am
And then I would explain that knowing about these biases was no great help in avoiding them.

"So, class, quick: what do cows drink?"

And then we did Simpson's paradox (still not even got to probability yet)

At a small U.S. college with only two faculties, for the larger Science Faculty, 825 applicants were white and 108 were non-white. Of the whites, 512 were admitted, (62%), whilst 89 (82%) non-white applicants were admitted. In the Arts Faculty, 272 whites and 241 non-whites applied. 65 whites were admitted (24%) and 83 non-whites(34%).

Plainly, this suggests favouritism towards non-whites, and should be stamped out! MAGA!
It was a student exercise to work out the ratio of successful white and non-white applicants to the College as a whole.
Well you'd need to know the proportions of white and non-white in the population I suppose. A bit like you need to know the prevalence of a disease in the population in order to evaluate the chance of false negatives and false positives; it's not enough to know the test's specificity and selectivity. As is discussed here.
having that swing is a necessary but not sufficient condition for it meaning a thing
@shpalman@mastodon.me.uk

Allo V Psycho
Catbabel
Posts: 734
Joined: Sat Nov 16, 2019 8:18 am

Re: So, you think you're smarter than Monty Hall?

Post by Allo V Psycho » Sat Oct 09, 2021 10:27 am

shpalman wrote:
Sat Oct 09, 2021 10:03 am
Allo V Psycho wrote:
Sat Oct 09, 2021 9:50 am
shpalman wrote:
Sat Oct 09, 2021 8:08 am

Especially as these problems are often deliberately designed to break our intuition in some way.
I used to lecture on assessing evidence, and touched on human biases, e.g. from 'thinking fast and slow'

"OK, class, How many animals of each kind did Moses lead into the Ark?"

While 'two' was the common answer, a rather more biblical student once answered '7 for some animals', but of course was still wrong.
The correct answer is of course none because that never happened.

Spoiler:
:D I do love that xkcd, but I don't think it is quite as bad as that example. I think it is more an example of Type 1 thinking in Khaneman's terms: that we rapidly fill in information rather than checking the information actually present. The response tended to be amusement, rather than slashing my arm off... (I do acknowledge there may have been a power imbalance...)

"So a man tells me he has at least one son but I can see in the garden that there are three doors and a goat what is the probability that it's the same Tuesday as last week on his birthday?"
Allo V Psycho wrote:
Sat Oct 09, 2021 9:50 am
And then I would explain that knowing about these biases was no great help in avoiding them.

"So, class, quick: what do cows drink?"

And then we did Simpson's paradox (still not even got to probability yet)

At a small U.S. college with only two faculties, for the larger Science Faculty, 825 applicants were white and 108 were non-white. Of the whites, 512 were admitted, (62%), whilst 89 (82%) non-white applicants were admitted. In the Arts Faculty, 272 whites and 241 non-whites applied. 65 whites were admitted (24%) and 83 non-whites(34%).

Plainly, this suggests favouritism towards non-whites, and should be stamped out! MAGA!
It was a student exercise to work out the ratio of successful white and non-white applicants to the College as a whole.
Well you'd need to know the proportions of white and non-white in the population I suppose. A bit like you need to know the prevalence of a disease in the population in order to evaluate the chance of false negatives and false positives; it's not enough to know the test's specificity and selectivity. As is discussed here.
Certainly from the information provided, no conclusions can be drawn about whether or not discrimination in either direction actually occurred. But the point was that different ways of calculating (and perhaps presenting) the same information can convey different impressions.

I'm glad I didn't include my probability example - we'd probably be arguing for hours....

User avatar
shpalman
Princess POW
Posts: 8241
Joined: Mon Nov 11, 2019 12:53 pm
Location: One step beyond
Contact:

Re: So, you think you're smarter than Monty Hall?

Post by shpalman » Sat Oct 09, 2021 10:34 am

Allo V Psycho wrote:
Sat Oct 09, 2021 10:27 am
shpalman wrote:
Sat Oct 09, 2021 10:03 am
Allo V Psycho wrote:
Sat Oct 09, 2021 9:50 am


I used to lecture on assessing evidence, and touched on human biases, e.g. from 'thinking fast and slow'

"OK, class, How many animals of each kind did Moses lead into the Ark?"

While 'two' was the common answer, a rather more biblical student once answered '7 for some animals', but of course was still wrong.
The correct answer is of course none because that never happened.

Spoiler:
:D I do love that xkcd, but I don't think it is quite as bad as that example. I think it is more an example of Type 1 thinking in Khaneman's terms: that we rapidly fill in information rather than checking the information actually present. The response tended to be amusement, rather than slashing my arm off... (I do acknowledge there may have been a power imbalance...)
It's a bit like that "if your peacock lays in egg in your neighbour's garden" one. But then there are times when you need to bring in your external background information into the question at hand, and other times when you absolutely should not do this. That whole family of "person gives incomplete information" probability questions breaks down if we start arguing about what the person is trying to hide and why.
having that swing is a necessary but not sufficient condition for it meaning a thing
@shpalman@mastodon.me.uk

User avatar
sTeamTraen
After Pie
Posts: 2553
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: So, you think you're smarter than Monty Hall?

Post by sTeamTraen » Sat Oct 09, 2021 10:43 pm

shpalman wrote:
Sat Oct 09, 2021 10:03 am
Allo V Psycho wrote:
Sat Oct 09, 2021 9:50 am
And then we did Simpson's paradox (still not even got to probability yet)

At a small U.S. college with only two faculties, for the larger Science Faculty, 825 applicants were white and 108 were non-white. Of the whites, 512 were admitted, (62%), whilst 89 (82%) non-white applicants were admitted. In the Arts Faculty, 272 whites and 241 non-whites applied. 65 whites were admitted (24%) and 83 non-whites(34%).

Plainly, this suggests favouritism towards non-whites, and should be stamped out! MAGA!
It was a student exercise to work out the ratio of successful white and non-white applicants to the College as a whole.
Well you'd need to know the proportions of white and non-white in the population I suppose. A bit like you need to know the prevalence of a disease in the population in order to evaluate the chance of false negatives and false positives; it's not enough to know the test's specificity and selectivity. As is discussed here.
No, you don't need to know the proportion of white/non-white in the population, because the population of interest here is the applicants.

There is, of course, an entire field of study as to why there might (indeed, almost certainly are) more white applicants as a percentage of white people in the population versus non-white, but that is not relevant to the question of establishing, by looking at percentages admitted, whether the admissions process discriminates among the applicants that are presented to it.

Example 3 (severe/less-severe disease) here may help to understand this better.
Something something hammer something something nail

User avatar
Grumble
Light of Blast
Posts: 4746
Joined: Mon Nov 11, 2019 1:03 pm

Re: So, you think you're smarter than Monty Hall?

Post by Grumble » Sun Oct 10, 2021 6:48 am

So, class, quick: what do cows drink?
Are you wanting the answer milk and water?
where once I used to scintillate
now I sin till ten past three

User avatar
shpalman
Princess POW
Posts: 8241
Joined: Mon Nov 11, 2019 12:53 pm
Location: One step beyond
Contact:

Re: So, you think you're smarter than Monty Hall?

Post by shpalman » Sun Oct 10, 2021 7:01 am

sTeamTraen wrote:
Sat Oct 09, 2021 10:43 pm
shpalman wrote:
Sat Oct 09, 2021 10:03 am
Allo V Psycho wrote:
Sat Oct 09, 2021 9:50 am
And then we did Simpson's paradox (still not even got to probability yet)

At a small U.S. college with only two faculties, for the larger Science Faculty, 825 applicants were white and 108 were non-white. Of the whites, 512 were admitted, (62%), whilst 89 (82%) non-white applicants were admitted. In the Arts Faculty, 272 whites and 241 non-whites applied. 65 whites were admitted (24%) and 83 non-whites(34%).

Plainly, this suggests favouritism towards non-whites, and should be stamped out! MAGA!
It was a student exercise to work out the ratio of successful white and non-white applicants to the College as a whole.
Well you'd need to know the proportions of white and non-white in the population I suppose. A bit like you need to know the prevalence of a disease in the population in order to evaluate the chance of false negatives and false positives; it's not enough to know the test's specificity and selectivity. As is discussed here.
No, you don't need to know the proportion of white/non-white in the population, because the population of interest here is the applicants.

There is, of course, an entire field of study as to why there might (indeed, almost certainly are) more white applicants as a percentage of white people in the population versus non-white, but that is not relevant to the question of establishing, by looking at percentages admitted, whether the admissions process discriminates among the applicants that are presented to it.

Example 3 (severe/less-severe disease) here may help to understand this better.
In that example, it seems like the Better hospital has a lower overall Success rate than the Normal one but it's because more More severe cases (which is harder to Success) go to the Better hospital while the Normal hospital has more Less severe cases. The "correct" thing to do is to compare More severe and Less severe success rates separately.

At the college there are a total of 825+272=1097 white applicants of which 512+65=577 are admitted (52.6%) while there are 108+241=349 non-white applicants of which 172 are admitted (49.2%). The "correct" thing to do now is apparently to not consider admission rates separately.
having that swing is a necessary but not sufficient condition for it meaning a thing
@shpalman@mastodon.me.uk

User avatar
sTeamTraen
After Pie
Posts: 2553
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: So, you think you're smarter than Monty Hall?

Post by sTeamTraen » Sun Oct 10, 2021 9:20 am

shpalman wrote:
Sun Oct 10, 2021 7:01 am
In that example, it seems like the Better hospital has a lower overall Success rate than the Normal one but it's because more More severe cases (which is harder to Success) go to the Better hospital while the Normal hospital has more Less severe cases. The "correct" thing to do is to compare More severe and Less severe success rates separately.

At the college there are a total of 825+272=1097 white applicants of which 512+65=577 are admitted (52.6%) while there are 108+241=349 non-white applicants of which 172 are admitted (49.2%). The "correct" thing to do now is apparently to not consider admission rates separately.
Simpson's Paradox is generally about something like that - very often there will be quite a difference in the sizes of the groups, which means that comparing percentages is tricky.

In a moderately recent example, PNAS failed to spot SP in a claim that women were being discriminated against for funding in Dutch science, when in fact they were applying disproportionately for areas that had less funding. Their response to the rebuttal (which, full disclosure, was written by my co-supervisor) was that this meant that the funding organisation should put more money into areas that women want to research. That's an entire honourable proposition, but again goes beyond the statistical question.
Something something hammer something something nail

User avatar
Martin Y
Stummy Beige
Posts: 3080
Joined: Mon Nov 11, 2019 1:08 pm

Re: So, you think you're smarter than Monty Hall?

Post by Martin Y » Sun Oct 10, 2021 11:51 am

sTeamTraen wrote:
Sun Oct 10, 2021 9:20 am
shpalman wrote:
Sun Oct 10, 2021 7:01 am
In that example, it seems like the Better hospital has a lower overall Success rate than the Normal one but it's because more More severe cases (which is harder to Success) go to the Better hospital while the Normal hospital has more Less severe cases. The "correct" thing to do is to compare More severe and Less severe success rates separately.

At the college there are a total of 825+272=1097 white applicants of which 512+65=577 are admitted (52.6%) while there are 108+241=349 non-white applicants of which 172 are admitted (49.2%). The "correct" thing to do now is apparently to not consider admission rates separately.
Simpson's Paradox is generally about something like that - very often there will be quite a difference in the sizes of the groups, which means that comparing percentages is tricky.

In a moderately recent example, PNAS failed to spot SP in a claim that women were being discriminated against for funding in Dutch science, when in fact they were applying disproportionately for areas that had less funding. Their response to the rebuttal (which, full disclosure, was written by my co-supervisor) was that this meant that the funding organisation should put more money into areas that women want to research. That's an entire honourable proposition, but again goes beyond the statistical question.
This sounds rather like what Mrs Y is currently banging her head against regarding ratios of female engineering students, when the data easily available accumulates students into equivalent numbers of full-timers (can't remember the acronyms they use for that) but that will include those who take maybe a tenth of their course in an engineering subject and are certainly not going to become engineers on graduation. Plus the ones who get an engineering qualification purely as a way to get a job in the City, plus the ones who are going home to China etc.

User avatar
dyqik
Princess POW
Posts: 7526
Joined: Wed Sep 25, 2019 4:19 pm
Location: Masshole
Contact:

Re: So, you think you're smarter than Monty Hall?

Post by dyqik » Sun Oct 10, 2021 5:00 pm

shpalman wrote:
Sun Oct 10, 2021 7:01 am
sTeamTraen wrote:
Sat Oct 09, 2021 10:43 pm

No, you don't need to know the proportion of white/non-white in the population, because the population of interest here is the applicants.

There is, of course, an entire field of study as to why there might (indeed, almost certainly are) more white applicants as a percentage of white people in the population versus non-white, but that is not relevant to the question of establishing, by looking at percentages admitted, whether the admissions process discriminates among the applicants that are presented to it.

Example 3 (severe/less-severe disease) here may help to understand this better.
In that example, it seems like the Better hospital has a lower overall Success rate than the Normal one but it's because more More severe cases (which is harder to Success) go to the Better hospital while the Normal hospital has more Less severe cases. The "correct" thing to do is to compare More severe and Less severe success rates separately.

At the college there are a total of 825+272=1097 white applicants of which 512+65=577 are admitted (52.6%) while there are 108+241=349 non-white applicants of which 172 are admitted (49.2%). The "correct" thing to do now is apparently to not consider admission rates separately.
It always depends on which hypothesis you are testing, and the context of that hypothesis.

In this kind of case, before deciding whether or not to combine the data, you have to ask if there is meaningful difference in the pre-application process for the two faculties and wider society that affects who applies for which course, whether it is appropriate in testing your specific hypothesis to combine the courses or not, and whether there are additional more or less specific hypotheses that need to be considered alongside that one when interpreting the results.

Separating the faculties and their individual courses with more and more specific hypotheses runs the risk of green jelly bean errors, lumping together runs the risk of hiding important differences between application processes. And when interpreting the results, you always need to consider the context.

IvanV
Stummy Beige
Posts: 2660
Joined: Mon May 17, 2021 11:12 am

Re: So, you think you're smarter than Monty Hall?

Post by IvanV » Mon Oct 11, 2021 3:13 pm

Sciolus wrote:
Fri Oct 08, 2021 8:34 pm
New York Times, July 21, 1991 wrote:... "The problem is not well-formed," Mr. Gardner said, "unless it makes clear that the host must always open an empty door and offer the switch. Otherwise, if the host is malevolent, he may open another door only when it's to his advantage to let the player switch, and the probability of being right by switching could be as low as zero." Mr. Gardner said the ambiguity could be eliminated if the host promised ahead of time to open another door and then offer a switch.
Nice to read a correct analysis of the real "Monty Hall Problem", ie as it actually was on the show, for once.

As Martin Gardner correctly notes, Monty Hall was not bound by specific rules. The classic analyses tend to analyse a different game in which he was so bound - what we can call "the deterministic Monty Hall problem". When the game show started, initially he did always offer a switch. But when it became well-known that there was a systematic advantage in switching, provided he always offered the switch,* then he stopped offering it routinely. Moreover he succeeded in being unpredictable in his behaviour - you couldn't even say what was his strategy, and you couldn't even say it was random. And he would suddenly bring in new unpredictable behaviours such as suddenly one day offering them money not to switch. He was a very clever host. By such unpredictable behaviour, he made the game more entertaining for the viewers, and more maddening for the participants. And the best the probability theorists could say about it was that "the problem is not well-formed" - it doesn't have an answer. Such situations are called "radical uncertainty", or "unknown unknowns", and Monty Hall delightfully illustrated what it can appear like in practice.

*Although surprisingly many clever mathematicians disputed that, even when they were analysing this deterministic version of the game. Typically they were trying to insist, implicitly or explicity, that the assumptions were different.

User avatar
sTeamTraen
After Pie
Posts: 2553
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: So, you think you're smarter than Monty Hall?

Post by sTeamTraen » Mon Oct 11, 2021 4:39 pm

dyqik wrote:
Sun Oct 10, 2021 5:00 pm
shpalman wrote:
Sun Oct 10, 2021 7:01 am
sTeamTraen wrote:
Sat Oct 09, 2021 10:43 pm

No, you don't need to know the proportion of white/non-white in the population, because the population of interest here is the applicants.

There is, of course, an entire field of study as to why there might (indeed, almost certainly are) more white applicants as a percentage of white people in the population versus non-white, but that is not relevant to the question of establishing, by looking at percentages admitted, whether the admissions process discriminates among the applicants that are presented to it.

Example 3 (severe/less-severe disease) here may help to understand this better.
In that example, it seems like the Better hospital has a lower overall Success rate than the Normal one but it's because more More severe cases (which is harder to Success) go to the Better hospital while the Normal hospital has more Less severe cases. The "correct" thing to do is to compare More severe and Less severe success rates separately.

At the college there are a total of 825+272=1097 white applicants of which 512+65=577 are admitted (52.6%) while there are 108+241=349 non-white applicants of which 172 are admitted (49.2%). The "correct" thing to do now is apparently to not consider admission rates separately.
It always depends on which hypothesis you are testing, and the context of that hypothesis.

In this kind of case, before deciding whether or not to combine the data, you have to ask if there is meaningful difference in the pre-application process for the two faculties and wider society that affects who applies for which course, whether it is appropriate in testing your specific hypothesis to combine the courses or not, and whether there are additional more or less specific hypotheses that need to be considered alongside that one when interpreting the results.

Separating the faculties and their individual courses with more and more specific hypotheses runs the risk of green jelly bean errors, lumping together runs the risk of hiding important differences between application processes. And when interpreting the results, you always need to consider the context.
It turns out that Simpson's Paradox is a special case example of the sign reversal phenomenon known (in the regression world) as suppression. I happen to know that because I co-authored what may be the first paper to get the relation between Simpson's Paradox, Lord's Paradox, and suppression correct. (My late colleague Carol Nickerson did most of the heavy lifting on that paper.)
Something something hammer something something nail

User avatar
basementer
Dorkwood
Posts: 1504
Joined: Mon Nov 11, 2019 1:03 pm
Location: 8024, Aotearoa
Contact:

Re: So, you think you're smarter than Monty Hall?

Post by basementer » Mon Oct 11, 2021 6:46 pm

IvanV wrote:
Mon Oct 11, 2021 3:13 pm
Nice to read a correct analysis of the real "Monty Hall Problem", ie as it actually was on the show, for once.

As Martin Gardner correctly notes, Monty Hall was not bound by specific rules. The classic analyses tend to analyse a different game in which he was so bound - what we can call "the deterministic Monty Hall problem". When the game show started, initially he did always offer a switch. But when it became well-known that there was a systematic advantage in switching, provided he always offered the switch,* then he stopped offering it routinely.
That's not a necessary condition. If you are shown that one of the doors has a goat, the advantage in switching is still there if die rolls were used to pick whether to open a door at all and then which one to open.
Money is just a substitute for luck anyway. - Tom Siddell

IvanV
Stummy Beige
Posts: 2660
Joined: Mon May 17, 2021 11:12 am

Re: So, you think you're smarter than Monty Hall?

Post by IvanV » Mon Oct 11, 2021 9:43 pm

basementer wrote:
Mon Oct 11, 2021 6:46 pm
IvanV wrote:
Mon Oct 11, 2021 3:13 pm
Nice to read a correct analysis of the real "Monty Hall Problem", ie as it actually was on the show, for once.

As Martin Gardner correctly notes, Monty Hall was not bound by specific rules. The classic analyses tend to analyse a different game in which he was so bound - what we can call "the deterministic Monty Hall problem". When the game show started, initially he did always offer a switch. But when it became well-known that there was a systematic advantage in switching, provided he always offered the switch,* then he stopped offering it routinely.
That's not a necessary condition. If you are shown that one of the doors has a goat, the advantage in switching is still there if die rolls were used to pick whether to open a door at all and then which one to open.
Indeed. You can calculate an optimal strategy if there is any predictability at all in what is happening, however constructed. It can be an explicit set of rules, or consistent behaviours that you deduce by observation.* Monty Hall's "skill" - in the sense of maintaining audience interest and defeating any attempt to work out a successful strategy - was in (1) not having any explicit process or consistent behaviours (2) behaving in a non-stationary way that defeated any attempt at deducing anything from what he did.

*And it would have been possible for Monty Hall to have devised and followed a strategy that made switch/non-switch a 50/50 bet. But that would have been more boring.

User avatar
jaap
Catbabel
Posts: 679
Joined: Mon Nov 11, 2019 2:05 pm
Location: Netherlands
Contact:

Re: So, you think you're smarter than Monty Hall?

Post by jaap » Mon Oct 11, 2021 9:51 pm

basementer wrote:
Mon Oct 11, 2021 6:46 pm
IvanV wrote:
Mon Oct 11, 2021 3:13 pm
Nice to read a correct analysis of the real "Monty Hall Problem", ie as it actually was on the show, for once.

As Martin Gardner correctly notes, Monty Hall was not bound by specific rules. The classic analyses tend to analyse a different game in which he was so bound - what we can call "the deterministic Monty Hall problem". When the game show started, initially he did always offer a switch. But when it became well-known that there was a systematic advantage in switching, provided he always offered the switch,* then he stopped offering it routinely.
That's not a necessary condition. If you are shown that one of the doors has a goat, the advantage in switching is still there if die rolls were used to pick whether to open a door at all and then which one to open.
If Monty only offered a switch if your chosen door has a car, then you will always lose by switching when offered.
If Monty only offered a switch if your chosen door has a goat, then you will always win by switching when offered.
Monty can completely control whether switching is advantageous or not, so if there is no consistent history to his method of offering switches, you cannot know if you should switch.

Another thing that is often not explained well is that Monty knows which door has the car behind it, and that when he opens one of the doors that you didn't choose, he will never open the door with the car. If he did not know that and opened a door at random, then every time he revealed a goat you would really be in a 50/50 situation.

User avatar
Sciolus
Dorkwood
Posts: 1313
Joined: Mon Nov 11, 2019 6:42 pm

Re: So, you think you're smarter than Monty Hall?

Post by Sciolus » Tue Oct 12, 2021 10:00 am

IvanV wrote:
Mon Oct 11, 2021 9:43 pm
Indeed. You can calculate an optimal strategy if there is any predictability at all in what is happening, however constructed. It can be an explicit set of rules, or consistent behaviours that you deduce by observation.* Monty Hall's "skill" - in the sense of maintaining audience interest and defeating any attempt to work out a successful strategy - was in (1) not having any explicit process or consistent behaviours (2) behaving in a non-stationary way that defeated any attempt at deducing anything from what he did.

*And it would have been possible for Monty Hall to have devised and followed a strategy that made switch/non-switch a 50/50 bet. But that would have been more boring.
I've never actually seen the programme, but it ran for thousands of episodes over decades, so you have to keep the interest up. One way to do that is the classic gambling-addiction techniques, encouraging the punters to think they've got a system, offering rewards just often enough, and occasionally pulling the rug from under them. Hall and his team were clearly pretty good at it.

Post Reply