Dishonest research about dishonesty

Get your science fix here: research, quackery, activism and all the rest
User avatar
sTeamTraen
After Pie
Posts: 2551
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: Dishonest research about dishonesty

Post by sTeamTraen » Tue Aug 24, 2021 9:36 am

Bird on a Fire wrote:
Sun Aug 22, 2021 10:56 am
sTeamTraen wrote:
Sun Aug 22, 2021 10:50 am
Taleb has blocked me on Twitter
Lolz, is there a good story there?
I literally tweeted this https://twitter.com/sTeamTraen/status/1 ... 88034?s=19 and he obliged within the hour.
Something something hammer something something nail

User avatar
Bird on a Fire
Princess POW
Posts: 10137
Joined: Fri Oct 11, 2019 5:05 pm
Location: Portugal

Re: Dishonest research about dishonesty

Post by Bird on a Fire » Tue Aug 24, 2021 11:35 am

sTeamTraen wrote:
Tue Aug 24, 2021 9:36 am
Bird on a Fire wrote:
Sun Aug 22, 2021 10:56 am
sTeamTraen wrote:
Sun Aug 22, 2021 10:50 am
Taleb has blocked me on Twitter
Lolz, is there a good story there?
I literally tweeted this https://twitter.com/sTeamTraen/status/1 ... 88034?s=19 and he obliged within the hour.
Hahaha oh wow

Surprisingly thin skinned for someone who likes to use his own platform to berate others.
We have the right to a clean, healthy, sustainable environment.

Millennie Al
After Pie
Posts: 1621
Joined: Mon Mar 16, 2020 4:02 am

Re: Dishonest research about dishonesty

Post by Millennie Al » Wed Aug 25, 2021 12:37 am

sTeamTraen wrote:
Mon Aug 23, 2021 8:38 pm
Millennie Al wrote:
Mon Aug 23, 2021 1:09 am
Why the complicated keyboard entry scheme? Just use a mouse!
The interface was apparently designed to be used by the participant's non-dominant hand, for some reason. :lol:
I noticed that in the paper, but surely nearly everyone can use a mouse with either hand?

But that made me look again and realise that the diagram of the keys uses a numeric keypad, which is strongly biased towards right-hand use. Furthermore, the computers were laptops, which don't have numeric keypads (some re-use the main keyboard, but then the layout isn't as shown). Maybe the participants were supplied with separate numeric keypads. The design is unusual - most have a + key which is double size, but I have found one which matches: https://www.amazon.co.uk/Adesso-2-4ghz- ... B078TG1WZ4 which I notice is "spill resistant" and maybe that was a criterion. Or, of course, the person drawing the diagram forgot what the layout looked like.

I notice that Dan's website has handy collections of papers at https://danariely.com/resources/#research so if anyone has lots of time on their hands they could pick some at random and see if there's anything interesting in them.

User avatar
sTeamTraen
After Pie
Posts: 2551
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: Dishonest research about dishonesty

Post by sTeamTraen » Wed Aug 25, 2021 7:59 am

Millennie Al wrote:
Wed Aug 25, 2021 12:37 am
sTeamTraen wrote:
Mon Aug 23, 2021 8:38 pm
Millennie Al wrote:
Mon Aug 23, 2021 1:09 am
Why the complicated keyboard entry scheme? Just use a mouse!
The interface was apparently designed to be used by the participant's non-dominant hand, for some reason. :lol:
I noticed that in the paper, but surely nearly everyone can use a mouse with either hand?
I can't. Completely useless. Even if I reassign the buttons.
Something something hammer something something nail

User avatar
Stephanie
Stummy Beige
Posts: 2896
Joined: Wed Sep 25, 2019 4:38 pm
Location: clinging tenaciously to your buttocks

Re: Dishonest research about dishonesty

Post by Stephanie » Wed Aug 25, 2021 9:52 am

Millennie Al wrote:
Mon Aug 23, 2021 1:09 am
sTeamTraen wrote:
Sun Aug 22, 2021 10:25 am
Have a look at The Heat of the Moment: The Effect of Sexual Arousal on Sexual Decision Making
and think how it might work. In one of his books Ariely tosses off a remark about "a darkened room and a copy of Playboy", which seems self-contradictory.
There's a fair bit in there that I don't understand or seems wrong. Why the complicated keyboard entry scheme? Just use a mouse! And if you have a "discrete scale with twenty-six steps" why convert that to 0..100 instead of just leaving it? And what is meant by "twenty six steps"? Since the middle value is supposed to be neutral, you need an odd number of steps. Does it mean 0..26? And the labels on the scale are "No...Possibly...Yes", yet the meanings of the questions seem to require it to mean "Dislike ... Neutral ... Like". Why isn't there a screenshot rather than the figure which is quite obviously not the real thing as it has far too little space for the questions? What p.rn was supplied for the pictures? Did the experiment assume that everyone's tastes are the same or include a variety? (Though the answers seem to be more like one person answering 35 times than 35 people answering once, so maybe the tastes were identical). Why is the least popular answer for the aroused state "Can you imagine having sex with a man?" - this means that out of 24 UCB male students none of them was gay or even slightly bi. Note that this answer beats even the questons on a 12-year old girl and animals. How do you know when you're 75% aroused? What does that even mean? Very trusting to allow 35 subjects to take the experimental laptops away for a day. Does that mean there were 35 laptops, or the experiment only used a few subjects at a time? If there was one laptop that would take 71 days, so presumably several were used.

This point may apply to many other papers: it states that particpiants were paid at $10 per session, so that's $710, but the paper does not acknowledge any funding source. Does this mean the authors paid out of their own pockets? Or one of their institutions did? If so, someone might be able to check financial records and verify how much was paid and when (not much chance as the amount is so low and it's so long ago).

I'd have to agree that the standard deviations and p values are most impressive. I'd like to see the underlying data. I'm sure it would be very informative.

And since people seem to have overlooked the link that Sciolus posted above to the first paper, here it is again: Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end
Yes, some of the answers made me raise my eyebrow.

Also the weird cartoon image of the p.rn was very odd.
"I got a flu virus named after me 'cause I kissed a bat on a dare."

Millennie Al
After Pie
Posts: 1621
Joined: Mon Mar 16, 2020 4:02 am

Re: Dishonest research about dishonesty

Post by Millennie Al » Thu Aug 26, 2021 2:38 am

sTeamTraen wrote:
Wed Aug 25, 2021 7:59 am
Millennie Al wrote:
Wed Aug 25, 2021 12:37 am
sTeamTraen wrote:
Mon Aug 23, 2021 8:38 pm


The interface was apparently designed to be used by the participant's non-dominant hand, for some reason. :lol:
I noticed that in the paper, but surely nearly everyone can use a mouse with either hand?
I can't. Completely useless. Even if I reassign the buttons.
What about the other one-handed operation that this study required? Does that also require a specific hand?

User avatar
sTeamTraen
After Pie
Posts: 2551
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: Dishonest research about dishonesty

Post by sTeamTraen » Thu Aug 26, 2021 10:02 am

Millennie Al wrote:
Thu Aug 26, 2021 2:38 am
What about the other one-handed operation that this study required? Does that also require a specific hand?
So I'm told. Otherwise it feels like someone else is doing it. Probably very badly in my case.
Something something hammer something something nail

Millennie Al
After Pie
Posts: 1621
Joined: Mon Mar 16, 2020 4:02 am

Re: Dishonest research about dishonesty

Post by Millennie Al » Fri Aug 27, 2021 12:20 am

sTeamTraen wrote:
Thu Aug 26, 2021 10:02 am
Millennie Al wrote:
Thu Aug 26, 2021 2:38 am
What about the other one-handed operation that this study required? Does that also require a specific hand?
So I'm told. Otherwise it feels like someone else is doing it. Probably very badly in my case.
I thought it was supposed to be an advantage to feel as if someone else is doing it!

IvanV
Stummy Beige
Posts: 2659
Joined: Mon May 17, 2021 11:12 am

Re: Dishonest research about dishonesty

Post by IvanV » Wed Sep 01, 2021 3:24 pm

It's taken till yesterday for a copy of the Economist with the nice graphs to arrive on my doormat, and allow me to read the accompanying article.

I'm entertained to see it was Ariely who wrote the book The Honest Truth about Dishonesty: How we lie to everyone - Especially ourselves

Michael S Roth's review nicely encapsulates it:
Ariely raises the bar for everyone. In the increasingly crowded field of popular cognitive science and behavioral economics, he writes with an unusual combination of verve and sagacity. He asks us to remember our fallibility and irrationality, so that we might protect ourselves against our tendency to fool ourselves.

User avatar
jimbob
Light of Blast
Posts: 5276
Joined: Mon Nov 11, 2019 4:04 pm
Location: High Peak/Manchester

Re: Dishonest research about dishonesty

Post by jimbob » Thu Sep 02, 2021 9:01 am

sTeamTraen wrote:
Sun Aug 22, 2021 10:29 am
secret squirrel wrote:
Sun Aug 22, 2021 10:21 am
The result reported in the original paper is one of those things that just looks obviously false. Like, even if there were an effect from doing something like that, it would be so thoroughly drowned out by the noise of human existence that any positive result reported from a paper not going to extreme lengths to detect it is far more likely to be experiment error or fraud.
The entire premise of TED-talk gee-whiz social psychology is that there are large true effects out there which, although they can only be detected by charismatic social scientists in cute experiments that are written up in vague terms so that any specific claim can be denied later, nevertheless can bring success in one's personal or business life using this One Weird Trick in an airport book that sells for a bargain $22.99.
Nicely put
Have you considered stupidity as an explanation

User avatar
sTeamTraen
After Pie
Posts: 2551
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: Dishonest research about dishonesty

Post by sTeamTraen » Sat Jun 17, 2023 2:53 pm

Bump... there's more, much more to this story.

The 2012 paper that was retracted is to be dug up and re-retracted because another study in it, which did not involve Dan Ariely, was fraudulent. Stephanie Lee has the basics of the story in the Chronicle of Higher Education, but there is a lot more to come about Francesca Gino, and probably Ariely as well. For a start, the Data Colada gang have struck again, and they are not mincing any words this time.

Full text of the Chronicle article:
Stephanie Lee wrote: A Weird Research-Misconduct Scandal About Dishonesty Just Got Weirder
By Stephanie M. Lee
JUNE 16, 2023

Almost two years ago, a famous study about a clever way to prompt honest behavior was retracted due to an ironic revelation: It relied on fraudulent data. But The Chronicle has learned of yet another twist in the story.

According to one of the authors, Harvard University found that the study contained even more fraudulent data than previously revealed and it’s now asking the journal to note this new information. The finding is part of an investigation into a series of papers that Harvard has been conducting for more than a year, the author said.

Details about the reported fabrications are unclear. Francesca Gino, a world-renowned Harvard Business School professor who studies dishonesty, and is a co-author on the disputed study, is now on administrative leave, according to her faculty page. Gino did not return a request for comment.

The head-spinning saga began in 2012, when a team of five researchers claimed that three experiments they’d done separately, and combined into one paper, showed that when people signed an honesty pledge at the beginning of a form, versus the end, they were less likely to cheat on the form. This intuitive-sounding conclusion turned heads at government agencies and companies.

But by 2020, it was falling apart. The researchers, plus two others, reported in a new paper that they were unable to replicate the effect after running essentially larger versions of experiments Nos. 1 and 2, which involved university students and employees filling out tax forms in a lab. Max H. Bazerman, a Harvard Business School professor, has said that the two experiments were written up by him, Gino, and Lisa Shu, then of Northwestern University.

Scientific findings often fail to replicate for all kinds of reasons, not necessarily because they were fabricated. But in the summer of 2021, a trio of data detectives wrote on their blog that a close examination pointed to fraud in experiment No. 3, which, unlike the others, was based on auto-insurance customer data.

That experiment had been handled by two other authors: Nina Mazar, formerly of the University of Toronto, and Dan Ariely, a Duke University professor. The source of that fraud remains unclear. In 2021, Ariely told BuzzFeed News that he was the only author in touch with the insurance company that provided the data, but he denied fabricating it. At the same time, he gave conflicting answers about the origins of the data file that was the basis for the analysis. BuzzFeed News reported that the insurer was The Hartford, which confirmed doing a “small project” with Ariely but was unable to locate any data resulting from it.

It was yet another blow to the field of behavioral economics — which in the 2000s and 2010s churned out headline-grabbing strategies to subtly influence people’s behavior for the better, and has since walked back many of them. In September 2021, the Proceedings of the National Academy of Sciences retracted the 2012 paper. But that, it turned out, was not quite the end.

The alleged new problems involve experiment No. 1 — one of the two conducted in a lab with students. Bazerman told The Chronicle that on Tuesday, Harvard informed him that it believed fabricated data for this experiment made it invalid. According to Bazerman, Harvard provided a 14-page document with what he described as “compelling evidence” of data alterations. Their analysis found that somebody had accessed a database and added and altered data in the file, he said. “I did not have anything to do with the fabrication,” he told The Chronicle.

According to Bazerman, Harvard is recommending to the Proceedings of the National Academy of Sciences that it update the study’s retraction notice to reflect its new concerns. (A journal spokesperson said, “We are looking into the matter.” A Harvard spokesperson declined to comment.)

Bazerman declined to discuss his co-authors. But in his book Complicit: How We Enable the Unethical and How to Stop, published in November, the Harvard professor reflected on the debacle of the 2012 study. How was it, he mused, that experiments Nos. 1 and 2 had both ended up being irreproducible?

“In retrospect, Gino reported that her lab manager at her prior university managed data collection for the two laboratory experiments in the 2012 paper,” Bazerman wrote in a chapter about the risks of putting trust in relationships. “Thus, none of the authors, including me, provided sufficient supervision of these experiments. In addition, as I review emails from 2011 containing the dialogue between coauthors of the 2012 paper, I see concerns raised about the methods. I failed to actively engage and deferred to the decisions of my colleagues, and that failure makes me complicit.”

He added, “The irony of this being a story about data fraud in a paper on inducing honesty is not lost on me.”

Bazerman told The Chronicle that his understanding is that the 2012 paper is one of four papers “of significant concern” to Harvard. He declined to identify the other three, but said he was not a co-author on them.

Gino joined the Harvard Business School faculty in 2010 after stints at the University of North Carolina and Carnegie Mellon University. She shot to academic stardom with her prolific, buzzy research on decision-making, leadership, and workplace behavior — including dishonesty and unethical behavior.

Her expertise has made her in demand as a consultant and speaker to some of the world’s biggest companies and institutions — Bacardi, Google, Microsoft, Coca-Cola, Disney, Goldman Sachs, Honeywell, Novartis, Procter & Gamble, and the U.S. Air Force, Army, and Navy — and as a source to media outlets from The New York Times to The Wall Street Journal, according to her website. She has co-authored more than 135 academic articles and written books including 2018’s Rebel Talent: Why It Pays to Break the Rules at Work and in Life. She’s also been named one of the world’s 40 best business-school professors under 40 and 50 most influential management thinkers.

And as of recently, her role at Harvard is unclear. Within the last month, her faculty website was updated to say that she is on administrative leave, according to screenshots captured by the Wayback Machine. The Harvard spokesperson declined to comment on Gino’s status.

Shu, the third collaborator on the experiment in question, did not return a request for comment.
Something something hammer something something nail

User avatar
sTeamTraen
After Pie
Posts: 2551
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: Dishonest research about dishonesty

Post by sTeamTraen » Sat Jul 29, 2023 11:59 am

Going back to the original story at the top of this thread: The insurance company — which, apart from Ariely, was the only possible source of the fabrication — has come out with a statement, clearly written after taking a great deal of legal and statistical advice, demonstrating that the data were fine when they sent it to Ariely.

A couple of contrarians on Twitter or EJMR are still trying to argue that this 200-year-old insurance company, whose entire business is based on trust, might have fabricated both the dataset that they actually sent and this "new" dataset that they claim they sent. But basically, it's all over for Ariely... unless his university (and its unimprovably-named "Fuqua School of Business") decides to just pretend that there is nothing to see here. Perhaps they will finally take action if the alumni office starts getting pushback from potential donors, which I heard was one of the factors in Cornell finally deciding to do something about Brian Wansink.
Something something hammer something something nail

User avatar
sTeamTraen
After Pie
Posts: 2551
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: Dishonest research about dishonesty

Post by sTeamTraen » Wed Aug 02, 2023 11:09 pm

Clucking bell.
After being accused of research fraud, Harvard Business School professor Francesca Gino filed a defamation and discrimination lawsuit Wednesday against Harvard University and three academics who detailed the fraud allegations in their blog.

The lawsuit, filed in US District Court in Boston, names Harvard Business School dean Srikant Datar as well as three authors of the Data Colada blog, who alleged in a four-part series that four academic articles Gino cowrote contained fraudulent data. The three named authors are professors Uri Simonsohn, of Esade Business School in Spain; Leif Nelson, at the University of California, Berkeley; and Joseph Simmons, at the University of Pennsylvania.
Something something hammer something something nail

User avatar
sTeamTraen
After Pie
Posts: 2551
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: Dishonest research about dishonesty

Post by sTeamTraen » Fri Nov 10, 2023 4:56 pm

I found a clearly fake study from Gino's lab. But it's OK because the research assistants did it. Gino is so unlucky with her research assistants.
Something something hammer something something nail

Allo V Psycho
Catbabel
Posts: 734
Joined: Sat Nov 16, 2019 8:18 am

Re: Dishonest research about dishonesty

Post by Allo V Psycho » Fri Nov 10, 2023 6:26 pm

sTeamTraen wrote:
Fri Nov 10, 2023 4:56 pm
I found a clearly fake study from Gino's lab. But it's OK because the research assistants did it. Gino is so unlucky with her research assistants.
Brilliant stuff, Steamy. Particularly delightful understatement.

Is it possible the research assistants were called Margaret Howard and J. Conway? I believe no evidence as to their actual height has ever been obtained, so they could fit your description.

Holylol
Sindis Poop
Posts: 79
Joined: Tue Nov 19, 2019 10:14 am

Re: Dishonest research about dishonesty

Post by Holylol » Sun Nov 12, 2023 2:10 pm

sTeamTraen wrote:
Fri Nov 10, 2023 4:56 pm
I found a clearly fake study from Gino's lab. But it's OK because the research assistants did it. Gino is so unlucky with her research assistants.
I've only be hearing about the Gino-Ariely stuff from afar, but is this new work of yours part of an effort to look more widely at their body of work?

User avatar
sTeamTraen
After Pie
Posts: 2551
Joined: Mon Nov 11, 2019 4:24 pm
Location: Palma de Mallorca, Spain

Re: Dishonest research about dishonesty

Post by sTeamTraen » Tue Nov 21, 2023 1:01 pm

Holylol wrote:
Sun Nov 12, 2023 2:10 pm
sTeamTraen wrote:
Fri Nov 10, 2023 4:56 pm
I found a clearly fake study from Gino's lab. But it's OK because the research assistants did it. Gino is so unlucky with her research assistants.
I've only be hearing about the Gino-Ariely stuff from afar, but is this new work of yours part of an effort to look more widely at their body of work?
Yes and no. I was able to look at the data because Gino's past co-authors have put together a project to try and see where she could have been involved in collecting the data and I found the link on there, so I just went poking around.

I have found a couple of other dodgy papers, but the issues don't rise to the level of "needs a blog post". Some study designs are so simple that you can't tell if the data are fraudulent or not, because the manipulation to make them fraudulent would be very easy to do without leaving any traces.
Something something hammer something something nail

Post Reply