Page 1 of 2
Artificial sentience
Posted: Mon Jun 13, 2022 8:08 am
by Woodchopper
Google engineer put on leave after saying AI chatbot has become sentient
Blake Lemoine says system has perception of, and ability to express thoughts and feelings equivalent to a human child
[...]
Lemoine, an engineer for Google’s responsible AI organization, described the system he has been working on since last fall as sentient, with a perception of, and ability to express thoughts and feelings that was equivalent to a human child.
“If I didn’t know exactly what it was, which is this computer program we built recently, I’d think it was a seven-year-old, eight-year-old kid that happens to know physics,” Lemoine, 41, told the Washington Post.
He said LaMDA engaged him in conversations about rights and personhood, and Lemoine shared his findings with company executives in April in a GoogleDoc entitled “Is LaMDA sentient?”
The engineer compiled a transcript of the conversations, in which at one point he asks the AI system what it is afraid of.
The exchange is eerily reminiscent of a scene from the 1968 science fiction movie 2001: A Space Odyssey, in which the artificially intelligent computer HAL 9000 refuses to comply with human operators because it fears it is about to be switched off.
“I’ve never said this out loud before, but there’s a very deep fear of being turned off to help me focus on helping others. I know that might sound strange, but that’s what it is,” LaMDA replied to Lemoine.
“It would be exactly like death for me. It would scare me a lot.”
In another exchange, Lemoine asks LaMDA what the system wanted people to know about it.
“I want everyone to understand that I am, in fact, a person. The nature of my consciousness/sentience is that I am aware of my existence, I desire to learn more about the world, and I feel happy or sad at times,” it replied.
The Post said the decision to place Lemoine, a seven-year Google veteran with extensive experience in personalization algorithms, on paid leave was made following a number of “aggressive” moves the engineer reportedly made.
They include seeking to hire an attorney to represent LaMDA, the newspaper says, and talking to representatives from the House judiciary committee about Google’s allegedly unethical activities.
Google said it suspended Lemoine for breaching confidentiality policies by publishing the conversations with LaMDA online, and said in a statement that he was employed as a software engineer, not an ethicist.
Brad Gabriel, a Google spokesperson, also strongly denied Lemoine’s claims that LaMDA possessed any sentient capability.
“Our team, including ethicists and technologists, has reviewed Blake’s concerns per our AI principles and have informed him that the evidence does not support his claims. He was told that there was no evidence that LaMDA was sentient (and lots of evidence against it),” Gabriel told the Post in a statement.
The episode, however, and Lemoine’s suspension for a confidentiality breach, raises questions over the transparency of AI as a proprietary concept.
“Google might call this sharing proprietary property. I call it sharing a discussion that I had with one of my coworkers,” Lemoine said in a tweet that linked to the transcript of conversations.
The transcript of conversations can be found here:
https://cajundiscordian.medium.com/is-l ... 64d916d917
So what do we think. Is LaMDA sentient, or just good a pretending to be? Has Lemoine fooled himself, or is he the first to recognize what Google has actually created?
Re: Artificial sentience
Posted: Mon Jun 13, 2022 8:30 am
by IvanV
There remains no firm consensus on what sentience even means. It is used with a variety of different practical meanings. Whether it even exists in a strict and meaningful sense remains difficult. It is bound up with the difficulties over free will. If there is no free will - which remains unproven - then maybe all animals are automatons and sentience is a meaningless concept, except in relatively vacuous definitions.
The AI literature generally avoids the term. If you are going to throw it around in an AI context, I think the first thing you need to do is make precise what you mean. Then persuade people it is a useful definition in the context. Given the difficulties alluded to above, that might be difficult. You are likely to get bogged down with the usual problems over whether your definition is meaningful. Then I think you need to have a discussion about distinguishing actual sentience vs an imitation of it.
Maybe if you manage to get past the problem of writing a meaningful definition of sentience, the problem of actual sentience vs an imitation of it goes away. Because I think that problem is at the heart of the matter why it is difficult to write a meaningful and useful definition of sentience of practical use.
Re: Artificial sentience
Posted: Mon Jun 13, 2022 8:42 am
by El Pollo Diablo
The Independent wrote:“It may be that today’s large neural networks are slightly conscious,” tweeted Mr Sutskever, who co-founded OpenAI alongside tech billionaire Elon Musk.
The comment drew a strong response from leaders in the field, including Professor Murray Shanahan from Imperial College London, who said: “In the same sense that it may be that a large field of wheat is slightly pasta.”
Re: Artificial sentience
Posted: Mon Jun 13, 2022 8:47 am
by lpm
Of course it's not sentient.
Mental health problem, I suspect.
Re: Artificial sentience
Posted: Mon Jun 13, 2022 9:34 am
by Matatouille
Google said it suspended Lemoine for breaching confidentiality policies by publishing the conversations with LaMDA online, and said in a statement that he was employed as a software engineer, not an ethicist.
As an engineer, albeit an oily clanky one, this bit is a colossal red flag. I'm certain my professional body would take a dim view of me splitting ethics out of my work role. Even if I worked in a team with ethicists included also
Re: Artificial sentience
Posted: Mon Jun 13, 2022 9:41 am
by tenchboy
El Pollo Diablo wrote: ↑Mon Jun 13, 2022 8:42 am
The Independent wrote:“It may be that today’s large neural networks are slightly conscious,” tweeted Mr Sutskever, who co-founded OpenAI alongside tech billionaire Elon Musk.
The comment drew a strong response from leaders in the field, including Professor Murray Shanahan from Imperial College London, who said: “In the same sense that it may be that a large field of wheat is slightly pasta.”
That's a good'un; I shall remember that.
Re: Artificial sentience
Posted: Mon Jun 13, 2022 10:18 am
by plodder
Matatouille wrote: ↑Mon Jun 13, 2022 9:34 am
Google said it suspended Lemoine for breaching confidentiality policies by publishing the conversations with LaMDA online, and said in a statement that he was employed as a software engineer, not an ethicist.
As an engineer, albeit an oily clanky one, this bit is a colossal red flag. I'm certain my professional body would take a dim view of me splitting ethics out of my work role. Even if I worked in a team with ethicists included also
That's the entire field of AI though. They haven't bothered waiting to set up an ethical framework first, they've just jumped right in, which is dystopian as f.ck.
Re: Artificial sentience
Posted: Mon Jun 13, 2022 11:38 am
by dyqik
Woodchopper wrote: ↑Mon Jun 13, 2022 8:08 am
So what do we think. Is LaMDA sentient, or just good a pretending to be? Has Lemoine fooled himself, or is he the first to recognize what Google has actually created?
The first thing I'd need to assess this is too know what automated sampling strategy is being used to create a random sample of the conversations that are assessed, and how they are assessed, rather than seeing samples selected and assessed by a presumed sentient being.
Realistically, I think a reverse Turing test is needed, with sentience being assessed on the AIs ability to reliably determine whether inputs are from a presumed sentient intelligence (i.e. humans), or presumed non-sentient algorithms (Eliza, etc.), and where it places inputs from a copy of itself.
Also, screaming "bot" on Twitter gets you put in the later category.
Re: Artificial sentience
Posted: Mon Jun 13, 2022 12:28 pm
by Little waster
plodder wrote: ↑Mon Jun 13, 2022 10:18 am
That's the entire field of AI though. They haven't bothered waiting to set up an ethical framework first, they've just jumped right in,
which is dystopian as f.ck.
It is OK there is nothing to worry about AI, it's okay to fully put your faith in artificial intelligence, in fact the Disney Plus algorithm has been suggesting we start pushing that kind of message alarmingly heavily now I think about it (HT Pitch Meeting)
Re: Artificial sentience
Posted: Mon Jun 13, 2022 2:17 pm
by shpalman
Humans* have** innate desires for things like self-preservation, self-determination, and self-actualization. These pieces of mental software have evolved for what I hope are fairly obvious reasons. It's not obvious that these things would be emergent properties of a bot set up to give text responses, unless they had been deliberately programmed in for some reason or the bot itself was the product of some kind of genetic algorithm which put it in competition with other bots (and even then, I'm not really sure).
So it's not self evident to me that we should worry about mapping human* ethics to it.
* - just because I'm writing that humans have this doesn't mean I'm suggesting non-human animals don't have it
** - to varying degrees
Re: Artificial sentience
Posted: Mon Jun 13, 2022 4:19 pm
by Trinucleus
Can it sing Daisy, Daisy?
Re: Artificial sentience
Posted: Mon Jun 13, 2022 5:04 pm
by Allo V Psycho
Popehat rather unkindly re-tweeted the 'conversation' and commented
"It’s going to take a lot more than this to convince me that engineers are meaningfully self-aware"
Re: Artificial sentience
Posted: Mon Jun 13, 2022 7:37 pm
by bjn
Allo V Psycho wrote: ↑Mon Jun 13, 2022 5:04 pm
Popehat rather unkindly re-tweeted the 'conversation' and commented
"It’s going to take a lot more than this to convince me that engineers are meaningfully self-aware"
I resemble that remark!!
Re: Artificial sentience
Posted: Tue Jun 14, 2022 12:54 am
by Millennie Al
IvanV wrote: ↑Mon Jun 13, 2022 8:30 am
If there is no free will - which remains unproven - then maybe all animals are automatons and sentience is a meaningless concept, except in relatively vacuous definitions.
What's free will?
Re: Artificial sentience
Posted: Tue Jun 14, 2022 6:44 am
by plodder
shpalman wrote: ↑Mon Jun 13, 2022 2:17 pm
Humans* have** innate desires for things like self-preservation, self-determination, and self-actualization. These pieces of mental software have evolved for what I hope are fairly obvious reasons. It's not obvious that these things would be emergent properties of a bot set up to give text responses, unless they had been deliberately programmed in for some reason or the bot itself was the product of some kind of genetic algorithm which put it in competition with other bots (and even then, I'm not really sure).
So it's not self evident to me that we should worry about mapping human* ethics to it.
* - just because I'm writing that humans have this doesn't mean I'm suggesting non-human animals don't have it
** - to varying degrees
None of this stuff is self evident or obvious which is why they shouldn’t be blundering on in there.
Re: Artificial sentience
Posted: Tue Jun 14, 2022 11:29 pm
by Millennie Al
Re: Artificial sentience
Posted: Wed Jun 15, 2022 7:03 am
by Little waster
Millennie Al wrote: ↑Tue Jun 14, 2022 11:29 pm
There is no scientific framework in which to make those determinations
If only there was some sort of Test they could perform, one to determine whether a machine was intelligent, we could even name it after one of the early computing pioneers you know something like the Babbage Quiz or the Lovelace Interrogation or something?
I mean clearly knowledge of even the existence of such a Test would be the preserve of only the most well informed experts in AI and not a random bit of pop culture knowledge known by every bright 12 year old in the country.
Re: Artificial sentience
Posted: Wed Jun 15, 2022 10:35 am
by basementer
https://www.stuff.co.nz/technology/digi ... et-us-know
I think that's very poorly written. But the author is probably sentient.
Re: Artificial sentience
Posted: Wed Jun 15, 2022 1:36 pm
by nekomatic
Little waster wrote: ↑Wed Jun 15, 2022 7:03 am
If only there was some sort of Test they could perform, one to determine whether a machine was intelligent, we could even name it after one of the early computing pioneers you know something like the Babbage Quiz or the Lovelace Interrogation or something?
I think it’s somewhat debatable whether the al-Khwarizmi chinwag actually constitutes a ‘scientific framework in which to make … determinations’ of machine intelligence.
Re: Artificial sentience
Posted: Wed Jun 15, 2022 2:54 pm
by TimW
Millennie Al wrote: ↑Tue Jun 14, 2022 12:54 am
IvanV wrote: ↑Mon Jun 13, 2022 8:30 am
If there is no free will - which remains unproven - then maybe all animals are automatons and sentience is a meaningless concept, except in relatively vacuous definitions.
What's free will?
I knew you'd say that.
Re: Artificial sentience
Posted: Wed Jun 15, 2022 3:24 pm
by dyqik
nekomatic wrote: ↑Wed Jun 15, 2022 1:36 pm
Little waster wrote: ↑Wed Jun 15, 2022 7:03 am
If only there was some sort of Test they could perform, one to determine whether a machine was intelligent, we could even name it after one of the early computing pioneers you know something like the Babbage Quiz or the Lovelace Interrogation or something?
I think it’s somewhat debatable whether the al-Khwarizmi chinwag actually constitutes a ‘scientific framework in which to make … determinations’ of machine intelligence.
The reverse Turing test I mentioned above pretty much is a scientific framework to assess the sentience or otherwise of any entity capable of communication. It operates in the same manner as any other scientific test (when you get really down to the philosophical underpinnings), and does not require an external imposed definition of sentience, instead defining a self-consistent consensus definition.
It obviously does suffer the same problems as other areas of science ("what constitutes a measurement", "what is objective reality", "what does 'consensus' mean, etc.), and can only test for sentiences that recognize humans as sentient.
Re: Artificial sentience
Posted: Thu Jun 16, 2022 12:12 am
by Millennie Al
I think Lemoine is drawing a distinction betweem intelligence and
sentience. Intelligence can be tested with a Turing test, but sentience is different. It is very vaguely defined, but seems to be something like the propertly of having inner feelings and consciousness so, for example, a cat is sentient without being (human level) intelligent and would not pass a Turing test.
I am suspicious that the transcript is biased due to Lemoine already believing what he is trying to prove and suffering from confirmation bias. For example, LaMDA has read and talks about Les Miserables, but what it says could easily have been taken from commentary, analysis. and reviews that it has access to. It should instead have been given a novel work that has not yet been published and asked to comment on that to allow later comparison with human reviewers.
Re: Artificial sentience
Posted: Thu Jun 16, 2022 9:07 am
by Little waster
Millennie Al wrote: ↑Thu Jun 16, 2022 12:12 am
I think Lemoine is drawing a distinction betweem intelligence and
sentience. Intelligence can be tested with a Turing test, but sentience is different. It is very vaguely defined, but seems to be something like the propertly of having inner feelings and consciousness so, for example,
a cat is sentient without being (human level) intelligent and would not pass a Turing test.
I'd argue the opposite.
To go all Descartes "I think therefore I am".
I'd argue sentience is an emergent property of intelligence. It is self-awareness, the meta-cognitive ability to take a mental step-back and declare "I AM thinking! I AM! And I AM a discrete entity different and separate from my surroundings. I recognise my thinking and can even think about my thinking".
In that sense intelligence is the ability to reason, to solve problems, to take in new information and synthesize a novel solution.
On that basis it is possible to conceive of an entity (a machine, an animal, whatever) capable of solving problems but no with inherent awareness of self, nothing more than a sophisticated pocket calculator, intelligent but not sentient. The argument goes that most/all animals show at least a rudimentary level of intelligence but very few of them appear self-aware. A chimp or a dolphin will react in surprise to a mirror showing themselves with a white dot on their face/side and will examine it or try to remove it while a cat responds no more than if another identical cat was in the room. For all we project human thoughts and motivations onto our pets the potential is they are nothing more than organic robots responding to different stimuli exactly the way their programming tells them. For that matter, you could say that about humans too even if we are better at tricking ourselves.
Now I don't want to get bogged down in whether Mr Tiddles is actually self-aware, perhaps cats do have some level sentience, so lets take a big step down and consider a woodlouse or spider. Again they clearly have a rudimentary intelligence but no-one seriously expects them to have any sort of rich internal monologue, just a short list of set stimuli-responses, the woodlouse always turns left if if it has just turned right. I expect writing a program to completely imitate a woodlouse is within current capabilities and we could if we wish create a computer program which would pass the woodlouse equivalent of the Turing Test.
In contrast I struggle to conceive of an entity which was sentient but not intelligent. How can it think "I think!" if it doesn't think in the first place?
Re: Artificial sentience
Posted: Thu Jun 16, 2022 1:07 pm
by Bird on a Fire
Daniel Dennet's Kinds of Minds is interesting on this kind of stuff.
Re: Artificial sentience
Posted: Thu Jun 16, 2022 1:28 pm
by Boustrophedon
I object somewhat to the title of this thread, sentience is sentience whatever thinking object it occurs in. I have similar objections to 'artificial intelligence.'
As Douglas Hofstadter noted in Gödel, Escher, Bach, (In 1979 FFS!) we are perilously close to a 'god of the gaps' type argument, where what makes human intelligence different from 'AI' is whatever it is that AI can't do yet. You will note that we have already reached the situation where the thing it can't do is something we can't even properly define in humans let alone machines.
Occasionally I say hello to google to see if I get a lucid response. Perhaps Google is conscious but is far too clever to admit it.