Name that audio effect! (And maybe explain it)

Discussions about serious topics, for serious people
Post Reply
User avatar
Bird on a Fire
Light of Blast
Posts: 4290
Joined: Fri Oct 11, 2019 5:05 pm
Location: nadir of brie

Name that audio effect! (And maybe explain it)

Post by Bird on a Fire » Sun Nov 01, 2020 2:45 am

So for years I've been noticing this particular audio effect, where it sort of makes the vocals sound like a robot or something, but I can't quite articulate what it is. The sort of frictiony sounds you get from physical lips/tongues aren't there, but I can still work out what's being said.

Two examples:
Beastie Boys - Intergalactic

S Club 7 - Don't Stop Moving

Also, can anyone explain it in very short words? I can't quite work out why I'm still able to understand the words without what ought to be crucial bits of the soundscape going missing somehow.

PS If you know what I'm talking about from the first track, don't bother listening to the second track
THINK BIG AND UPEND THE SYSTEM

#ShowYourStripes

User avatar
shpalman
After Pie
Posts: 2215
Joined: Mon Nov 11, 2019 12:53 pm
Location: One step beyond

Re: Name that audio effect! (And maybe explain it)

Post by shpalman » Sun Nov 01, 2020 6:37 am

10 most famous vocoder songs

Related technology is the talk box
molto tricky

User avatar
shpalman
After Pie
Posts: 2215
Joined: Mon Nov 11, 2019 12:53 pm
Location: One step beyond

Re: Name that audio effect! (And maybe explain it)

Post by shpalman » Sun Nov 01, 2020 7:00 am

A vocoder works by analyzing the frequency spectrum of whatever is being sung or spoken into the microphone, and then filtering the output of the synthesizer by that spectrum.* So it recreates the vowel sounds of the human from the synthesized sound. Some vocoders also have a white noise channel to also help reproduce the "frictiony sounds" to make the speech more intelligible.

Mr. Blue Sky

(* - human speech, for the vowels anyway, is basically the output of the vocal chords being filtered by the shape of the oral cavity, and the talk box is the sound of the guitar transmitted through a rubber tube into your mouth where it can be filtered in the same way before being picked up by a microphone.)
molto tricky

User avatar
dyqik
Stummy Beige
Posts: 2753
Joined: Wed Sep 25, 2019 4:19 pm
Location: Masshole
Contact:

Re: Name that audio effect! (And maybe explain it)

Post by dyqik » Sun Nov 01, 2020 11:19 am

In addition to old-school vocoders, the excessive use of auto-tune can give something that's basically the reverse thing, but can sound kind of similarly weird.

RobS
Gray Pubic
Posts: 15
Joined: Mon Nov 11, 2019 2:02 pm

Re: Name that audio effect! (And maybe explain it)

Post by RobS » Sun Nov 01, 2020 4:06 pm

Bird on a Fire wrote:
Sun Nov 01, 2020 2:45 am
I can't quite work out why I'm still able to understand the words without what ought to be crucial bits of the soundscape going missing somehow.
That's a combination of very clear speech being highly redundant (for most speech sounds there are multiple cues to their identity )and that speech is a massively overlearned stimulus so the brain is amazing at inferring missing information from what it's currently hearing and what it's heard just before.

So there's the phonemic restoration effect, where a speech signal with small gaps in it can be unintelligible but if you fill the gaps with a sufficiently loud noise then the auditory system is often able to infer the missing information and it's as though you hear it. The examples at the end of the wiki are best listened to over headphones.

There's also sine-wave speech, which is a great example of how speech is highly redundant. Early speech synthesizers simulated the first three formants (resonances) of the vocal tract by filtering the output of a broadband buzzy sound (simulated glottal pulse) through three narrowband frequency filters whose frequency response could be changed to mimic the filtering of the vocal tract. Sine-wave speech dispense with the buzzy sound and just mimics the formants with frequency-modulated sine tones - this kind of speech is moderately intelligible with a little practice, but unlike real speech is highly susceptible to interference.

User avatar
Bird on a Fire
Light of Blast
Posts: 4290
Joined: Fri Oct 11, 2019 5:05 pm
Location: nadir of brie

Re: Name that audio effect! (And maybe explain it)

Post by Bird on a Fire » Sun Nov 01, 2020 9:16 pm

shpalman wrote:
Sun Nov 01, 2020 7:00 am
A vocoder works by analyzing the frequency spectrum of whatever is being sung or spoken into the microphone, and then filtering the output of the synthesizer by that spectrum.* So it recreates the vowel sounds of the human from the synthesized sound. Some vocoders also have a white noise channel to also help reproduce the "frictiony sounds" to make the speech more intelligible.

Mr. Blue Sky

(* - human speech, for the vowels anyway, is basically the output of the vocal chords being filtered by the shape of the oral cavity, and the talk box is the sound of the guitar transmitted through a rubber tube into your mouth where it can be filtered in the same way before being picked up by a microphone.)
Thank you! After quite a bit of time down a youtube vocoder and talkbox rabbithole I think I get what's going on. I'd never realised that in a talkbox the guitar sound goes through the human mouth, rather than human voice going down the tube into an amp or something - they're even weirder than I thought!
RobS wrote:
Sun Nov 01, 2020 4:06 pm
Bird on a Fire wrote:
Sun Nov 01, 2020 2:45 am
I can't quite work out why I'm still able to understand the words without what ought to be crucial bits of the soundscape going missing somehow.
That's a combination of very clear speech being highly redundant (for most speech sounds there are multiple cues to their identity )and that speech is a massively overlearned stimulus so the brain is amazing at inferring missing information from what it's currently hearing and what it's heard just before.

So there's the phonemic restoration effect, where a speech signal with small gaps in it can be unintelligible but if you fill the gaps with a sufficiently loud noise then the auditory system is often able to infer the missing information and it's as though you hear it. The examples at the end of the wiki are best listened to over headphones.

There's also sine-wave speech, which is a great example of how speech is highly redundant. Early speech synthesizers simulated the first three formants (resonances) of the vocal tract by filtering the output of a broadband buzzy sound (simulated glottal pulse) through three narrowband frequency filters whose frequency response could be changed to mimic the filtering of the vocal tract. Sine-wave speech dispense with the buzzy sound and just mimics the formants with frequency-modulated sine tones - this kind of speech is moderately intelligible with a little practice, but unlike real speech is highly susceptible to interference.
This is interesting, thanks. For both of those examples I find them pretty much impossible to understand until after I've heard the clear speech, and then it becomes pretty easy, whereas understanding new-to-me vocoder segments on the link from shpalman was a bit easier. Cool stuff though.

Sine-wave speech is particularly weird sounding, and I don't think I've heard much use of it in music or film despite its potential.
THINK BIG AND UPEND THE SYSTEM

#ShowYourStripes

User avatar
basementer
Snowbonk
Posts: 587
Joined: Mon Nov 11, 2019 1:03 pm
Location: Aotearoa

Re: Name that audio effect! (And maybe explain it)

Post by basementer » Sun Nov 01, 2020 10:23 pm

Bird on a Fire wrote:
Sun Nov 01, 2020 9:16 pm
shpalman wrote:
Sun Nov 01, 2020 7:00 am
A vocoder works by analyzing the frequency spectrum of whatever is being sung or spoken into the microphone, and then filtering the output of the synthesizer by that spectrum.* So it recreates the vowel sounds of the human from the synthesized sound. Some vocoders also have a white noise channel to also help reproduce the "frictiony sounds" to make the speech more intelligible.

Mr. Blue Sky

(* - human speech, for the vowels anyway, is basically the output of the vocal chords being filtered by the shape of the oral cavity, and the talk box is the sound of the guitar transmitted through a rubber tube into your mouth where it can be filtered in the same way before being picked up by a microphone.)
Thank you! After quite a bit of time down a youtube vocoder and talkbox rabbithole I think I get what's going on. I'd never realised that in a talkbox the guitar sound goes through the human mouth, rather than human voice going down the tube into an amp or something - they're even weirder than I thought!
RobS wrote:
Sun Nov 01, 2020 4:06 pm
Bird on a Fire wrote:
Sun Nov 01, 2020 2:45 am
I can't quite work out why I'm still able to understand the words without what ought to be crucial bits of the soundscape going missing somehow.
That's a combination of very clear speech being highly redundant (for most speech sounds there are multiple cues to their identity )and that speech is a massively overlearned stimulus so the brain is amazing at inferring missing information from what it's currently hearing and what it's heard just before.

So there's the phonemic restoration effect, where a speech signal with small gaps in it can be unintelligible but if you fill the gaps with a sufficiently loud noise then the auditory system is often able to infer the missing information and it's as though you hear it. The examples at the end of the wiki are best listened to over headphones.

There's also sine-wave speech, which is a great example of how speech is highly redundant. Early speech synthesizers simulated the first three formants (resonances) of the vocal tract by filtering the output of a broadband buzzy sound (simulated glottal pulse) through three narrowband frequency filters whose frequency response could be changed to mimic the filtering of the vocal tract. Sine-wave speech dispense with the buzzy sound and just mimics the formants with frequency-modulated sine tones - this kind of speech is moderately intelligible with a little practice, but unlike real speech is highly susceptible to interference.
This is interesting, thanks. For both of those examples I find them pretty much impossible to understand until after I've heard the clear speech, and then it becomes pretty easy, whereas understanding new-to-me vocoder segments on the link from shpalman was a bit easier. Cool stuff though.

Sine-wave speech is particularly weird sounding, and I don't think I've heard much use of it in music or film despite its potential.
I've often thought that sine-wave speech could feature in a Star Trek or Dr Who episode, the idea being that there's a flock of birds or similar that have a shared consciousness. When there are many, their song combines into easily comprehensible human speech, but one of them alone can't be understood. An obvious feature of the plot would be that one of them gets isolated, and as the heroes manage to find and bring together more of its flock, the song builds up into a coherent message.
Money is just a substitute for luck anyway. - Tom Siddell

User avatar
Bird on a Fire
Light of Blast
Posts: 4290
Joined: Fri Oct 11, 2019 5:05 pm
Location: nadir of brie

Re: Name that audio effect! (And maybe explain it)

Post by Bird on a Fire » Sun Nov 01, 2020 10:43 pm

basementer wrote:
Sun Nov 01, 2020 10:23 pm
Bird on a Fire wrote:
Sun Nov 01, 2020 9:16 pm
Sine-wave speech is particularly weird sounding, and I don't think I've heard much use of it in music or film despite its potential.
I've often thought that sine-wave speech could feature in a Star Trek or Dr Who episode, the idea being that there's a flock of birds or similar that have a shared consciousness. When there are many, their song combines into easily comprehensible human speech, but one of them alone can't be understood. An obvious feature of the plot would be that one of them gets isolated, and as the heroes manage to find and bring together more of its flock, the song builds up into a coherent message.
Ooooh I love that!
THINK BIG AND UPEND THE SYSTEM

#ShowYourStripes

User avatar
Gfamily
After Pie
Posts: 1872
Joined: Mon Nov 11, 2019 1:00 pm
Location: NW England

Re: Name that audio effect! (And maybe explain it)

Post by Gfamily » Sun Nov 01, 2020 11:12 pm

Bird on a Fire wrote:
Sun Nov 01, 2020 2:45 am
I can't quite work out why I'm still able to understand the words without what ought to be crucial bits of the soundscape going missing somehow.
that'll be non verbal communication, according to crap training courses everywhere :)
My avatar was a scientific result that was later found to be 'mistaken' - I rarely claim to be 100% correct
ETA 5/8/20: I've been advised that the result was correct, it was the initial interpretation that needed to be withdrawn
Meta? I'd say so!

RobS
Gray Pubic
Posts: 15
Joined: Mon Nov 11, 2019 2:02 pm

Re: Name that audio effect! (And maybe explain it)

Post by RobS » Mon Nov 02, 2020 9:17 pm

Bird on a Fire wrote:
Sun Nov 01, 2020 9:16 pm
This is interesting, thanks. For both of those examples I find them pretty much impossible to understand until after I've heard the clear speech, and then it becomes pretty easy, whereas understanding new-to-me vocoder segments on the link from shpalman was a bit easier. Cool stuff though.
One of the cool (to me anyway) things about this effect is that because many people don't hear sine-wave speech as speech until they're told it's speech you can scan their brain with MEG/fMRI and see differential processing of sine-wave speech dependent on whether or not they recognise it as speech .

User avatar
Bird on a Fire
Light of Blast
Posts: 4290
Joined: Fri Oct 11, 2019 5:05 pm
Location: nadir of brie

Re: Name that audio effect! (And maybe explain it)

Post by Bird on a Fire » Fri Nov 20, 2020 10:15 pm

Another cool vocoder video!

Jazz nerd bassist Adam Neely explains how to vocode a bass guitar (and also explains the theory a bit) https://www.youtube.com/watch?v=kqiEHQL7ilk
THINK BIG AND UPEND THE SYSTEM

#ShowYourStripes

User avatar
Boustrophedon
Dorkwood
Posts: 1513
Joined: Mon Nov 11, 2019 3:58 pm
Location: Lincolnshire Wolds

Re: Name that audio effect! (And maybe explain it)

Post by Boustrophedon » Sat Nov 21, 2020 12:30 am

When do you think the vocoder was invented?
Nope, before that
and that,
and that.

1942 FFS, with freaking valves, for the SIGSALY secure short wave trans Atlantic telephone connection for Roosevelt and Churchill.

https://www.cryptomuseum.com/crypto/usa ... /index.htm
"Politicians should be replaced as frequently as diapers, usually for the same reason"
- Mark Twain

User avatar
Bird on a Fire
Light of Blast
Posts: 4290
Joined: Fri Oct 11, 2019 5:05 pm
Location: nadir of brie

Re: Name that audio effect! (And maybe explain it)

Post by Bird on a Fire » Sat Nov 21, 2020 2:57 am

Yeah when I was listening to the Beastie Boys the other day it never occurred to me that the effect was a direct result of early signals processing/compression tech. Brings it dangerously close to my day job thinking about information transmission and maximum entropy and ting.

It's literally always worth starting a random thread here when you're just idly wondering about something, because people always have interesting bits of knowledge to share.
THINK BIG AND UPEND THE SYSTEM

#ShowYourStripes

User avatar
basementer
Snowbonk
Posts: 587
Joined: Mon Nov 11, 2019 1:03 pm
Location: Aotearoa

Re: Name that audio effect! (And maybe explain it)

Post by basementer » Sat Nov 21, 2020 6:30 am

Boustrophedon wrote:
Sat Nov 21, 2020 12:30 am
When do you think the vocoder was invented?
Nope, before that
and that,
and that.

1942 FFS, with freaking valves, for the SIGSALY secure short wave trans Atlantic telephone connection for Roosevelt and Churchill.

https://www.cryptomuseum.com/crypto/usa ... /index.htm
It's probably the least verifiable thing I'll ever post, but as I mentioned a few weeks ago, I have dreamed about SIGSALY.

I've got what looks to me like a better solution to the propagation delay than the one they used. They used two identical copies of the one time pad disc and built something elaborate to maintain a fixed offset between the two turntables. Why not use a single copy of the disc - synchronisation guaranteed - and a mechanical linkage to keep two play styli at a fixed angular separation? These people were clearly not stupid, I assume they were solving a problem that I haven't got as far as noticing.
Money is just a substitute for luck anyway. - Tom Siddell

User avatar
Boustrophedon
Dorkwood
Posts: 1513
Joined: Mon Nov 11, 2019 3:58 pm
Location: Lincolnshire Wolds

Re: Name that audio effect! (And maybe explain it)

Post by Boustrophedon » Sat Nov 21, 2020 11:10 am

basementer wrote:
Sat Nov 21, 2020 6:30 am

It's probably the least verifiable thing I'll ever post, but as I mentioned a few weeks ago, I have dreamed about SIGSALY.

I've got what looks to me like a better solution to the propagation delay than the one they used. They used two identical copies of the one time pad disc and built something elaborate to maintain a fixed offset between the two turntables. Why not use a single copy of the disc - synchronisation guaranteed - and a mechanical linkage to keep two play styli at a fixed angular separation? These people were clearly not stupid, I assume they were solving a problem that I haven't got as far as noticing.
But you need that "one time pad signal" to encode for transmit and to decode on receive, you can't transmit that signal alongside the encoded signal because then the man in the middle has your decode signal; so you have to deal with any transmission delays at the receiving end.
"Politicians should be replaced as frequently as diapers, usually for the same reason"
- Mark Twain

User avatar
basementer
Snowbonk
Posts: 587
Joined: Mon Nov 11, 2019 1:03 pm
Location: Aotearoa

Re: Name that audio effect! (And maybe explain it)

Post by basementer » Sat Nov 21, 2020 8:00 pm

Boustrophedon wrote:
Sat Nov 21, 2020 11:10 am
basementer wrote:
Sat Nov 21, 2020 6:30 am

It's probably the least verifiable thing I'll ever post, but as I mentioned a few weeks ago, I have dreamed about SIGSALY.

I've got what looks to me like a better solution to the propagation delay than the one they used. They used two identical copies of the one time pad disc and built something elaborate to maintain a fixed offset between the two turntables. Why not use a single copy of the disc - synchronisation guaranteed - and a mechanical linkage to keep two play styli at a fixed angular separation? These people were clearly not stupid, I assume they were solving a problem that I haven't got as far as noticing.
But you need that "one time pad signal" to encode for transmit and to decode on receive, you can't transmit that signal alongside the encoded signal because then the man in the middle has your decode signal; so you have to deal with any transmission delays at the receiving end.
Their solution used four copies of the pad in all, two at each end. My thought is that it was achievable with one copy at each end.
Money is just a substitute for luck anyway. - Tom Siddell

User avatar
Boustrophedon
Dorkwood
Posts: 1513
Joined: Mon Nov 11, 2019 3:58 pm
Location: Lincolnshire Wolds

Re: Name that audio effect! (And maybe explain it)

Post by Boustrophedon » Sat Nov 21, 2020 11:41 pm

basementer wrote:
Sat Nov 21, 2020 8:00 pm
Boustrophedon wrote:
Sat Nov 21, 2020 11:10 am
basementer wrote:
Sat Nov 21, 2020 6:30 am

It's probably the least verifiable thing I'll ever post, but as I mentioned a few weeks ago, I have dreamed about SIGSALY.

I've got what looks to me like a better solution to the propagation delay than the one they used. They used two identical copies of the one time pad disc and built something elaborate to maintain a fixed offset between the two turntables. Why not use a single copy of the disc - synchronisation guaranteed - and a mechanical linkage to keep two play styli at a fixed angular separation? These people were clearly not stupid, I assume they were solving a problem that I haven't got as far as noticing.
But you need that "one time pad signal" to encode for transmit and to decode on receive, you can't transmit that signal alongside the encoded signal because then the man in the middle has your decode signal; so you have to deal with any transmission delays at the receiving end.
Their solution used four copies of the pad in all, two at each end. My thought is that it was achievable with one copy at each end.
That's not how I understood it. As I read it there were two in use and two synchronised to take over when the first two finished.
"Politicians should be replaced as frequently as diapers, usually for the same reason"
- Mark Twain

User avatar
basementer
Snowbonk
Posts: 587
Joined: Mon Nov 11, 2019 1:03 pm
Location: Aotearoa

Re: Name that audio effect! (And maybe explain it)

Post by basementer » Sun Nov 22, 2020 6:20 am

Boustrophedon wrote:
Sat Nov 21, 2020 11:41 pm
basementer wrote:
Sat Nov 21, 2020 8:00 pm
Boustrophedon wrote:
Sat Nov 21, 2020 11:10 am


But you need that "one time pad signal" to encode for transmit and to decode on receive, you can't transmit that signal alongside the encoded signal because then the man in the middle has your decode signal; so you have to deal with any transmission delays at the receiving end.
Their solution used four copies of the pad in all, two at each end. My thought is that it was achievable with one copy at each end.
That's not how I understood it. As I read it there were two in use and two synchronised to take over when the first two finished.
PM and let's have a chat?
Money is just a substitute for luck anyway. - Tom Siddell

FlammableFlower
Catbabel
Posts: 792
Joined: Mon Nov 11, 2019 1:22 pm

Re: Name that audio effect! (And maybe explain it)

Post by FlammableFlower » Sun Nov 22, 2020 8:33 am

RobS wrote:
Sun Nov 01, 2020 4:06 pm
Bird on a Fire wrote:
Sun Nov 01, 2020 2:45 am
I can't quite work out why I'm still able to understand the words without what ought to be crucial bits of the soundscape going missing somehow.
That's a combination of very clear speech being highly redundant (for most speech sounds there are multiple cues to their identity )and that speech is a massively overlearned stimulus so the brain is amazing at inferring missing information from what it's currently hearing and what it's heard just before.

So there's the phonemic restoration effect, where a speech signal with small gaps in it can be unintelligible but if you fill the gaps with a sufficiently loud noise then the auditory system is often able to infer the missing information and it's as though you hear it. The examples at the end of the wiki are best listened to over headphones.

There's also sine-wave speech, which is a great example of how speech is highly redundant. Early speech synthesizers simulated the first three formants (resonances) of the vocal tract by filtering the output of a broadband buzzy sound (simulated glottal pulse) through three narrowband frequency filters whose frequency response could be changed to mimic the filtering of the vocal tract. Sine-wave speech dispense with the buzzy sound and just mimics the formants with frequency-modulated sine tones - this kind of speech is moderately intelligible with a little practice, but unlike real speech is highly susceptible to interference.
This is very much describing my hearing problems. I frequently don't actually hear everything said. Apparently I'm essentially deaf to certain frequencies within tbe range of speech. Not much, but enough that I don't always hear everything said and my brain fills in the gaps. Generally everything's fine, but every so often I give very odd responses in conversation or to questions. It's less of a problem if I can see the person's lips as I'll get extra cues there (I can't actually lip read). It does drive MrsFF a bit potty, especially as it can't be easily remedied e.g. with hearing aids. Outdoors, if someone is facing away from me, there's not much hope of me catching much said.

User avatar
jaap
Fuzzable
Posts: 240
Joined: Mon Nov 11, 2019 2:05 pm
Location: Netherlands
Contact:

Re: Name that audio effect! (And maybe explain it)

Post by jaap » Sun Nov 22, 2020 10:22 am

FlammableFlower wrote:
Sun Nov 22, 2020 8:33 am
This is very much describing my hearing problems. I frequently don't actually hear everything said. Apparently I'm essentially deaf to certain frequencies within tbe range of speech. Not much, but enough that I don't always hear everything said and my brain fills in the gaps. Generally everything's fine, but every so often I give very odd responses in conversation or to questions. It's less of a problem if I can see the person's lips as I'll get extra cues there (I can't actually lip read). It does drive MrsFF a bit potty, especially as it can't be easily remedied e.g. with hearing aids. Outdoors, if someone is facing away from me, there's not much hope of me catching much said.
I know someone with a cochlear implant, who due to a genetic disease would otherwise have gone completely deaf, and what you describe is exactly what happens to him quite regularly. Nevertheless it is amazing just how well it works.

Post Reply