What audio plug-ins actually do
Love Music More · hosted by Luke F. Walton · Solo episode
The podcast feed is titled Love Music More with Scoobert Doobert. Scoobert Doobert is Luke F. Walton's primary music project.
Listen
The Signal-Shifting Shape of Plug-ins
Topics discussed
Host note
I don't think music is math. I think recording is. The music is magic, and the math is how that magic gets into a speaker and stays there. I start with a sine wave. Add another note and the wave gets less smooth, more rub, more dissonance. I can change the wave shape on a synth without changing the note on the piano. Lay two signals together and the sum is a new wave. From two there is one. Harmonics are why that synth does not sound like a guitar, a violin, or an oboe. The ear hears the fingerprint, the microphone does too, and the speaker has to build it again.
A computer cannot store a continuous wave. Cut the gap between two steps in half forever and you never arrive, so digital audio samples the wave. It is an approximation, the way a movie is frames. Sample rate and bit depth are the longer version of that, on What is sample rate and bit depth?. Once the wave is numbers, convolution is available. An impulse response is a clap with the clap removed: an elevator, a hallway, a cathedral, outside. Apply it to a dry synth and the synth can sit in the Taj Mahal, or inside the body of an acoustic guitar. A voice convolved with a synth is the start of a vocoder, and more than that is going on. Spring, chamber, and plate are on The Types of Reverb. Convolution is the one that uses the clap. That clap and its decay are the time domain. The frequency version is a Fourier transform. Run a pure sine through a preamp and the preamp leaves its character on the wave. Flip the phase and two perfectly opposed signals cancel. That is how noise-cancelling headphones work, and it is the same cancellation I use for width on Panning, Psychedelic Effects, and Headphone Vectors. An airplane hum is the easy case. A scream is sharp, with a lot of attack, and it comes through. Miss the flip and the math can make the noise louder.
The same subtraction is how a plug-in learns a piece of gear. Run a sine through a circuit, including something like the Beatles' Abbey Road recording desk, subtract the sine, and keep the difference. A dying battery on a fuzz pedal is one of those differences, a nonlinear voltage, and the supply on my pedalboard has a setting that fakes the sag. A plug-in is a person who did that math and sells it inside a DAW. Sausage Fattener puts an angry mascot on a clipping circuit. Universal Audio goes into studios and tries to capture a compressor or an EQ I cannot afford, the kind of control people call more musical. Around 2012 the computers got fast enough to hear it. Early digital wanted to sound clean and leave tape behind. I put fake tape on the inserts so the music still sounds like my music, including on a plane. The longer case for that color is Analog vs. Digital.
A mix for someone who wants more sauce can stack hundreds of these moves, in frequency, in time, and in amplitude. This is the first computer I have owned that almost keeps up. Recording stays a representation of the music, not the music. The microphone is a kind of ear. The cable, the preamp, and the processing are further choices, and a choice can beat the source. If I look at those choices the way I look at a synthesizer, the waveform, how loud it is, how hard it clips, how it attacks and decays, I stop freezing. There are infinite right answers. The set of moves is small.
Selected moments
- Recording is the math 0:59 I refuse the line that music is math. The math is the path from the thing I play to a speaker that can keep it.
- From two, one wave 3:09 I add two signals and the sum is a new wave. From two there is one.
- You never arrive, so you sample 4:13 I cut the gap between two steps in half forever, then treat digital audio as samples, the way a movie is frames.
- A clap with the clap removed 5:26 I use an impulse response: elevator, hallway, cathedral, outside, then the room without the clap.
- A synth inside a guitar 6:40 I put a dry synth through the body of an acoustic guitar, and I treat a voice convolved with a synth as the start of a vocoder.
- Flip the phase 8:28 I move from the clap's decay to a Fourier transform, then to phase. An airplane hum cancels. A scream comes through.
- The mascot is a clipping circuit 12:47 I strip Sausage Fattener down to the math. The angry face is a clipping circuit, and the price is that math inside a DAW.
- A small set of moves 17:29 I treat a recording choice like a synthesizer. Infinite right answers, and a small set of techniques.
Selected excerpts
I don't want to convince you that music is math because I don't really actually believe that too much. But I do think that recording is math.
I think the music is magic. But you need a little bit of math to put the magic onto Spotify.
So you have two parts, this, you play them together, now it's a new wave. From two there is one.
If you clap in an elevator, then you clap in a hallway, then you clap in a cathedral, then you clap outside.
So now my synthesizer is running through an acoustic guitar as a reverb chamber.
But a sudden screaming baby that's very sharp, very transient, has a lot of attack that can poke through suddenly through your noise cancellation
There's this weird little mascot that got like angrier and angrier as he turned it up, but like what's really going on.
You pay them for that math and the simplicity of having it in your DAW.
FAQ
Is music math?
I don't think so. The music is the magic. Recording is the math, the work of taking a sound off an instrument and putting a copy of it in a speaker. You need a little of that math to get the magic onto Spotify.
What is an impulse response?
It is what a room does to a clap after you take the clap away. An elevator, a hallway, a cathedral, and the outdoors each leave a different decay. Convolution applies that decay to a dry sound, so a synth can sit in the Taj Mahal or inside an acoustic guitar.
What does an audio plug-in actually do?
Somebody worked out the math and applied it to the signal. The picture on the screen can be a joke. Sausage Fattener's mascot gets angrier as you turn it up, and under that is a clipping circuit. You pay for the math and for having it in the DAW. Universal Audio's plug-ins are an attempt to capture a compressor or an EQ I cannot afford.
Why do noise-cancelling headphones miss a sudden sound?
They play the flipped phase of a steady outside sound, and two perfectly flipped signals cancel. An airplane hum is the easy case. A scream is sharp, with a lot of attack, and it pokes through. If the flip is wrong, the math can make the noise louder instead.
Transcript
0:53 What's good everybody? Welcome back to Love Music More. I am Scoobert Doobert today. I don't want to convince you that music is math because I don't really actually believe that too much. But I do think that recording is math. I do think that the math of taking a sound, pulling it from the stuff you're playing, and putting it into your speakers for people to hear it forever and ever and ever, that is definitely math.
1:21 I think the music is magic. But you need a little bit of math to put the magic onto Spotify. So what I want to talk to you about today is how we actually take these synthesizers, change them, the math that we have behind these very simple sounds, and then how we can put them on top of each other through a thing called convolution. So let's start with a sine wave.
1:49 Just a simple frequency that's going up and down, this is more or less a sine wave. From there you can do all sorts of cool stuff with it, like you can add another note. So when you actually look at the waveform, like picture rolling waves, if you've seen it in math class, that up and down smooth or waves in the ocean.
2:11 When you have the one sound, it's very smooth when you add in another one, it gets a little less smooth. The more clustered and crazy it gets, you're able to get more rub, more dissonance, and the waveform, when you're looking at it visually and when you're looking at it mathematically, gets a little bit more jagged. You can also control the waveform of an individual sound.
2:40 So say for example, we've got this, I'm not changing the note that I'm playing on the piano, but I am playing with the wave shape on my synthesizer. So that's one thing that we can do. We can lay audio on top of each other that takes on some sort of summation, some sort of addition of the different component parts. They become one new part. So you have two parts, this, you play them together, now it's a new wave.
3:14 From two there is one. One basic idea of what we can do on synthesizers is we can change the way that these waves are shaped. We can change the way that these waves have harmonics, like the things that kind of emerge from them. It's the reason why this synthesizer sounds different than my guitar. That sounds different from a violin, sounds different from an oboe.
3:36 They all have different kind of fingerprints, timbral structures, different notes that can emerge above or below the note that you're actually playing. Then your ear picks up all of this as does your microphone, and then it gets replicated back out through your speakers.
3:51 This is the way that we are converting this analog signal, the thing that I'm calling music is magic, into recording is math, because it needs to be replicated. With digital audio especially, you're kind of taking a sample of it. It is not a perfectly continuous thing because it would blow up your computer because it would basically be infinite. What is the distance between two steps?
4:15 It's like, well, you can cut that in half, but you can cut that in half. You can cut that in half again, and suddenly you find out you're never able to walk anywhere because everything is too far because there's infinite space in between every step, which is stupid. But that is the way that your computer can think about the waveforms if you are not doing something called sampling in between each of those.
4:35 It is an approximation of what the audio actually is in the same way that a movie. It's a series of moving pictures, just individual frames, but your brain is able to reconstruct that and see a movie. You don't even start, you know, you don't think, oh yeah, the frame rate of this, unless you're like a real movie buff, you just watch a movie.
4:57 But once you go from the world of continuous waves, you start to break them down, you turn them into math, you can start to do some crazy stuff just like I was doing, fiddling with the wave shape. But you can also create artificial reverbs. So there's this process called convolution, and that's a fancy word, but one of the cool ways that you can convolve something is you can use something called an impulse response.
5:23 So an impulse response is like picture a clap in a room. If you clap in an elevator, then you clap in a hallway, then you clap in a cathedral, then you clap outside. The echoes, the kind of energy of those delays and the decay of your clap kind of give the sense of that space.
5:42 Now if you subtract out the clap, all you have is the space, and you could apply that same kind of mathematical formula to any, you know, just plain old dry wave form. So you can take this synthesis and you can add in like a whole bunch of reverb or you can put it into the Taj Mahal.
6:06 Of course, this isn't perfect, and it is all a representation, so you're stacking one representation onto another, but because you've kind of gone from the world of analog and infinite into the world of mathematics and recording and digital, you can have so much more control.
6:23 And you can start to ask other questions of, wait, if I'm able to take my dry audio and put it in to the Sistine Chapel, what if I took my dry audio and instead put it inside of the body of an acoustic guitar? So now my synthesizer is running through an acoustic guitar as a reverb chamber. What if you think even weirder?
6:46 What if I took my voice and then I used that to convolve a synthesizer or vice versa and you get a vocoder? That's a little, you know, a little bit more stuff is going on, but that's the basic concept of you can put one things like a room or essential like audio, the qualities of one form of audio and you can tuck it into another.
7:15 The same way that you can have these two notes playing together, you can also take those two notes and put them in a room or put them into the tambourine or like audio spectrum. Like if you've ever seen a spectrograph, you can take some of that and give some of that quality through convolution from one into the other. Essentially you're transforming audio, but I want to be more specific.
7:36 So you're transforming audio with convolution in the time domain. So like when I'm saying I clap and then boom, it explodes into the reverb or I play my synth and it goes into the chambers of the acoustic guitar. That is the time. So something happens. There's an impulse. There's the clap and then it decays or it has a series of delays or whatever.
8:03 You can do the same thing to the actual frequencies because like what is music other than frequencies, time and amplitude, how intense are those frequencies? And also, you know, everything adds up. So you have a pure sine wave, but even if you record that pure sine wave into a really cool preamp, that preamp or the thing that is amplifying the sine wave can impart some of its character.
8:28 So now we're in the world of the Fourier transform. Don't have to remember all these vocab words, but basically the same idea of convolution like an impulse response, sticking something into a fake room or sticking something into another thing that shouldn't be there. You can do the same thing with frequency transformations. So you can transform things like the phase.
8:49 So phase is if you picture the sine wave going up and down, it goes up and then it goes down. You can flip that. You can invert it. I've talked about this a lot on the podcast. And that's how you get noise canceling headphones is you have the external space that's red with a very fast microphone.
9:08 And then it plays the opposite of that external space and it cancels out that sound because if you have two things that are perfectly flipped, this is part of the reason why noise cancellation can have trouble with certain environments and be great at others, like an airplane, a static airplane, airplane sound is kind of the ideal, the platonic ideal of dealing with external audio.
9:32 Phase flipping the hum of the engine. But a sudden screaming baby that's very sharp, very transient, has a lot of attack that can poke through suddenly through your noise cancellation in the same way that the math can go wrong or it can accidentally amplify something because it didn't flip the phase. But these are, you know, controls that we have.
9:55 And it's kind of fun that recording really does distill down to these really small things. It's like we can deal with up and down in terms of the frequency, we can deal with the time and the decay and like what's going on when that frequency happens and when that frequency stops. And we can deal with transforming that frequency.
10:16 So intensity, the amplitude, the frequency, the actual notes that we're playing and then the time domain, like every frequency is happening on a timeline, otherwise it's not music. So how do we actually do that? I don't want to get super heavy into the math, but there are incredible ways to transform the frequencies.
10:39 So you can make it so that it sounds like you're recording through, say, the Beatles, um, Abbey Road, uh, recording desk. It's not perfect. It's an approximate approximation, but you can take each individual circuit. You can take each, um, there are either, um, Schematics that you can read or you can actually run just like we're talking about like a clap. You can run a sine wave.
11:05 You have the perfect sine wave on the front end. You run it through, uh, given piece of gear and you check and see what changed mathematically. You just subtract what you started with and then you see what are the deltas and you can apply those deltas to other things.
11:21 You can make it more intense too, as you can track that over time because say, for example, uh, guitar pedals, one of the things that can be really cool about guitar pedals is that they can have non, um, linear, uh, voltages, voltages. So a guitar pedal, one of the reasons why people, um, like to use batteries is sometimes they can get something special right as that battery is dying.
11:50 Sounds crazy, but it's very true, especially with distortion and fuzz pedals. Um, and so my power supply that I have on my pedal board actually has like a guitar pedal setting that acts as though it's giving the non linear voltage that a dying battery would to a given guitar pedal. So you can get that all the time again, an approximation, but you can play with that in the recording space too.
12:16 These are all just kind of tools in the toolkit. And I bring this up because this is what's going on with plugins. This is what's going on with guitar pedals, especially, um, the digital ones is somebody is sitting down and figuring out the math and applying that math to the given signal. And it is, I think really powerful when you kind of strip it down to that.
12:37 This is a lot of things that try and make you kind of get seduced by the plugin, like a beautiful graphic user interface or something that's like a really funny, um, there's like this one called sausage fattener. There's this weird little mascot that got like angrier and angrier as he turned it up, but like what's really going on. It's going into like a clipping circuit.
13:00 It's going in through some distortion, basically the people that created that plug and made a bunch of mathematical decisions that affect your source audio and you pay them for that math and the simplicity of having it in your DAW. And same thing with, um, like I really like universal audio. I use a bunch of their stuff.
13:20 They go into all these recording studios and try their best to grab a real representation or as close of a representation as possible to enter, uh, something that I cannot afford, like a great compressor or great EQ that has what somebody would call like more musical controls.
13:39 It sounds less digital because they're essentially taking all of that time domain stuff, the convolution and then the frequency domain stuff, the Fourier transform and doing their best to make it very efficient and clean and then apply it to your audio signal. So you can take that wherever you go. It's also very convenient and there's a kind of funny thing.
14:01 I think that happened right around 2010 ish, maybe a little bit later, like 2012, 2013, you can really start to hear the computers get good enough at this math. So one of the challenges with digital, and I also think like you launch a new technology and people try to go really far on that technology. So like digital comes out, people wanted to be very clean.
14:25 And, uh, also the computers, uh, were much better at just replicating a source, very sterile kind of way. And a lot of people like that because they, they're getting away from tape. They want a more modern sound. Now we don't want that at all. We want a tape sound. And so I put a bunch of fake tape and all sorts of things onto my, um, you know, my inserts to make my music sound like my music.
14:48 And also I can't afford all the tape machine. Also, I want to be able to make music on an airplane or on a train or touring or whatever, whatever I'm doing. I want to be able to make music. And so I don't want to be held up where it's like, oh, I'm not in the perfect studio environment. So I can't make music.
15:01 It is, uh, it is great to be able to have access to these tools and a computer that's strong enough to be able to run them. But one of the cool things about these, um, these mathematicians is they figured out ways to make them faster and faster and faster and put them onto different parts of your computer so that it can run all of this stuff at one time.
15:22 I mean, you should see some of my mixing sessions, especially for artists that asked me to kind of put a little more sauce on it. Uh, there can be hundreds of different combinations of exactly what I'm talking about, frequency domain, time domain, and amplitude stuff going on all over the place in some crazy, crazy ways. And it's kind of a miracle that my computer can keep up.
15:46 This is the first computer that I've ever had that could actually keep up with me and it doesn't always, but it's close. But anyway, credit to the engineers. It's a really cool thing.
15:58 And I also think it gives us more, I don't know, ability to interact or evaluate or even build when we recognize that the recording process is distinct from the music making process in that the recording is a representation of the music that we're playing, not a perfect, um, not a perfect one.
16:19 And every choice you have across that recording process is another choice of representation, which can be used to be way cooler than the source audio, the microphone that you choose.
16:29 Thinking the microphone like a bunch of different types of ears, the cabling that the microphone uses, the preamp that you choose, all of the digital signal processing that you choose, those are all decisions with huge, huge, uh, impacts into the final state in the same way that transforming this to this, that's all, that's all basically what we're doing, we're affecting the waveform, we're affecting the
17:01 transformation of that waveform, how loud it is, how clipped it is, uh, how it attacks, decays, sustains, releases. If you can think of almost every musical decision you make in the recording process, almost through the lens of a synthesizer, I think you're going to make better decisions without being paralyzed. You know, there are, there are infinite right answers.
17:23 So I think it simplifies it, at least for me to be like, okay, when I'm recording, I'm basically taking my source audio, turning it into math, manipulating it in a very small set of ways, through a bunch of different techniques, plugins, and engineers' works. If you enjoyed this, I'd like to invite you to like and subscribe, tell your friends, you can check out my music at scoobertdoobert.pizza.
17:47 And I've also got some crazy stuff on there. I made a video here, kind of thing. So go check out scoobertdoobert.pizza. And we've got some more, um, awesome guests coming on next, next week. Pretty much here every week. And I don't know what else to say. Go learn math, I guess.
The transcript above is a reviewed transcript of this episode (JSON). The recording is the Spotify episode.