Oversampling Data (Explained with Audio) - Computerphile

0:00 Right, so in the last episode, we discussed saturation.

0:05 Which creates harmonics, which are higher frequencies from a low harmonic,

0:09 which is all very well and good.

0:11 Let's say you've got a 1,000 hertz, a 1K pure sine wave,

0:15 and you create a harmonic at two and four and 8,000 hertz or whatever.

0:19 That's all nice and good.

0:20 We can do that nice and easily within the digital realm.

0:24 But what happens when you've got a 20,000 hertz signal and you want

0:28 to create a second harmonic of that, and your sample rate is say 48K,

0:34 which means because of Nyquist theory,

0:36 24K is the maximum sample rate, which you can't do it.

0:40 We did do a video about that.

0:41 You'd think that's enough different levels,

0:44 but realistically it's kind of half that.

0:47 Effectively, just as a recap,

0:50 the sample rate is the number of times per second that the measuring device,

0:56 the analog-to-digital converter,

0:58 stabs in and says, "What's the voltage of the signal?"

1:00 And it will stab in so many times per second and say,

1:03 "This is the voltage of the signal."

1:05 And Nyquist-Shannon theory says that the uh the maximum

1:11 frequency that you can sample has to be half of the rate that you sample at.

1:16 So, you need at least two points to be able to create a sine wave.

1:21 And if you have at least two points,

1:23 you can always recreate the same sine wave back, which is fantastic.

1:28 But it produces problems when you're doing what we did in the last video,

1:32 which is saturation.

1:33 And to cure that, you have to do something called oversampling.

1:37 And again, this can apply to all

1:38 sorts of digital signal processing, not just audio,

1:41 but I'm going to give it to you in an audio context, cuz that's what I know.

1:46 So, let's start by drawing something very badly in the tradition

1:50 of all geeky people that come on Computerphile and can't use a pen.

1:53 So, there is a sine wave.

1:55 Let's say this is a block of time.

1:57 Doesn't matter what the block of time is.

1:58 So, we've got 1 2 3 4.

2:00 We've done four full cycles of that.

2:03 But, if our sample rate only looked there and only looked there,

2:09 when we were trying to reconstruct that wave, how would we get that wave back?

2:14 So, basically, what you're saying here is that sine wave is a noise or whatever.

2:18 Yeah, yeah, it's a pitch.

2:19 It's a sine wave.

2:19 So, these bits where you've put crosses are where

2:22 we're sample where we're That's where we're sampling the voltage.

2:24 You've only got those two points,

2:25 so it's going to be one big wave between those two points.

2:27 So, it's it's going to turn out sort of looking something like that, okay?

2:31 Which means we've lost all of that data.

2:33 But, also, it's going to sound a lot different.

2:35 It's going to It's going to sound lower in pitch.

2:37 So, your which I can't whistle, will end up being woo, will being lower.

2:42 Okay, which you really don't want that, do you?

2:44 And this is the problem is that well, you could say, "Well,

2:47 we're putting another sample point there." But, then that's going to produce

2:50 a sine wave that's like that that's going to go through that.

2:54 I'm not drawing this very well.

2:56 So, you'll have to forgive me on that, but regardless,

2:59 if you try and put frequencies in that are

3:02 way too high for the sample rate you're using,

3:05 you're going to end up with lower frequencies than what you want.

3:09 And the the way it works like the way actually works is that the more you go

3:15 over you basically measure the amount you go over

3:19 and then you deduct that from your highest frequency,

3:21 and that's the frequency it's going to produce.

3:22 So, for example, if your sample rate is 10,000 Hz, 10,000 times per second so,

3:29 your maximum frequency that you can produce is 5,000 Hz.

3:34 If you pass it a 6,000 Hz signal, it's 1,000 Hz above the limit so,

3:39 that means it's going to produce a frequency at 4,000 Hz.

3:42 So, effectively, we've got a ceiling okay, of 5K, 5,000 Hz and we've got 0 Hz,

3:49 and we can put in any frequency between 0

3:52 and 5,000 Hz in our 10,000 Hz sample rate.

3:56 And then the moment it gets to the top,

3:57 instead of continuing and we get 6K above there,

4:00 it will actually flip back down and mirror

4:03 back down to zero and then mirror back up.

4:05 So, by the time you put 10,000 Hz in, it will be back down to zero.

4:09 By the time you put 15,000 Hz in, it will be back up to to 5,000 Hz.

4:14 And it might, every now and again on the odd frequency, work musically.

4:17 But, trust me, if you're putting a 6,000 Hz signal in you're getting 4,000 back,

4:21 that's going to be inharmonic and it's going to sound awful.

4:25 So, that could be just part of the sound you're recording.

4:28 Like, I don't know one of the frequencies the guitar's making Exactly.

4:31 So, so if you had a a microphone that was over the top

4:34 of the drum kit and you've got symbol noise or something like that.

4:39 Or, let's say you've got an analog measurement thing

4:42 that's that's measuring the the moisture in the soil.

4:46 Okay?

4:47 And that is producing higher frequency voltage readings

4:52 than your sampler that's then recording what they

4:55 are and putting them in the spreadsheet for you to make your your wave out of.

4:58 If those things are mismatched,

5:00 if you've got higher frequency going in than the sample rate can take,

5:04 you're going to end up with Incorrect samples.

5:06 Incorrect samples and you're going to end up with a sort

5:09 of anomalies in your the frequencies that you actually care about.

5:13 When you go above your frequency range and it reflects back down into something

5:18 that you don't like the sound of, then

5:20 that is called aliasing and aliasing sounds horrible.

5:24 So, in the same way that we have with photos, well, that's exactly the same,

5:27 but there are aliasing filters,

5:29 anti-aliasing techniques and that's one thing I'm going to show you now.

5:32 I'm going to load up Plugin Doctor to show you it in in theory.

5:36 This is where we had we had this nice little harmonic pattern in the last video.

5:40 So, I all I did was just do input times input.

5:44 So, effectively I was squaring the input to get

5:46 a pattern of harmonics and then I was multiplying it

5:49 by the sign just so that it wasn't making all

5:51 the values positive cuz squaring a number automatically makes it positive.

5:54 So, we were wave shaping in an interesting way

5:57 and our fundamental frequency here that I got was was 50 hertz.

6:01 If I just clamp that to 50 hertz,

6:02 you can see So, if I got a 50 hertz signal here,

6:05 then our third harmonic of that is going

6:08 to be 150 hertz cuz it's three times, okay?

6:10 And our fifth harmonic of that's going to be 250 hertz and etc.

6:14 So, we can see this nice little pattern.

6:16 But, if I turn up this frequency,

6:18 this pattern of harmonics here can't just keep going on above forever.

6:24 You see what's happening the higher up we get,

6:26 we're getting all these frequencies that are being

6:28 mirrored down which are completely in harmonic to what's

6:33 going on and you see them traveling around

6:35 and you see as I push the fundamental frequency up,

6:38 you can see this second harmonic third harmonic here

6:42 the the nearest one to it is then traveling down.

6:44 It's traveling in the opposite direction.

6:46 So, it looks messy.

6:47 Would we normally see just one normal spike?

6:49 Well, really what you want to do is you want all of those harmonics

6:52 to keep going up there and nothing

6:53 to be happening below the fundamental frequency.

6:56 But, what's happening is is as I showed in that diagram,

7:00 as those harmonics are being created outside of the frequency spectrum,

7:03 they're getting mirrored back down

7:05 into the lower thing which when you're dealing

7:07 with music just creates things that just sound out of tune and sound horrible.

7:11 It can do.

7:12 I mean, sometimes it's a desirable effect on a keyboard

7:15 or on a or on a drum kit or something like that.

7:18 But, most of the time it's not something that you'd want

7:21 particularly if you're getting those harmonics

7:23 into something that's actually audible.

7:25 If they're sitting so quiet then, you know, you don't care about it.

7:28 But, um the other interesting thing is is

7:32 you can see as I move this frequency about,

7:36 the higher it goes then those get lower.

7:38 Let's say I had a keyboard or a guitar where I

7:41 was going nee nee nee nee nee nee nee nee nee

7:42 nee nee nee nee nee nee nee nee nee and I

7:43 was putting some modulations so the pitch was just slightly bending.

7:46 Your frequencies and all your harmonics will all be moving together like

7:50 that, but the mirrored ones that are coming

7:51 down will be going in the opposite direction.

7:53 So, they'll be going down and then back up.

7:56 Let me show it to you in another way

7:57 as well because you can actually present it on a sweep,

8:00 okay, in this software which is which is lovely.

8:02 I can actually have it as a 2D plot and I'm going

8:04 to do it linear and I'm going to psych out some of the noise.

8:07 But, you can actually see here.

8:09 So, that says reflections.

8:11 That's reflection.

8:12 when they put laser into a mirror and it bounces Exactly.

8:14 So, what what's happening there is you can see

8:16 there's our fundamental that's going up nice and linear.

8:20 This is our third harmonic which is going up and it's hitting

8:23 the top and then it's coming back down and then it's going back up.

8:27 And if you were to listen to that, you'd hear

8:29 at the beginning the sound go if I generate a sweep here.

8:37 And if I put not my dodgy plug-in That extra little bit that just disappeared.

8:45 You know the the little woo woo woo Turn it down so we can talk.

8:51 But, what you'll hear is this like this chirping at the top end.

8:54 So, it's woo woo Yeah, like a chattery tone.

8:58 And if I turn up the digital, should I really go for like a heavy saturation?

9:05 If I now turn on over sampling Oh, so it clips all that out because it's not

9:13 You're literally listening there

9:15 to these frequencies bouncing off and interacting

9:18 and they're all added into each other and sounding awful, right?

9:21 And how do we get rid of that?

9:23 Well, let's go back to the paper for a minute cuz I can't code that easily.

9:26 It's not something I'm going to do,

9:27 but I'm going to explain theoretically one way that like

9:31 everything there's always 100 ways to do things, right?

9:33 And all I'm going to show you is one way,

9:35 probably what I would consider the most simple way.

9:39 And the most simple way of doing it is

9:40 let's say you've got a very badly drawn wave, right?

9:45 And we've got data here that's like sampling at a regular interval.

9:51 So, these are our data points.

9:55 If we want to double that, you might think, "Well,

9:57 actually, what we'll do is we'll kind of in the background,

10:00 we'll draw the wave and and we'll just interpolate and add more data points

10:04 in." But, the easiest way to do that is to say if this is our zero line,

10:08 I'm just going to put in some in between points all at zero.

10:13 So, we end up with a wave that kind of goes

10:15 to zero and then up to there and then back down to zero,

10:17 up to there, uh you know, to there, back down to zero, up to there.

10:21 And it it looks much crazier than the original wave.

10:25 That is the worst drawing I've ever done,

10:27 but basically, what we're doing is we're zero padding.

10:29 Do a nice animation, Shawn, where you've got a wave and then each

10:34 of the points in between are basically going to zero.

10:37 What you end up with is is let's let's change

10:40 the other way a bit like the the frequency domain.

10:43 If you had a signal which looked a bit like this, let's say,

10:47 on a spectrum analyzer.

10:48 So, we've got a load of low frequencies and then we got

10:52 a little bit of mid and then we got a load of high frequencies.

10:54 So, that's what it would look like on a on a spectrum analyzer.

10:57 If you put zeros between each one,

11:00 so you effectively double the number of samples,

11:02 you actually then if you analyze the frequency spectrum,

11:04 cuz we doubled the sample rate at that point,

11:07 you actually get a mirror the other way of that.

11:09 We would get another load of frequencies that would be

11:12 that mirrored effect that we're seeing there coming up the other way.

11:16 Now, how do we then get rid of all of that?

11:18 Well, that's simple, you know, we don't care about any of the frequencies here.

11:22 So, we just use an EQ to do low-pass and get rid of all

11:26 of that, and then we are left

11:27 with our original signal but at double the sample rate.

11:31 So, here again, I'm using Saturn 2 rather

11:33 than my dodgy code to produce some harmonics.

11:36 And you can see that there is is clearly producing

11:39 harmonics that mirror down and will give you that chirping sound.

11:43 But you can see how very quickly,

11:44 if I switch on their oversampling, so the good level oversampling,

11:49 you can see how most of that goes and you're seeing very very little.

11:54 And if I put it up to superb quality, even more goes.

11:58 So, if I want to push that even further,

12:01 you can see I'm still not getting any of that at all.

12:04 But if I turn it down to just good quality,

12:06 you'll probably get a little bit of reflection back.

12:09 And then if I turn that off completely,

12:11 you'll get all the mirroring and all the aliasing problems.

12:14 So, let's actually do this with some numbers.

12:16 So, let's say I'm going to have 0.8 as my first number.

12:19 So, that's going to be a point on the sine wave there.

12:22 And then I'm going to do -0.8.

12:25 So, that might be down there.

12:26 And then I'm going to 0.8 again.

12:29 So, we're basically creating some data points.

12:32 So, I've got an array of four numbers

12:34 and I want to turn that into eight numbers.

12:36 So, effectively, I'm doubling the sample rate.

12:38 I'm doubling the amount of data.

12:40 If we put a zero between each of those and zero in the end,

12:44 we've got eight numbers.

12:46 Okay?

12:47 We've got eight numbers.

12:48 Now, that will then look and sound dodgy.

12:52 Okay?

12:52 Because it will be doing the equivalent of what we're seeing here,

12:55 where each inserting all of those zeros will make every

12:58 frequency also create a harmonic and mirror in the upper frequencies.

13:02 So, if this is your frequency, okay?

13:06 And this is your amplitude.

13:09 And we've got some stuff sitting in the low end and we got some stuff sitting

13:12 in the middle end and we got some

13:13 stuff sitting in the high end, which is louder.

13:15 And we double the sample rate,

13:18 that high frequency is then going to end up in the lower part

13:21 of this and the mid will end up even higher and the low will end up even higher.

13:25 So, you'll end up with a with a mirror.

13:27 So, put my hand there.

13:28 You double the amount of samples,

13:31 put all the zeros in, and you end up with a mirror of the frequencies,

13:37 but just mirrored the other way in the same way that you see that.

13:40 But, obviously with the magic of EQ,

13:42 we can just come in and chop all of that out.

13:47 And then all of these zeros, if we run it through like in the first video,

13:50 remember I had that that low-pass filter and it was just a moving average?

13:55 But, if you run the moving average over that and you

13:57 were looking at averaging those two and then averaging

13:59 those two and averaging those and you do that, you

14:01 can see those zeros will stop being zeros very, very quickly.

14:04 And you end up back with a smooth curve,

14:06 but with double the amount of double the amount of points.

14:09 And that's just a really important thing

14:11 to do in audio and probably many other disciplines.

14:26 That's pretty cool, isn't it?

14:27 Being able to play with audio like that, literally in a plug-in,

14:30 just start writing C.

14:31 So, um I wanted to introduce you to that because I