DeepSeek V4 AI Beats Billion Dollar Systems…For Free

0:00 Finally, DeepSeek 4 is here, and it is described in a 58-page research paper.

0:06 And finally, nothing is held back here.

0:09 I’ll be honest I am feeling a little shy today,

0:13 so I will do classic Two Minute Papers, mikrophone, but no camera.

0:18 This is one of the biggest open and free

0:20 AI models that we can use and…excuse me?

0:24 Do you see that?

0:26 What?

0:27 A 1 million token context window?

0:30 In open weights AI?

0:32 If you ask it to inhale about 1,500 pages of dense documentation it will do it.

0:40 But that was the main feature in Google’s Gemini not so long ago.

0:45 I remember flipping out about it 2 years ago.

0:49 And now, this for free?

0:51 This sounds absurd!

0:53 And when I look at the Pro model, you’ve got to be kidding me.

0:57 Its results roughly match the many billion dollar

1:00 frontier models from just a few months ago.

1:04 Now it is gifted to us mortals.

1:06 I am trying to emphasize the kind of gift

1:09 that we are getting here and my words fail me.

1:13 Is this heaven?

1:14 What a time to be alive!

1:18 And…wait.

1:18 There is a Flash model that is much smaller,

1:22 and is somewhat competitive with the Pro?

1:24 I mean, what is happening?

1:27 And it doesn’t end there.

1:28 This is just the start!

1:30 As it keeps outputting more and more text, the new Pro model requires about 3

1:35 times less computing power than the previous one,

1:38 and the lighter, Flash model requires about 10 times less computing power.

1:44 What am I even reading?

1:46 How is that even possible?

1:48 Dear Fellow Scholars, this is Two Minute Papers with Dr.

1:52 Károly Zsolnai-Fehér!

1:53 Well, it does three things that are absolutely magical.

1:57 One: Compression.

1:58 Namely, compression for the KV cache- this is a scratch

2:02 pad where you write your prompts and add your documents.

2:05 Imagine reading a book.

2:07 You can find answers so much quicker if

2:10 you compress each paragraph down into one sentence.

2:14 You keep the book.

2:15 But now you can search it faster.

2:18 They call it token-level compression.

2:20 But even these little summaries add up.

2:23 What do we do?

2:24 Well, two.

2:25 We want to know the overall plot of the James Bond book?

2:28 See if it’s one that we read already?

2:31 Well, of course, we look at the table of contents.

2:35 If each chapter has a short name,

2:37 we can grasp the whole story from that tiny piece of information.

2:42 The paper describes it as a 128-to-1 compression.

2:46 They call it Heavily Compressed Attention.

2:48 Now, the AI sees the whole story at a glance.

2:53 But scientists at DeepSeek say this is still not enough compression.

2:59 We need more!

3:00 Three.

3:01 Imagine that we want to search for a fight in the book.

3:04 Table of contents helps a bit, but may not tell us exactly where the fights are.

3:10 So, we look at an index.

3:12 A list of words and phrases and their locations.

3:16 Okay, so looking for a fight, and bingo!

3:19 The index gives us the top 5 pages that have fights in them.

3:25 This is genius, and they call it Compressed Sparse Attention.

3:29 So, three layers of compression: summaries, structure, index.

3:34 And suddenly, the three pieces click together.

3:38 These three reduce memory needs for the KV-cache by about 90%.

3:43 I had to look twice.

3:45 Down about 90%.

3:47 Squashing down a 100 words into a storage space of 10?

3:52 And you are saying that we are not losing basically every piece of information?

3:57 Yes.

3:58 That is exactly what they are saying.

4:00 But we are Fellow Scholars here, we look at proofs and experiments.

4:05 Now just to make sure, this is KV-cache compression.

4:09 You still need to load the whole model.

4:11 So it does not mean that you can load the full DeepSeek Pro AI onto a toaster.

4:17 Just want to make sure you know that because media headlines and hype…you know.

4:22 And now…hold on to your papers Fellow Scholars, because this one delivers.

4:27 They tested it by hiding 8 facts inside increasingly long contexts.

4:32 So how good is it?

4:34 Well, they report that the Pro version recalls it better than Gemini 3.1 Pro.

4:40 That is Google’s flagship product.

4:44 Wow.

4:45 That is unbelievable.

4:47 But note that like many other systems,

4:49 it starts to degrade as you start approaching the limits of the context window.

4:55 Then, models forget.

4:56 Drift.

4:57 Hallucinate.

4:57 More text means less truth.

5:00 Also, let’s look at its accuracy versus the previous DeepSeek,

5:04 especially since this new version is heavily compressing things.

5:08 Ha.

5:09 Look at that.

5:10 This is crazy.

5:11 It is also fantastic at coding.

5:13 If you are a coder, great.

5:16 If you are not a coder, well, you are now.

5:19 It is so easy to ask it to generate javascript

5:22 code that you can paste into a website and run,

5:25 and in some cases, you can even run

5:28 programs in the DeepSeek window with one click.

5:30 I am a light transport researcher by trade, that is ray tracing if you will,

5:35 so I had to try a little coding task related to that and…this is fantastic.

5:42 It still failed to properly implement more advanced algorithms so

5:46 I am excited to see what the next version brings.

5:51 It is crushing benchmarks…and the competition.

5:54 At the low-low price of… free.

5:58 If you can self-host it yourself, hardware is pricey,

6:02 they also provide online access to it and it is so cheap,

6:05 I feel like numbers are losing their meaning.

6:08 Soon, intelligence will get too cheap to meter.

6:11 Depending on whether there is a discount or not,

6:14 you can easily get pricing that is 30 times cheaper than Anthropic’s Claude.

6:20 Even with no discount, things can get 8 to 20 times cheaper.

6:26 Crazy.

6:26 Now, let’s temper expectations a bit.

6:29 Limitations.

6:30 That’s what is missing from the media headlines.

6:33 One, you can almost hear the 1,500 pages fluttering as it churns through it.

6:40 But wait.

6:41 I did not say also 10 hours of audio, or full feature-length movie.

6:46 There is a reason for that.

6:49 This system is unimodal.

6:50 Not multimodal.

6:51 No images or audio.

6:53 It is blind and deaf, if you will.

6:57 Two, this system is not fully understood, not even by its creators.

7:02 They report two techniques that magically stabilize training,

7:06 and they say that they are not quite sure why.

7:09 I’ll note that this is something that happens to every researcher,

7:13 and I have nothing but respect for the transparency.

7:17 And three, we noted that if you are

7:20 pushing against the limits of the context window, things break down a bit.

7:24 Be careful.

7:25 Just want to make sure that you don’t get oversold on what is going on here,

7:29 this still has limitations.

7:31 Not small ones.

7:33 But, overall…this is not a small step in open and free AI systems.

7:38 Congratulations to the team and thank you so much.

7:42 Now here’s what I think.

7:44 I think this is a great release and a great paper and great life advice too.

7:49 Why?

7:49 Well, you can adapt so many of these ideas to your thinking.

7:54 Imagine walking in the forest.

7:56 You want to look at the amazing views in front of you.

8:00 But then, you trip.

8:01 Or you look mainly in front of your feet so you don’t trip.

8:05 You watch your step… or you enjoy the view.

8:08 Not both.

8:09 So what is the solution?

8:12 You do both.

8:14 Scan near, glance far.

8:16 Step and look.

8:17 Local detail, global context.

8:19 It is the same as what DeepSeek does.

8:21 Try it out next time you are on a walk, it’s weird.

8:26 You’ll see.

8:27 Let me know in the context, I mean comments how it went.

8:31 They also use a technique called Engram- normally,

8:34 an AI recalculates nearly every fact from scratch every time.

8:39 Engram lets it just recall those facts instead.

8:43 It’s not as easy as it sounds,

8:45 we have a separate video on it, link in description.

8:48 And we are still just scratching the surface here.

8:52 Now this is a really advanced research paper,

8:54 with all the good and the bad, not just the hype.

8:58 Also, this video was not super fast, I rewrote this over and over again.

9:03 Why is that?

9:05 Because distilling complex ideas into simple explanations takes time.

9:10 You get fewer views than others who publish something as quickly as possible.

9:14 But that’s what I try to do here, and it is an honor to do this for such

9:20 an incredibly smart and receptive audience like you Fellow Scholars.

9:24 And thank you so much for appreciating it- this one really made my day.

9:29 Subscribe and hit the bell if you enjoyed this.