DeepMind’s New AI: A Gift To Humanity

0:00 Google DeepMind gave an amazing gift to humanity.

0:04 And it is full of surprises.

0:06 Here’s why.

0:07 Today, we are living in the age of AI where these smart assistants

0:11 and agents can do things we could only dream of 10 years ago.

0:17 But.

0:17 Many of these solutions are proprietary,

0:20 require a subscription, and run in the cloud.

0:24 And then this happens.

0:26 Yup, some OpenClaw users reported losing access

0:29 to their Claude AI subscription citing “heavy workloads”.

0:33 Now, maybe they did something unsavory, I don’t know.

0:36 I also understand you pay a fixed rate, you can’t eat all you want.

0:42 I respect that.

0:43 However, this is the point.

0:45 We have to rely on the goodwill of these companies for our workflows.

0:49 So this is why I keep saying over

0:51 and over that we should always look for options

0:54 where you we own these AIs and run them on our own systems for free, forever.

1:00 No one can take them away.

1:02 NVIDIA came out with their Nemotron 3 Super,

1:06 which has super capabilities…but its hardware requirements are also super.

1:13 Not so much with Google DeepMind’s new AI, Gemma 4.

1:17 This is a free and open family of models, and yes,

1:21 finally, the smallest ones require only a few gigabytes of memory.

1:27 No need for an expensive GPU.

1:29 So much so that I wanted to wait a bit before publishing

1:33 this video to see how you Fellow Scholars use it in practice.

1:39 And…look at that!

1:40 It runs on your phone without an internet connection.

1:44 And folks are already using it in practice

1:47 to create offline translation and summarization apps.

1:51 Also, real time image classification running

1:54 in your browser while talking like a bard?

1:58 No problem.

1:59 You can already fine tune it with Matt’s work.

2:03 It is so good, it has a little ecosystem around it already in just a few days.

2:11 Because of the brilliance of you Fellow Scholars.

2:15 Nice work.

2:16 But it gets better.

2:18 You see, the smallest Gemma is so small, it runs on…oh my.

2:24 Look…I love that.

2:26 It runs even on an old beat up nintendo switch, first generation.

2:32 Not exactly something with a lot of memory or processing power.

2:36 Still runs the 2 billion parameter Gemma 4 model.

2:39 Now that is a gift to humanity.

2:42 But it gets really strange from here on out.

2:45 Here are 4 things that I found really surprising.

2:49 Dear Fellow Scholars, this is Two Minute Papers with Dr.

2:52 Károly Zsolnai-Fehér.

2:53 One, they also have a bigger, 31B model which was the #3 best open model,

3:00 and now hold on to your papers Fellow Scholars,

3:04 because it beat some models that are 10 times larger.

3:09 And still competitive with some that are 20 times larger.

3:14 On some measurements.

3:16 And it is a dense model.

3:20 What?

3:20 What is going on here?

3:24 You see, many of the modern AI systems you

3:27 encounter are what they call mixture of experts models.

3:31 MoE.

3:31 These are huge AI models with many parameters,

3:36 and to make sure we don’t burn down all of our hardware using them,

3:39 it splits up this big brain into many small ones.

3:42 If you have a biology question, it chops it up into small parts,

3:47 and routes them to the parts of the brain

3:50 that it thinks are the best experts at processing it.

3:53 Typically, to the top 2 to 8 experts.

3:57 Only ask them.

3:58 Yes, with that, we only activate one small part of a brain at a time.

4:05 It makes sense, right?

4:06 It’s not a simple process, but it is possible.

4:10 This enables us to create huge intelligent models that are still efficient.

4:15 Dense models, however, just light up every parameter of the system.

4:21 These are not new, and in some ways, these are very inefficient.

4:26 You light up all the 31 billion parameters in the brain all the time,

4:32 no matter how simple or complex the question is.

4:36 But this one…this one is somehow magically good.

4:40 How?

4:41 They did four amazing things: one,

4:43 Google didn’t just dump half the internet into it to learn about us.

4:49 They apply super strict filters to give it only highly curated training data.

4:54 That is actually good advice for our thinking too.

4:59 Don’t let everything in, curate your information diet.

5:03 There is lots of noise out there- ignore it.

5:08 That is excellent.

5:09 Two, they use an interesting attention mechanism that has

5:13 a sliding window and also global attention at the same time.

5:18 What does that mean?

5:20 Well, when you read a book, you read it line by line to finish a page.

5:26 That is a local sliding window.

5:28 With that, you get all the details.

5:31 But sometimes you want to zoom out and ask, okay, what book are we reading?

5:38 Which chapter is this?

5:40 That is global attention.

5:42 Here, they use both, and call the mechanism hybrid attention.

5:47 Three, it is better at understanding images.

5:50 You know, Gemma 3 had weird glasses

5:53 on, and its image understanding was kind of a lie.

5:57 If you gave it a landscape image,

6:00 it squished it back to a square image before processing it,

6:04 losing some information.

6:06 It squishes everything into its own preconceived box.

6:10 Not good.

6:11 Gemma 4 understands the image as-is,

6:15 and the difference really shows on any benchmark that has to do with images.

6:20 Four, it has a shared KV-cache.

6:23 KV-cache is short term memory for what you are currently talking with it,

6:28 documents, questions.

6:29 Now the layers of this neural network

6:32 like to recompute their fresh memory from scratch.

6:36 This one doesn’t, it essentially borrows

6:39 the memory already computed by earlier layers.

6:42 Less work, nearly the same result.

6:45 This is one of those ideas where we are

6:49 wondering why we didn’t always do it like this.

6:52 Okay, and all this was just part of my first surprise.

6:57 Second surprise.

6:58 It is fantastic at agentic workflows.

7:01 This is where we don’t just have an AI assistant that spits out a bunch of text,

7:07 this is when we give it arms and legs and ask it to do stuff.

7:12 Tool use, local coding, and a ton more.

7:15 Plug it into OpenClaw and it can book a plane ticket.

7:19 Look for news and summarize it in a more unbiased way.

7:23 Or write silly emails to Károly from Two Minute Papers.

7:28 That sort of thing.

7:30 It is really good at that.

7:33 So when any company decides that you

7:36 can’t use their system anymore, that’s alright.

7:38 Just plug in Gemma 4, and you are good to go.

7:42 For free.

7:43 People find that if you give it custom instructions,

7:46 sometimes you don’t even notice the difference.

7:49 That is huge.

7:51 Surprise number three, the context window was improved to 256k,

7:56 twice as big as Gemma 3 had.

7:59 This is pretty expensive to compute, so don’t take it for granted.

8:03 Here, you are not going to chuck gigabytes of movies into it,

8:07 but for a few long documents, it is perfectly fine.

8:11 Four, the license.

8:13 Oh my, the license.

8:15 This one gets overlooked so much.

8:18 Gemma 3 came with a Gemma license.

8:22 In other words, it comes with strings attached.

8:26 The model comes with handcuffs, if you will.

8:29 If you use it to create training data for a derivative model.

8:34 Yup, that one inherits the handcuffs too.

8:38 But, with Gemma 4, not anymore.

8:42 Look, Apache 2.0 license.

8:45 Now we’re talking, yes!

8:48 This license is true to the open source spirit.

8:51 You can modify it, sell it, deploy it commercially with almost zero friction.

8:56 Make derivative models, do a ton of stuff with far fewer restrictions.

9:03 This is huge.

9:05 Thank you so much!

9:07 Now, not even this technique is perfect.

9:09 For instance, the model does not have a live database.

9:13 Without an agent harness, it cannot browse or look up stuff.

9:19 Meaning?

9:20 Well, meaning that it can be confidently incorrect.

9:25 The internet special.

9:27 Also for highly complex, open-ended tasks- it’s not great at that.

9:33 Or, when you have images with lots of high-frequency visual details,

9:37 thin structures, blades of grass, or a fence from far away.

9:42 Not great at that, it’s going to need even better glasses.

9:47 But, adding this all up, this is an amazing gift to humanity,

9:51 one that cannot be taken from us.

9:54 This is not for Mr moneybags, this is for the little man,

9:58 and it is free, for all of us, forever.

10:02 Hugely appreciated.

10:03 Absolutely loving it.

10:05 What a time to be alive!

10:09 Also, I waited with this video because I did not just

10:12 want to take the marketing messaging and copy-paste it to you.

10:16 I wanted to see how you Fellow Scholars are actually using it in practice.

10:20 Read through your experiences with it.

10:23 Does it really work in practice?

10:25 Super important.

10:26 That’s what we are here for, not the copy-pasted media headlines.

10:32 That needs time.

10:34 Trying to explain all this in simple words also takes time.

10:37 I don’t have a team here,

10:40 I do everything from the writing to recording, video editing.

10:45 I am trying my best here.

10:48 But it gives you more accurate information,

10:51 and that is the most important for me.

10:54 So, now, after 10 million downloads in the first week and more thorough testing.

11:00 My opinion is that yes, this thing rocks.

11:03 I would like to send a big thank

11:06 you to every single scientist who worked on this!

11:09 And hold on to this one for dear life because

11:13 a frontier model just got locked down for a few select clients.

11:17 You know, it’s a big club.

11:19 And we ain’t in it.

11:21 If you enjoyed this, consider subscribing and hitting that bell.