NVIDIA’s New AI Shouldn’t Work…But It Does

0:00 Dear Fellow Scholars, this is Two Minute Papers with Dr.

0:03 Károly Zsolnai-Fehér.

0:04 We’ve been trying out doing the videos with a camera.

0:08 I really enjoyed it, and your feedback was also absolutely incredible.

0:12 I’ve never seen anything like this.

0:15 So many comments, thank you so much for the kind words everyone.

0:19 So, we will try to do more of this.

0:22 But note that this one is a classic voice paper that we’ve always done here.

0:27 It was done before we did the camera thing,

0:30 I thought I would record this little intro now so you don’t get surprised.

0:34 And then next video I’ll be back.

0:37 And for now, please enjoy this super fun paper.

0:40 How do robots learn how to be a good robot?

0:44 Well, surely not like this.

0:46 Haha.

0:46 Not by just running around in the real world.

0:49 Of course!

0:50 I mean, imagine a real robot doing this for years and years.

0:54 It would be dangerous to others and itself.

0:57 So here is a better question:

1:00 how do we teach a robot to be a helpful, good robot safely?

1:04 Well, we put it inside of a video game.

1:07 Start learning there first!

1:09 In the game, we simulate physics, and let it fail.

1:12 A lot.

1:13 Then, over time, get better.

1:15 Now, I’ve been to a bunch of AI and robotics labs around the world,

1:20 and let me briefly summarize what I saw:

1:23 things work fantastically well in a simulation, and then,

1:26 when you put them into the real world, huge disappointment.

1:30 Something that worked really well suddenly does not work well or at all.

1:36 Why?

1:36 Well, the main reason is that simulations are often just not good enough.

1:41 They often mimic reality, but they are not a substitute for reality.

1:47 So what do we do?

1:49 Well, of course, try to use reality.

1:52 In this work, DreamDojo, scientists said okay,

1:55 let’s feed the AI 44 thousand hours of videos of humans doing stuff.

2:01 That sounds great, except the fact that it is completely useless.

2:06 Why?

2:06 Well, humans and robots have entirely

2:09 different physical bodies, hands, and joints.

2:12 Also, the video does not contain action information.

2:15 It’s just a soup of data that doesn’t

2:19 say what joints are exerting forces and how.

2:22 Nothing.

2:23 So why do this?

2:24 Does this even make sense?

2:26 Well, they propose 4 genius ideas,

2:28 and I hope that will make this work, because it would be a miracle.

2:34 One, if the video does not have labels on what actions are taking place, well,

2:38 then let the AI try to understand it

2:41 and make up its own story of what is happening.

2:44 If you see someone waving at a bus that is pulling away.

2:48 You don’t need a text label to know that someone has just missed their ride.

2:53 Two, this dataset is stupendously large.

2:55 It has more than 4 billion frames, and probably more than 1 quadrillion pixels.

3:02 Okay, that is too much information.

3:05 It is almost impossible to handle.

3:08 So the AI has to learn what is important and what isn’t.

3:13 How?

3:13 Well, it is forced to compress information.

3:16 A musician does not need to know every song in the universe.

3:21 They have to know that there are 12 notes in a scale,

3:24 and every song is just built as a combination of these fundamental notes.

3:29 This forces the AI to look at only the most critical information.

3:34 But guess what, it is still not enough to just

3:37 dump videos into the robot and make it work.

3:41 Why?

3:41 Well, three, if you train a robot to pick up a cup at a global position,

3:46 it learns to reach for that exact spot in the world.

3:50 That’s no good.

3:51 Why?

3:51 Well, if you move the cup a few inches to the left,

3:56 the global coordinates change entirely, and the robot has no idea what to do.

4:01 So, what scientists said, instead of using absolute robot joint poses,

4:06 let’s transform the inputs into relative actions.

4:09 If you are cooking, sometimes you don’t need absolute coordinates.

4:14 Here, the knife only needs to know where it is relative to the carrot’s spot.

4:19 And believe it or not, this is still not working.

4:23 We need something more.

4:25 What do we need?

4:27 Well, four, the goal is that the AI learns cause and effect.

4:31 Jelly bunny hits the wall, and something happens.

4:34 Try to learn that by predicting the next frame.

4:39 The problem is that the AI is cheating.

4:42 Like a student, it just looks at the solution at the end,

4:46 and says, oh yeah, I was gonna say exactly that.

4:50 So how did they prevent that?

4:52 Well, they fed it actions in small blocks of 4 at a time,

4:57 so it cannot cheat by peeking at the future to guess what happens right now.

5:03 Okay, this was a lot of genius stuff, so it better give us something amazing.

5:09 Let’s see what we got.

5:11 Previous method.

5:12 Can’t predict the future…oh my, look,

5:15 that hand clips through the piece of paper.

5:18 Now hold on to your papers Fellow Scholars for the new method and….oh my!

5:25 Look at that!

5:27 The paper finally crumples beautifully!

5:30 And with previous methods, the clipping gets even worse.

5:35 Look.

5:35 That’s not predicting reality, that’s just guessing.

5:39 New technique- now we’re talking!

5:44 Looking good!

5:45 Also, previous technique, hand moves the lid, and the lid refuses to move.

5:52 No good.

5:53 New technique, the lid moves!

5:56 Woo-hoo!

5:57 Yes, this is the corner of the internet

6:00 where we get unreasonably happy about a moving lid.

6:03 And these are not some cherry-picked results,

6:06 the new technique is so much better than previous methods.

6:09 This is a huge leap forward!

6:12 Now, this gets even better.

6:14 So it finally understands the world better than previous techniques.

6:18 So what do we pay for this?

6:20 How much slower is this than previous methods?

6:23 Well, it is pretty slow because it requires

6:27 35 heavy denoising steps just to generate one prediction.

6:31 But wait, don’t despair!

6:33 We can use distillation here.

6:35 Distillation is a training phase where a fast student

6:39 model is used to learn the predictions of the slower,

6:43 high-quality teacher model.

6:44 The goal is that the student would be nearly

6:48 as good as the teacher model, but much faster.

6:51 Well, let’s test that!

6:53 Oh my, now the student is a heck of a lot faster.

6:59 It seems that it is 4 times faster than the teacher that was used to train it.

7:04 It runs at about 10 frames per second.

7:07 Understanding the world and predicting how it

7:10 will change at a speed that is interactive.

7:13 That is absolutely insane.

7:15 Well done!

7:16 And the kicker is that they also predict very similar outcomes.

7:22 This is an absolute slam dunk paper.

7:25 Wow.

7:26 Now for you wise Fellow Scholars out there,

7:28 I’ll note that we talked about a technique called NeRD, Neural Robot Dynamics.

7:33 That was a robot AI that trained in its own imagination.

7:38 So how does this relate to that?

7:41 Now NeRD was building a perfect 3D environment.

7:44 This one thinks in 2D.

7:46 It just sees the world as a bunch of 2D video pixels on a flat TV screen.

7:53 Thus, this one is able to learn about thousands of everyday objects.

7:58 So cool!

7:59 This finally gives us smarter AI robots,

8:02 and robots that we can all own ourselves.

8:05 In a world full of subscriptions,

8:07 it is so refreshing that we get all of this for free.

8:11 A ton of code and pre-trained models are available for free for all of us.

8:16 No silly subscriptions and proprietary code.

8:19 A free brain that you can upload

8:21 to your own devices and use it however you want.

8:25 Love it.

8:25 So this finally puts us one step closer to having a robot fold our laundry,

8:31 or cook a healthy meal.

8:33 Or help a specialist doctor perform surgery

8:36 from the other side of the planet via teleoperation.

8:39 What a time to be alive!