Inside the World's Smartest Robot Brain [VLA]

0:00 This may be the most significant moment in modern robotics.

0:04 In 2023, a researcher at Google set up a table with a Coke can,

0:08 pictures of Tom Cruise,

0:09 Snoop Dogg, and Taylor Swift, and asked Google's newest robot brain,

0:13 RT2, to move the Coke can to Taylor Swift.

0:17 RT2 was too large to run on the robot itself.

0:21 The robot sent one image at a time from its onboard camera to a TPU cluster,

0:25 which sent back control signals.

0:28 The robot controlled by RT2 slowly picked up the Coke can

0:32 and awkwardly placed it on the edge of the picture of Taylor Swift.

0:36 A couple years later, in 2025,

0:38 one of the researchers on the team, Carol Hausman,

0:41 would describe this scene as the moment it became

0:44 clear to him that this was going to work.

0:47 Within a year of the 2023 Coke can demo,

0:50 Hausman and many of the key members of the RT2 team had left Google,

0:54 and reassembled to form a startup called Physical Intelligence.

0:58 And their robots have gotten better, a lot better.

1:02 The latest robot brains from Physical Intelligence can open padlocks,

1:06 fold your laundry, peel an orange, make a grilled cheese sandwich, make coffee,

1:11 and clean up bedrooms and kitchens that it's never seen before.

1:15 Why was this unimpressive Coke can demo such a breakthrough?

1:19 And how did it enable Physical Intelligence to improve their robots so rapidly?

1:24 In this video, we'll first explore the fascinating build-up to RT2 at Google.

1:29 From here, we'll take a deep dive

1:31 into the Physical Intelligence Robotics Foundation Models,

1:35 and see what makes these incredibly impressive robot brains tick.

1:41 In 2022, the year ChatGPT was released,

1:44 researchers at Google began exploring what role

1:46 large language models might play in robotics.

1:50 Their first notable result, a system known as SayCan,

1:53 used a large language model as a planning

1:55 system to break down complex tasks into subtasks.

2:00 In this demo, SayCan breaks down cleaning up

2:02 a spill into the subtasks of finding a sponge,

2:05 picking up the sponge, going to the spill, and so on.

2:09 From here, the team expanded on this work,

2:11 creating a more capable iteration of the idea called inner monologue,

2:15 and another interesting variant where the team used an LLM

2:19 to write code to control the robot on the fly.

2:23 However, these early efforts were effectively

2:25 bottlenecked by the available robot controls algorithms.

2:30 Once the LLM in SayCan decided to pick up a sponge,

2:33 a completely separate neural network that had been trained

2:36 to imitate humans controlling robots to perform various small tasks,

2:40 was used to compute the actual robot control signals.

2:44 This meant that SayCan was effectively limited

2:46 to a menu of actions the LLM could choose from.

2:50 To get SayCan to place a Coke can on an image of Taylor Swift,

2:53 behaviors involving Coke cans and Taylor

2:55 Swift would have to be explicitly trained.

2:59 At the end of 2022, the team

3:01 made a significant improvement to their control layer,

3:04 introducing Robot Transformer 1, or RT1.

3:08 Like the team's previous control algorithms, RT1 was trained to imitate humans,

3:13 but used a significantly larger dataset, with over 130,000 human demonstrations,

3:19 and used a larger transformer-based architecture.

3:23 RT1 was able to perform a significantly

3:25 broader range of actions than its predecessors.

3:28 This effectively gave the planning layer a much

3:30 larger menu of actions to choose from.

3:34 The RT1 team showed that using the planning LLM from SayCan,

3:38 coupled with RT1 to control the robot,

3:40 significantly improved performance on long-horizon tasks,

3:44 like finding certain items in kitchens that the robot hadn't seen before.

3:50 As Google incrementally improved their robot brains,

3:53 large language models were also rapidly advancing.

3:57 The LLM used for planning in the SayCan

3:59 and RT1 systems was the text-only Palm 540B model, trained in early 2022.

4:06 This meant that the robot's planning layer couldn't actually see the world.

4:11 After breaking down a task like helping clean up the kitchen into text subtasks,

4:15 Google's robots relied on the RT1 control layer

4:18 to take in images from the robot's camera,

4:21 and iteratively send control signals to the robot's actuators,

4:24 until each subtask was complete.

4:27 This approach worked fine for some tasks,

4:30 but having a planning layer that was effectively blind was clearly not ideal.

4:35 On March 6th, 2023, about a week before the release of GPT-4,

4:40 Google researchers demonstrated Palm-E,

4:43 a variant of the Palm large language model

4:45 that directly incorporated images and other data sources.

4:49 Using the multimodal Palm-E instead of the purely

4:52 text-based Palm LLM as a planner, with RT1 as the control layer,

4:58 the team demonstrated a significant expansion in capabilities.

5:02 Now that the planning layer had access to vision information,

5:05 the robot could perform more complex tasks that require adaptive planning,

5:10 like moving objects out of the way to reach a desired object,

5:14 and fully autonomously recovering from setbacks.

5:17 Here, a robot using Palm-E as its planning layer and RT1 as its control layer,

5:22 is asked to retrieve a bag of chips.

5:25 And when a researcher repeatedly puts the chips back in the drawer,

5:28 Palm-E is remarkably able to recognize

5:30 that something has changed and adapt its plan.

5:34 Now, let's zoom out a little and consider the full Palm-E plus RT1 robot brain.

5:40 Although Palm-E and RT1 were designed to work at different levels of the stack,

5:44 they have some really interesting similarities.

5:48 Both models take in images from the robot's camera,

5:51 and use a vision encoder neural network to process the images.

5:55 From here, in both models,

5:57 these encoded image representations are passed into a transformer.

6:01 This is the same type of compute

6:03 block used fairly universally in large language models.

6:07 The big difference here is what these transformers are trained to do.

6:11 The RT1 transformer was trained to directly output robot control

6:15 signals by imitating humans controlling robots to solve various tasks.

6:20 While the Palm-E transformer is trained to output

6:22 text across a wide variety of tasks,

6:26 including simple next token prediction on internet text,

6:29 as we see in standard LLM pretraining,

6:32 but also language-vision tasks, like image captioning.

6:36 And importantly, Palm-E was also trained

6:38 to break apart robotics tasks into smaller subtasks.

6:43 The similarities between Palm-E and RT1,

6:46 and the fact that the team was able to expand the Palm

6:49 language-only model to effectively make use of other types of data,

6:53 all beg the question, do we really need two separate models here?

6:58 Why not just continue expanding the Palm language

7:01 model to not only take in image data,

7:04 but also to directly output robot control data,

7:08 effectively absorbing RT1 into a single powerful end-to-end model?

7:14 Said differently, can large language models,

7:17 by far the most powerful AI systems we trained so far,

7:20 be trained to become robot brains?

7:25 This brings us to Taylor Swift and the Coke can.

7:28 In July 2023, a few months after the Palm-E paper came out,

7:33 the Google Robotics team demonstrated RT2.

7:36 Taking Palm-E and another multimodal LLM known as PaLI-X as starting points,

7:42 the Google team trained these LLMs to directly output robot control signals,

7:47 training on the same human control demonstration data

7:50 they had used to train RT1 6 months earlier.

7:54 And incredibly, it worked.

7:57 RT2 was able to generalize shockingly well to objects, environments,

8:01 and tasks that were not in the human demonstration data.

8:05 This is what makes the Taylor Swift demo so impressive.

8:09 The robot control training data definitely did not include Taylor Swift.

8:14 So, for RT2 to solve this task,

8:17 it had to learn how to bring together abstract concepts it

8:20 had learned in its internet-scale pretraining with the robot control episodes.

8:25 This means that these models can learn

8:27 to connect the vast amounts of image, video,

8:30 and text data on the internet with real-world actions,

8:34 potentially harnessing the full knowledge of the internet into robot brains.

8:39 This is why this demo is such a big deal.

8:42 It answers the question,

8:44 can large language models be trained to be robot brains?

8:48 With a shaky, but definitive, yes.

8:51 The RT2 team coined a new name for this type of model, vision-language-action,

8:56 or VLA, linking together vision, language,

9:00 and action into a single unified model.

9:03 This video is about to get technical.

9:06 To see how language models can learn to become robot brains,

9:10 we're going to reference transformers, embedding vectors,

9:12 diffusion models, attention heads, softmax, and more.

9:16 The required context for all these concepts is way

9:19 more than we can fit into a single video, which is why I wrote this book.

9:24 The Welch Labs Illustrated Guide to AI breaks

9:27 down all of these concepts using hundreds of figures,

9:30 detailed descriptions, and exercises.

9:33 You can pick up a copy at welchlabs.com.

9:35 And we're very excited to announce

9:37 that we're beginning to offer international shipping.

9:40 Stay tuned to the end of the video for more updates on the book,

9:43 and to see the poster that goes along with this video that nicely

9:47 breaks down the vision-language-action model architecture.

9:52 By early 2024, a number of key members of the RT2

9:56 team had left Google and reassembled to form the startup physical intelligence.

10:01 In October of that year, the team demoed their first robot brain, Pi Zero.

10:07 Compared to the RT2 Coke can Taylor Swift demo 15 months before at Google,

10:12 Pi Zero is remarkable.

10:15 It starts to really feel like a robot that could help you around the house,

10:19 performing tasks like getting laundry out of the dryer,

10:21 folding the laundry, and cleaning up tables.

10:24 How was the physical intelligence team able

10:26 to improve on RT2 so significantly and so quickly?

10:31 Like RT2, Pi Zero is a vision language action

10:35 model built on top of a pre-trained multimodal LLM.

10:40 Based on Pi Zero's strong performance,

10:43 you might guess that the physical intelligence

10:44 team increased the model size relative to RT2, but Pi Zero is actually smaller.

10:51 The RT2 model family ranged from 5 to 55 billion parameters,

10:56 and Pi Zero remarkably only uses 3.3 billion parameters,

11:01 allowing the model to run on the robot itself using a consumer-grade

11:04 Nvidia RTX 4090 GPU at a very respectable 73-millisecond inference time.

11:11 Here's what Pi Zero looks like hooked up to a two-arm

11:14 robot platform called Aloha and tasked with uncapping a pen.

11:19 Pi Zero takes images from an overhead camera and from one

11:22 camera on the wrist of each robot arm and a text prompt.

11:27 At each time step, Pi Zero returns 14 numbers.

11:31 One number for the position of each of the seven actuators on each arm.

11:35 Here we're plotting these outputs as a time series.

11:38 This movement in our pink curve here shows us where Pi

11:42 is telling the left gripper to grab onto the pen cap.

11:48 Pi Zero is built on top of PaliGemma, an open-weight multimodal LLM from Google.

11:55 PaliGemma is built from two other open-weight models,

11:58 the SigLip image encoder and the Gemma large language model,

12:02 that are trained together to solve vision language tasks like image captioning.

12:07 Now, following the RT2 approach, the underlying language model,

12:12 in this case PaliGemma, would be trained to directly output control values.

12:17 However, the physical intelligence team made a clever improvement

12:21 here that makes Pi Zero significantly better at dexterous manipulation.

12:25 Instead of having the underlying language model directly output control values,

12:29 Pi Zero introduces a second neural network the team calls an action expert.

12:35 Interestingly, the Pi Zero action expert uses the same architecture as Gemma.

12:40 In fact, in the Pi Zero code base,

12:42 the action expert is instantiated as a Gemma model.

12:46 The only differences are that the action

12:48 expert is randomly initialized instead of pre-trained,

12:52 and the action expert is not as wide as Gemma,

12:55 using fewer parameters within each layer.

12:58 Now, this may sound like we're going back to the earlier SayCan system,

13:02 where a high-level LLM performed planning

13:05 and a lower-level network handled robot control.

13:09 The key distinction here is that in the SayCan system,

13:12 the interface between models was natural language.

13:16 The planning LLM told the control network

13:18 what to do using predetermined text instructions.

13:22 Pi Zero, in contrast, uses a much richer interface between the two models.

13:27 Since the Gemma LLM and action expert effectively share the same architecture,

13:32 it's possible for these models to almost think

13:34 as one while retaining some really nice benefits of modularity.

13:39 Let's have a closer look at how our Gemma LLM learns to act as a robot brain.

13:44 Then we'll have a closer look at how

13:46 the interface between these two models works.

13:49 The Gemma LLM processes both the images and text prompts that come into Pi Zero.

13:54 Each image is broken into a grid of patches,

13:57 resulting in 256 image patches per image and 768 total patches.

14:03 The patches from each image are passed into an image encoder model,

14:07 resulting in 768 embedding vectors, each of length 2048.

14:14 These vectors are sometimes referred to as soft tokens.

14:18 Here we're coloring each embedding vector

14:19 to approximately match its corresponding image patch.

14:23 This will help us keep track of our data as it flows through our model.

14:27 These embedding vectors live in a semantically rich embedding space,

14:32 meaning they should contain lots

14:33 of easily accessible information about our images,

14:37 like whether a given image patch contains a pen.

14:41 For more on embedding spaces and image encoders,

14:43 check out the Welch Labs video on AlexNet,

14:46 the AI image generation video we did with Three Blue One Brown,

14:50 or the Welch Labs illustrated guide to AI.

14:53 The text prompt we give Pi Zero,

14:55 in this case uncap the pen, is broken into four tokens,

15:00 and each token is mapped to an embedding vector

15:02 of the same length as our image patch embedding vectors.

15:07 So, we now have 772 total embedding vectors,

15:12 768 from our images and four from our text prompt.

15:16 From here, these embedding vectors are passed into our Gemma LLM.

15:21 Gemma is composed of 18 transformer blocks,

15:24 each containing an attention and multi-layer perceptron compute block.

15:29 Each attention block contains eight attention heads.

15:33 These attention heads are arguably the most

15:34 critical part of the transformer architecture and are

15:38 the key to the tight integration between Pi

15:40 Zero's underlying LLM Gemma and Pi Zero's action expert.

15:45 In a given attention head, the incoming embedding vectors are multiplied

15:49 by three separate matrices of learnable weights,

15:53 producing three new matrices known as queries, keys, and values.

15:59 Each of these matrices has 772 rows, one for each input to our transformer.

16:06 We don't have enough space to visualize all 772 rows of our matrix.

16:11 Here we're showing the first row,

16:13 which corresponds to the upper left patch of our overhead image.

16:17 Next, we're showing rows 373 to 376,

16:22 which correspond to these four patches of our left wrist image.

16:27 This will be important shortly as we see how Gemma figures out how to connect

16:30 the word for pen to the parts of the images that contain the pen.

16:35 As we did with our embedding vectors, we'll color each row of our matrix

16:39 with the approximately average color from its corresponding image patch.

16:44 Our two patches that contain the orange pen get colored orange.

16:48 And finally, the light and dark parts of each

16:50 vector correspond to the actual numerical values of the vector.

16:54 Dark regions are lower numbers and light regions are higher numbers.

16:58 Finishing out our matrix,

17:00 these last four rows come from our input text with one row for each token,

17:05 and we'll color all our text rows blue.

17:08 From here, Gemma's attention head searches

17:10 for similar query and key matrix rows.

17:14 This attention head may have learned, for example,

17:17 to specialize in searching the incoming images for objects

17:20 that match words that appear in the prompt.

17:23 After all, if our robot brain is going to uncap the pen,

17:26 it needs to know where the pen is in our images.

17:30 The word pen shows up at our very last token input position,

17:34 and its query vector looks like this.

17:37 The attention head computes the dot product between

17:39 this row and every row in our key matrix,

17:43 and larger dot products indicate closer matches between queries and keys.

17:49 Interestingly, our highest dot products in this sample by far

17:53 occurred at two image patches that contain the pen.

17:57 From here, our attention head normalizes

17:59 these dot product values using a softmax operation.

18:02 We can take our visualization one step further here and show

18:06 these attention values as a heat map on top of our images,

18:10 where brighter shades of magenta correspond to larger attention values.

18:15 So, the two orange rows of our key matrix with high attention

18:19 values that correspond to these two image patches get colored bright magenta,

18:24 and their neighboring patches with low attention scores do not.

18:28 So, the idea here is that our heat map visualization shows us the strongest

18:32 matches in our images to our query vector for the word pen in our prompt.

18:38 And remarkably, our best matches occur at the patches

18:41 in all three images that show the pen.

18:44 Playing our video and running this analysis at each frame,

18:47 we see impressive pen tracking results.

18:52 Our model is clearly using this attention head to connect the word pen

18:55 in our prompt to the parts of our images that contain the pen.

19:01 Now, our attention head doesn't just search for matches to our pen query.

19:06 All 772 query vectors, corresponding to all input images and prompt tokens,

19:12 are compared to all 772 key vectors.

19:16 The resulting attention values from all these comparisons are

19:20 collected in a 772 by 772 attention pattern matrix.

19:25 Each row of the attention pattern corresponds to a single query.

19:30 The final query row corresponds to the pen

19:32 token in the prompt that we've been visualizing.

19:35 So, our heat map values end up in the bottom row of our attention pattern.

19:40 At the beginning of our attention head,

19:42 we computed three matrices, our queries, keys, and values.

19:47 We've used our queries and keys to create our attention pattern,

19:51 and now our attention pattern is multiplied by our value matrix,

19:55 creating this attention heads output, a new 772 by 256 matrix.

20:01 Multiplying our value matrix by our attention

20:03 pattern effectively moves information between token positions.

20:08 The large attention values we see between

20:10 our pen query and pen image patches mean

20:13 that these image patch rows are copied and added

20:16 to the pen position in our final output.

20:19 One way to think about this operation is

20:21 that our attention head is forming a unified representation of the text for pen

20:26 and the parts of our images that contain pens.

20:30 Now, this is just a single head in a single layer of our 18-layer Gemma LLM.

20:36 And we expect different heads to learn

20:37 to pick up on different types of patterns.

20:40 And remember that our Gemma LLM is just one part of the Pi Zero system.

20:45 Let's now turn to Pi Zero's action expert model and see how the physical

20:49 intelligence team was able to get these models to work together so seamlessly.

20:54 While the PaliGemma portion of Pi Zero takes

20:56 in our 772 image and text prompt tokens,

21:00 the action expert takes in information about our robot state.

21:04 That is the position of all of its joints.

21:08 On the Aloha platform we've been experimenting

21:10 with, each arm has a movable waist,

21:13 shoulder, elbow, forearm rotation, wrist, wrist rotation, and gripper.

21:19 This makes for seven joints per arm or 14

21:23 total numerical values that we need to control our robot.

21:27 Just as our text prompt and input images are mapped to embedding vectors,

21:31 our vector of 14 joint positions is also mapped to an embedding vector.

21:36 This mapping is done by multiplying our joint vector

21:38 by a 14 by 1024 matrix of learned weights.

21:43 Note that while the Gemma LLM in Pi

21:45 Zero uses an embedding vector of length 2048,

21:48 the action expert uses embedding vectors of length 1024.

21:52 This reduces the compute requirements and inference

21:55 time of the action expert model.

21:57 So the robot's current state fits into a single embedding vector or soft token.

22:02 This is one of the inputs to our action expert.

22:06 The action expert has one more set of inputs,

22:09 the joint positions of the robot over the next 50 time steps,

22:13 generally referred to as actions.

22:16 Now, this might seem backwards.

22:18 The whole point of the action expert is to predict the future robot actions.

22:23 How could the model take predicted actions as an input?

22:27 In a fascinating transfer of ideas from AI video and image generation,

22:32 Pi Zero's action expert uses a method called flow matching.

22:36 The idea is that instead of outputting robot actions in one go,

22:40 the model iteratively shapes completely random actions into a final trajectory.

22:46 The comparison to AI image generation is really interesting here.

22:50 A final set of actions produced by our action

22:52 expert will be of dimension 14 by 50,

22:56 with one row to control each robot joint and one

22:59 column for each of the next 50 time steps.

23:02 We can visualize this matrix as an image,

23:05 as we have with other matrices in our model.

23:08 In this set of actions, we see an increase in the values in our ninth row.

23:13 We can plot these values as a time series.

23:16 This set of actions is telling our robot to move its right shoulder,

23:20 reaching its right gripper towards the pen.

23:23 In AI image generation, we can create an image of a cat by iteratively

23:27 refining a pure noise image into a detailed cat image.

23:32 Pi Zero's action expert does the same thing,

23:35 refining a 14 by 50 random image of joint trajectories

23:39 into a detailed plan for how to move each robot joint.

23:44 One reason this flow matching or diffusion process works so well

23:47 for generating natural images is

23:50 that the distribution of natural images is multimodal.

23:53 There are many ways to create an image of a cat.

23:56 Analogously, there are many ways we can move our 14 robot joints to uncap a pen.

24:03 So to generate a set of actions, the action expert starts with completely

24:07 random actions and predicts how these actions

24:10 should be updated to produce a slightly

24:11 more realistic and accurate set of trajectories.

24:15 These trajectories are added to the input

24:17 actions and then passed back into the model,

24:20 which then computes a new set of updates.

24:23 This process is repeated 10 times in Pi

24:25 Zero until we have a nice set of trajectories.

24:30 The fact that we can use the same exact flow matching process

24:33 to generate images and videos and control robots is so interesting to me.

24:38 It's such a surprisingly effective abstraction on top

24:41 of what feel like very different applications of AI.

24:46 So our action expert model can

24:47 iteratively shape pure noise into robot trajectories.

24:52 But how does it know what trajectory is to generate?

24:54 The action expert needs to know what the goal is,

24:57 in the case of our example, uncapping the pen.

25:00 And of course, it needs lots of information about the scene,

25:03 like where the pen is in space.

25:06 As we saw earlier, this is exactly the type of information

25:09 our Gemma LLM is already processing in its attention heads.

25:14 The question from here is,

25:16 how do we best give our action expert access to this information?

25:20 As we saw earlier, the action expert

25:22 uses the same architecture as our Gemma LLM.

25:26 This means that like Gemma,

25:27 our action expert has 18 attention blocks with eight attention heads each.

25:33 As we saw earlier, each Gemma attention head computes a separate query,

25:37 key, and value matrix.

25:40 Our action expert attention heads perform the same operations,

25:43 but with different inputs.

25:46 Our action expert has 51 inputs, one for the robot's current state,

25:51 and 50 for the robot's predicted actions over the next 50 time steps.

25:56 So within each attention head,

25:58 our action expert's query matrix will have 51 rows, one for each model input.

26:04 Now, using the standard attention mechanism,

26:07 each query is able to search for matches in the keys.

26:10 This could allow, for example,

26:12 our second action step to use information from our first action step,

26:17 which would help our model create a nice

26:18 smooth trajectory from time step to time step.

26:22 Of course, to figure out where these trajectories should go at all,

26:25 our action expert's queries ideally need access

26:28 to the prompt and image information from our Gemma LLM.

26:32 This is where the team's decision to use the same

26:34 architecture for the LLM and action expert really pays off.

26:39 All we have to do at this stage

26:40 is take the keys and values from the corresponding

26:43 attention head of our Gemma LLM and append them

26:46 to the keys and values from our action expert.

26:50 So we now have 51 plus 772,

26:53 making for a total of 823 keys that our action expert can query.

26:58 These keys contain all the information the action expert needs, the text prompt,

27:04 the encoded images, the robot state,

27:07 and other time steps in the planning process.

27:10 This gives the attention heads in our action expert an immediately available,

27:14 incredibly rich information source.

27:17 It's a really clever design.

27:20 This modular design allows for some impressive efficiency gains.

27:25 After the images and prompt are passed into PaliGemma,

27:28 the computed keys and values in each attention head are cached.

27:32 This is a common step in LLM inference,

27:35 preventing redundant computation as new tokens come along.

27:39 However, in this case, the physical intelligence team uses PaliGemma's KV cache

27:44 to feed into each action expert's attention head.

27:49 Since the action expert uses a flow matching process,

27:52 it needs to run multiple times to produce final smooth trajectories,

27:56 but is able to use the same KV cache each time

28:00 because the input images don't change until the next time step.

28:04 The fact that all these components can be

28:06 trained to work together so well is absolutely incredible.

28:10 At each step, Pi Zero takes in its prompt and images,

28:13 runs them through PaliGemma, caches all the key and value matrices,

28:18 and then runs the action expert

28:20 to iteratively denoise random trajectories into final paths.

28:25 The robot then follows these paths for a few steps,

28:28 and the process is repeated, controlling the robot to achieve the task at hand.

28:34 Since Pi Zero was first demoed in October of 2024,

28:37 the physical intelligence team has made various

28:39 improvements to their models and training approach,

28:43 but their core VLA architecture,

28:45 using a tightly coupled multimodal LLM with a flow matching action expert,

28:49 has remained unchanged.

28:52 Looking back on the RT-2 Taylor Swift Coke can demo in 2023,

28:56 it's incredible to see how far VLA models have come.

29:00 And what's perhaps even more impressive to me is that the physical

29:04 intelligence team had the foresight to realize

29:07 what this unimpressive demo really meant,

29:10 that large language models could be trained to be robots,

29:14 potentially leveraging the full knowledge of the internet into robot brains.

29:20 Now, as impressive as these demos are, they're still demos.

29:24 In 1995, a team from Carnegie Mellon demonstrated a self-driving system,

29:28 Ralph, that drove across the US at 98.2% autonomously.

29:34 This clearly did not mean that self-driving cars were around the corner.

29:38 [music]

29:38 And the generation of self-driving cars we have today works very differently.

29:42 And interestingly, there's a different

29:44 paradigm emerging for building robot brains, broadly known as world models,

29:50 that actually do not use large language models as a backbone.

29:54 Yann LeCun, AI pioneer and long-time chief AI scientist at Meta,

29:59 recently left his role at Meta to start a new venture focused on world models.

30:04 Yann was kind enough to chat with us about it

30:07 and wasn't shy about giving his opinion on VLA models.

30:11 What's your expectation here?

30:12 Do you think JEPA-based approaches will eventually overtake VLA approaches?

30:16 Oh, absolutely.

30:17 Yeah, VLA are doomed.

30:18 I mean, they they basically don't work really well.

30:20 Okay.

30:21 I mean, Next time, we'll dig into Yann's approach.

30:27 If you enjoyed this video, check out the companion poster.

30:31 The poster walks through the full Pi

30:32 Zero architecture with helpful descriptions along the way.

30:36 Fitting everything on screen was a huge challenge when animating this video.

30:40 And the large format of the poster

30:42 is perfect for getting everything into one place.

30:45 The poster is printed on high-quality large format photo

30:48 paper with genuine Canon inks for excellent colors and details.

30:53 For a limited time, you can get a discount on the poster

30:55 when bundled with the Welch Labs' Illustrated Guide to AI using code VLA.

31:01 Speaking of the Welch Labs' Illustrated Guide to AI,

31:03 I'm very excited to announce that international

31:05 shipping is now available in these nine countries.

31:08 And we're planning to expand to these countries next.

31:12 I know this has taken a really long time.

31:14 Thank you for your patience.

31:16 A ton of you have emailed us and joined our international shipping waitlist.

31:20 Today, all of our books are printed in the US.

31:23 We have a great relationship with our printer and the quality is outstanding.

31:27 We've received a bunch of nice feedback about this.

31:30 This viewer told us that the construction quality is the best they've ever seen.

31:35 However, our print costs are fairly high

31:37 and being self-published makes international logistics a real challenge.

31:41 Until very recently, my family and I packed all the books ourselves.

31:46 Here's 7,000 lbs of books getting dropped off

31:48 on the street in front of my house late last year.

31:51 We use high-quality boxes and corner protectors

31:53 to make sure your book arrives in pristine condition.

31:56 The other packaging options we've tried it

31:58 just don't protect this heavy book very well.

32:01 Here's some outgoing shipments and a van we rent sometimes for post office runs.

32:06 Here's part of another shipment that was delivered during a snowstorm this year.

32:10 And here's some more books heading to the post office in my family's SUV.

32:14 It's definitely been an adventure.

32:16 Early this year, we started looking at ways to improve and scale our process.

32:20 We've had some interesting calls with publishers,

32:23 but the deals we've seen so far either significantly

32:25 reduce print quality or cut too deeply into our margins,

32:29 effectively introducing one or two layers of middlemen to our supply chain.

32:34 So, we've decided to stay self-published for now.

32:37 We did find a great local packing logistics partner who now,

32:40 thankfully, is handling fulfillment.

32:43 They also ship enough volume to get some nice international rate discounts.

32:47 This is what has allowed us to start to tackle international shipping.

32:51 Starting today, we're offering flat rate shipping to Canada, Mexico,

32:54 the UK, Ireland, Germany, France, the Netherlands, Italy, and Belgium.

32:59 And the flat rate includes all relevant VAT, GST, and duties.

33:04 We chose these countries by cross-referencing

33:06 the countries with the highest demand

33:07 on our waitlist and where we're able to ship without exorbitant shipping costs.

33:12 Next, we're looking at expanding to India, Australia, New Zealand,

33:16 Singapore, Japan, South Korea, Hong Kong, Thailand, and Malaysia.

33:21 Although higher shipping costs due to a global increase in fuel

33:24 prices is making this a bit more challenging than we expected.

33:28 I'm really happy that we now have some international options,

33:31 but the price we need to charge to cover

33:33 printing and shipping is still higher than I would like.

33:36 If we continue to see strong demand,

33:38 this will allow us to invest in larger print runs, bringing down printing cost.

33:43 And we're even looking at doing some

33:44 of our printing regionally, starting in Europe.

33:47 This would significantly bring down our European

33:49 shipping costs and allow for lower prices.

33:53 Sometimes, I really question if this is all

33:55 just crazy and really a distraction from making videos,

33:58 especially on days when 7,000 lbs of books show up at my house.

34:03 However, at the end of the day, the mission of Welch Labs is to make

34:06 these complex topics as understandable as possible,

34:09 and books are a big part of that mission.

34:12 One supporter on Patreon, Lauren Steely,

34:14 put this really nicely when talking about the book.

34:17 It's not just a condensed version of the videos.

34:20 The book actually adds so much more

34:22 detail that the videos couldn't possibly contain.

34:26 Finally, a big thank you to all the readers

34:28 who have helped find errors and made suggestions for improvements.

34:32 These readers are listed on the credits page of the latest version of the book.

34:36 And we published an errata at welchlabs.com/ai-book.

34:41 I especially want to thank Robert Blumoff.

34:44 He's been incredibly meticulous at rooting out little issues in the book

34:48 and has even made his own perceptron machine and build guide.

34:51 I'll include a link in the description below.

34:54 Thank you so much for your patience and to everyone who's bought a book.

34:57 It really helps the business work and means a lot to us.

35:00 Thank you.