Meta's Display Glasses and the AR/VR Continuum | Sharp Tech with Ben Thompson

0:00 shift gears entirely.

0:02 Let's go to the meta display glasses.

0:04 You're not wearing the meta display glasses, unfortunately,

0:08 here on the Oh, he's Wait, he's going off screen.

0:10 Is he going to put them on?

0:12 Going to Of course, he has to keep his normal glasses on to wear.

0:15 So, they they do work uh on top of [laughter] double glass it.

0:21 Great.

0:22 Double glass it for [laughter] double glass it for the segment here.

0:26 So, you hit me up last week and we were going to discuss it on the podcast.

0:31 We didn't end up discussing it on the podcast,

0:33 but you hit me up saying you had a great experience with the Meta Display.

0:38 And I just love when you love technology.

0:42 So, take me inside the Meta Display.

0:45 [laughter] You look so ridiculous.

0:47 We'll have to make a video out of this and put it on YouTube for people.

0:51 They keep falling down.

0:52 Yeah.

0:52 Um

0:53 Yeah.

0:53 I don't think you're gonna be able to sustain it,

0:55 but um what did you love about them?

0:57 I've used them I've used them for a long time this way.

1:00 Great.

1:00 Okay.

1:01 Well, why do you like these glasses so much and what's

1:04 different from your experience with Orion roughly nine months ago?

1:09 Yeah.

1:09 So, this I mean I was not able to attend

1:11 I this I went to that YouTube event last year.

1:14 It was the same week um as med.

1:18 Yeah.

1:18 Yeah.

1:18 There was kind of rumored they were going to announce these glasses.

1:20 But part of my thinking at the time was I already used Orion.

1:23 This is going to be like derivative of that.

1:25 So I kind of know what it is.

1:27 So I didn't use them at the time.

1:28 I'm like, "Oh, you know,

1:29 I I'll get a pair and try them then." Turned out to get a pair,

1:32 you had to like go to a store and do this whole thing.

1:35 And the nearest store for me was like in Chicago, like downtown Chicago.

1:40 And the near the sign up was like two months away.

1:45 I'm like, I cross that bridge when we get there.

1:49 Yeah, [laughter] this is not going to happen.

1:50 Um, so yeah, so I hadn't tried them and I'm actually glad

1:57 because I happen to be in San Francisco and so I ping someone.

2:00 I'm like, I would really like to try the display.

2:04 Um, you know, I've never been able to get a sign up XYZ.

2:07 Um, can I stop by and buy try it out?

2:09 And not only was I able to try them out, but also I was able to try Orion again.

2:13 So, use them side by side like take one off,

2:16 put one on, take one off, put one on.

2:18 Uh, and I thought that was the single most uh interesting aspect.

2:23 So, just to make clear the differences,

2:25 and I feel like I might have not done this enough in the update.

2:27 Um, to make clear the difference, Orion, they announced,

2:32 I think it was, it's been a while now,

2:34 year and a half, was it 2024, where where Mark Zuckerberg wore them on stage.

2:37 Uh these are quite thick glasses that you put on and there it's unbelievable.

2:44 Like it's it's pretty like they're totally clear

2:47 glasses that then a a display is projected.

2:50 It's on both eyes.

2:51 It's it's pretty bright.

2:53 Um and it's like a computer in your glasses frames.

2:59 It's like a vision pro but

3:01 it's clear.

3:03 There's no pass through.

3:04 You're actually looking through glass.

3:06 Mhm.

3:07 And so you have this and it does so and it's

3:09 the same as the vision pro with the eye tracking.

3:11 So the eye tracking so you look around you look

3:13 at an object and then you select it by doing your fingers together.

3:16 And this is the other big thing part of the reason

3:19 they can do it with just a pair of glasses.

3:20 And again we're being generous by saying just pair of glasses.

3:22 They are pretty thick.

3:24 There is a brick that has the compute like like a vision pro but no cord.

3:27 It's all wireless.

3:28 Um the vision pro has cameras in the bottom of it that looks at your hands.

3:35 Your hands have to be visible to the vision

3:37 pro to actually control the vision pro.

3:39 Uh Meta has this thing called the neural band uh which you

3:43 know I'm wearing one right now with my uh glasses over glasses.

3:46 And the neural band it has sensors all along the bottom and along

3:50 the top that feels your muscles moving and like the electrical signals.

3:54 I'm not sure exactly how it works.

3:55 I should probably be be more exact on that.

3:57 But what it means is you can do the same thing in the Vision Pro.

3:59 like you touch two fingers together

4:01 and the vision pro sees that and registers it.

4:07 The Meta Neural Band feels it and registers it.

4:11 And so this has a lot of advantages because

4:12 you could have your hand in your pocket for example.

4:14 You your hand could be behind your back.

4:16 It could be pitch black and and you still have control.

4:19 Can I say as a Vision Pro power user,

4:22 one of the only Vision Pro power users on the face of the earth,

4:26 one issue that I run into frequently with the Vision

4:28 Pro is I'll be watching the Vision Pro,

4:31 watching a basketball game in bed and in the dark.

4:36 Um, it is very hard to pick up the finger gestures and whatnot,

4:40 the pinching gestures.

4:42 So I'll have to turn like my iPhone light on fertively

4:45 next to my wife and and use it that way.

4:48 This is automatic.

4:50 So I need I need a little neural band on the wrist so

4:53 it could just pick up my like wrist gestures or whatever or muscle movements.

4:58 This whole thing's awesome to be clear,

5:00 but because it's still very early tech, it's pretty low resolution.

5:04 Um, and what was interesting, I actually tried a high resolution one.

5:08 It was still amazing because the whole world's in high resolution, right?

5:11 So, it's like whatever is in your in your in your eye

5:13 is it's not like it's going to be that clear.

5:15 What was interesting about about what I I this made me happy cuz I

5:20 feel like I my intuition with the Orion

5:23 was actually ended up being quite accurate.

5:25 I just didn't realize why.

5:29 Okay.

5:28 In that I did not like the Orion software.

5:32 It felt like Quest software.

5:34 It's like stuff in front of you.

5:35 There's like an Instagram app and there was games.

5:37 Too much noise.

5:39 Well, it's just like I couldn't figure out quite why I didn't like it,

5:41 but I knew it wasn't right.

5:43 What I said at the time was my favorite Orion experience was you're wearing

5:48 it and the person I was dubbing with, so I had a step out.

5:50 They stepped out.

5:51 They called me and a notification popped up.

5:54 I clicked my fingers together, the neural band,

5:57 and now I was on a call with them.

5:58 I think it might even been a video call.

5:59 So, I could see I could see them and be talking to them.

6:03 And what was so compelling about that was the point

6:08 of glasses is I want to see the real world.

6:12 That UI only appeared at the exact moment that it needed to be there.

6:17 I could interact with it and then it went away when it was gone.

6:20 This sort of idea of just in [clears throat] time UI.

6:22 I've been writing about this a lot about where I think UI interfaces are going

6:26 and and it was quering with Orion that was the best

6:29 part of it and I didn't love the other but I

6:31 kind of gave them an excuse in that it's super early

6:33 they've been they're just getting the hardware to work no surprise

6:37 Orion if I recall was just an amazing piece

6:40 of hardware generally and kind of a technological marvel.

6:43 So you fast forward to to the display glasses.

6:45 The display glasses this is how they're different than Orion.

6:48 So number one, there's only one kind of square

6:51 of display and it's in the right right display.

6:54 It was uh and so and it kind of like shoots it onto the glasses

6:57 whereas the uh Orion uses this I think it's silk and carbide like glasses.

7:01 It's like it's like it's crazy stuff.

7:04 It like filters through the glass and like makes an image.

7:06 In this case, it's kind of like it's like spraying

7:08 the image onto the onto the lens or something like that.

7:10 You don't see it from the outside.

7:12 Um but it it it's less impressive technology.

7:17 it is somewhat higher resolution.

7:19 Um it's it wasn't as bright.

7:21 Um and so it's just sort of there in your vision.

7:24 And whereas Orion was a true like like V Apple Vision Pro,

7:31 if you're using pass through,

7:32 you can like put something in the living room and put something

7:35 over there and put something there and it's it's it stays there, right?

7:38 It's like you're in a virtual environment and that thing you put there is there.

7:41 Display is not like that.

7:42 It's just always in the corner of your eye.

7:44 So it's there.

7:45 It has no relation or conception of the outside world in terms of that display.

7:50 It's not like I can pin something in a place and I go back and it's there.

7:53 It's just no just something that just are always in my eye.

7:56 Sorry, I'm pointing to it.

7:57 You can't see, [laughter] but it's not but it's it's it's there right now.

8:00 It's sort of and you can see my eyes

8:01 sort of move down because it's it's a little down.

8:03 It's not in the center of my vision.

8:04 It's a little down and down into the corner.

8:07 So, uh the but most of the time it's not there at all.

8:15 Mhm.

8:14 Basically what it delivers is the best part of the Orion experience which

8:20 is I could get a message right now so it's nothing there and boom

8:24 suddenly a notification comes up that I can act on it or I

8:28 can dismiss it and go on with my day and then it's gone and

8:31 otherwise you're just present in the physical world.

8:34 Otherwise I'm just present in the physical world.

8:35 And then number two you actually do navigation with the neural link.

8:40 So like you do so I'm going to turn it on here.

8:42 I'm double clicking my fingers together.

8:43 turn on and I move my thumb around to basically like swipe.

8:47 Um, one thing's very funny I realized is I always

8:52 like there'll be these demos and they're like swipe right,

8:54 swipe left and I always am not sure what

8:56 they mean by swipe right is a grab or whatever.

9:00 Okay, because we're like the post or pre-Tinder

9:02 generation just there's a whole vocabulary around [laughter]

9:07 I just I don't know it.

9:08 I don't what is what is swipe right?

9:11 Which direction [laughter] is it?

9:13 Everyone younger than me, they immediately neither of us ever swiping.

9:16 Yeah.

9:17 Um, so I thought that was very funny.

9:19 Um, which I related.

9:20 I should say fortunately, by the way, neither of us were ever swiping.

9:24 Um, but okay.

9:24 So I I'm being exact.

9:25 Do you see my but this is the key thing.

9:28 I can be looking.

9:29 So now I was showing you that I was doing it.

9:33 I'm like I am also swiping right now and I'm looking directly at you.

9:37 There always is the TV that's right there.

9:38 Yeah.

9:38 I'm not seeing any of it.

9:40 Wow.

9:41 Why?

9:41 because it's I my hand can be doing whatever it

9:45 wants and my eyes don't need to be paying attention.

9:52 M to me the difference in you the vision

9:55 eye control which Orion is just like the vision

9:58 pro versus using the neural link for everything is

10:03 dramat I can't overstate what a difference this is.

10:07 I if I was talking to someone wearing Orion,

10:09 I had to it was and I could feel this so abruptly.

10:14 I had to disconnect from the conversation,

10:19 interact with Orion, and then reconnect to the conversation.

10:23 I could not do both at the same time.

10:25 It was just not not possible.

10:27 And even though I did I moved my head here, in this case,

10:30 I wouldn't have needed to move my head

10:32 because I could have the display right there.

10:33 But it was a total mental shift.

10:36 Whereas I can be talking to you with display and it's it's

10:41 in my peripheral vision and I can be at the same time.

10:44 You can wear those bad boys for every single

10:46 podcast we record and nobody will know the difference.

10:49 You'll be fine.

10:50 Um what comes to mind when you lay all that out is that the Orion

10:55 might as well be a vision pro separating

10:58 you from the rest of the physical world.

11:00 The Orion is awesome.

11:02 and also I think a technological dead end at least

11:05 for the for the next few years and why

11:07 because it's sort of halfway between what the display glasses

11:11 are and the high-end experience that vision pro is offering.

11:15 Yes.

11:15 To me the immersion is is

11:20 the the glasses versus headset is an implementation detail.

11:24 Immersion is a function of the degree of focus it takes from me

11:29 to interact [clears throat] with it.

11:30 If there is if you're the eye control is amazing.

11:34 It's so cool.

11:35 This is part of why putting on a vision pro still is so fun.

11:38 It's like this is so cool.

11:39 I like look like I'm navigating with my eyes.

11:41 It's amazing.

11:42 It's so cool.

11:43 It is also a modal shift.

11:46 And to me, if I'm going to be forced

11:50 to disconnect from my environment to engage with a display,

11:56 I rather have a high resolution, high brightness display,

12:00 which I'm going to get from a headset.

12:02 I would just

12:04 The Vision Pro is just better than an Orion

12:07 in every way in terms of actual like an immersive experience.

12:12 Yeah.

12:12 on the other axes is being having something that's with me

12:18 that doesn't that is beneficial but doesn't

12:21 require context switching for me mentally

12:25 display is better than Orion in every way when it comes to that I'd rather

12:30 have a smaller display that shows stuff just in time than have this this immersy

12:36 shift and so there's a Pareto frontier

12:39 which is the best immersive experience And then

12:43 the best augmented experience and there's a curve

12:46 along it and Orion is inside that curve.

12:49 It's not on it's not on the edge.

12:51 And uh and so yeah, which that's why I'm so glad I

12:54 it worked out this way that I could use them back to back.

12:57 I found Orion unpleasant because of this aspect.

13:01 It like if I'm shifting it's low resolution.

13:04 It's not that that great.

13:05 What's the point?

13:06 Why is it taking me out of the conversations that I'm having?

13:09 Yeah.

13:10 Well, and it's interesting, too, because from Apple's standpoint,

13:14 one of my experiences as a Vision Pro user

13:17 is like the pass through vision is actually incredible,

13:20 like what they do with the cameras,

13:22 but it's not useful.

13:23 [laughter]

13:23 Well, no, it's useful in terms of not stepping on your MacBook

13:27 Air next to your Were you wearing a Vision Pro?

13:29 No, I wasn't wearing a Vision Pro.

13:30 I can't even blame the Vision Pro.

13:32 Can't blame my kids.

13:33 This one's entirely on me.

13:35 But yeah, it is like that's the extent to which it's useful.

13:37 It's useful so that I don't hurt myself if I'm walking around.

13:42 Um, but like I've never and that's the argument for Orion if you want

13:45 to do like for example there's like a a use case

13:47 like oh say you're working on something and you want

13:50 to have the instructions your field like cooking for example, right?

13:52 Joanna Stern did that video where she's like cooking

13:55 with Vision Pro like the first week or whatever.

13:57 Super funny.

13:58 O'Ride is way better for that just

14:00 because you it's better for seeing the outside

14:02 world and you get you can get enough information so it's not a worthless device.

14:06 Obviously, you can't even buy one anyway,

14:07 but the um you know, and you could see,

14:11 but also I could get get the recipes in in display like you know I

14:16 and I think if you I raised this with meta their push back was well

14:19 it's going to be important for the AI to know what you're looking at and yeah

14:25 by all means track my eyes to know what I'm looking at but the moment

14:29 I need my eyes for navigation uh I'd rather just have a vision pro.

14:34 Yeah, but they have the rather have a vision.

14:37 What do you mean?

14:38 This is if if I'm gonna be if I'm you're gonna force immersion on me,

14:43 then give me a better display than what gives me.

14:45 Totally.

14:46 Totally.

14:46 The Neurolink is awesome.

14:47 It's like that this is this is such a winner.

14:51 Uh it's so this is so clearly the right control for wearables,

14:57 for the outside world.

14:58 It's a total win.

15:00 it like and you could see this being like you could imagine just

15:03 slapping a watch on top of this so it just looks like a watch.

15:06 Um you know right now even just looking wearing

15:08 this so no one's going to question me like wearing this.

15:10 It's not approved at all.

15:11 It looks normal and you know here let me try take off my glasses.

15:16 You can give me evaluation.

15:18 How how do how do they how does the display look?

15:20 They look actually really normal.

15:23 You put them on on a you look at them closely they're like oh

15:26 that's kind of bulky to wear your [laughter] computer.

15:28 It's not like Orion where it's like what?

15:31 That's weird, right?

15:32 Well, yeah, it the Orion um the pictures I saw,

15:36 they definitely just look like a computing device more than glasses.

15:39 And I remember Meta talking about how stylish they looked,

15:42 and I was like, well, they don't look that stylish.

15:44 Um but those look mostly normal, like normal big rimmed glasses.

15:49 Um, and with Apple and future VR products,

15:53 lean into the immersion and don't worry about trying to split

15:57 the difference and have people be in the physical world.

16:00 It really accentuates everything about the Vision Pro.

16:02 That's a problem, right?

16:03 The um uh

16:06 Oh, for some reason I I started Spotify.

16:09 I have the goat playing uh on [laughter] my display glasses.

16:13 This is great.

16:13 I saw your bed golf.

16:14 It's all going in the YouTube video.

16:15 [laughter] Uh yeah, fantastic.

16:18 I was just like, "What the hell?

16:19 Did Ben just have a stroke?" [laughter] I was so confused.

16:22 So, you're not entirely present in the meta display glasses.

16:26 No, there are pitfalls, but

16:28 it's worth noting like I'm very single track in general.

16:31 Like I like [laughter] But that means I'm very aware of what I'm mode switching.

16:34 Um, so I almost forgot I was talking about Oh,

16:37 Vision Pro, dump the stupid outside eyes.

16:42 Yeah.

16:41 Uh, like Well, here's the deal.

16:43 like at you've heard the next Quest is going

16:47 to um be it's going to be have Apple Vision Pro

16:52 I think resolution high screen and stuff like that that could

16:55 be a really compelling product because I want the neural

16:59 uh so my to your point eye control is fine

17:02 in a in a VR experience but just capturing the clicks and stuff

17:05 like that the neuroban's great uh have the minimum necessary external cameras

17:10 so that you can walk around with it on without bodily injury.

17:14 Yeah, that should be the goal.

17:15 And then keep the cost down by and make it lighter.

17:18 Not making it.

17:19 Get rid of all these extra cameras.

17:21 Yeah.

17:22 Aluminum and glass.

17:24 It's unbelievable.

17:25 What?

17:26 Right.

17:26 Well, there there would be a market for it as an entertainment device.

17:30 And maybe that's what the Quest will take advantage of if

17:32 it were $1,000 and not $3,500 or some of the cost.

17:38 Uh well that too that too of course are

17:41 did you hear did they like disband the vision for team crusade?

17:44 Um yeah I actually German came out and clarified that it has

17:49 not been abandoned entirely and that there are still people working on it.

17:53 So after an initial report that everyone who had been working

17:56 on the vision pro had been reassigned to a smart glasses project at Apple,

18:01 he came out and softened that reporting which

18:04 was heartening for me personally and probably nobody else.

18:07 I I Yeah, I wouldn't I wouldn't uh I wouldn't count on that.

18:10 I think what we're shaping up for though, speaking of court cases,

18:13 um the mother of all patent lawsuits.

18:17 How so?

18:19 The risk this the neural band is so obviously the way

18:23 forward that Apple's going to have to rip it off.

18:26 Um and uh and that's going to be one hell of a lawsuit.

18:30 Like I said, my Vision Pro would work much better

18:32 if it just had the neural band instead of the gestures.

18:35 Although the gestures are pretty delightful

18:37 for AR, I just firmly believe for AR, the eye control does not work.

18:43 It ju it it just doesn't work.

18:45 And that's why the neuroban does

18:47 and for what people could potentially use just in time

18:52 AI for in the future to years ambient computing.

18:56 Yeah, the map like the map walking map thing's awesome.

18:59 like like you're walking and it pops up and says

19:03 turn here or it can show or it can show

19:05 you the landmark or whatever the the other thing

19:06 by the way is this is absolutely positively not an app platform.

19:12 The worst aspects of this are when I felt like I was using apps.

19:16 So like if WhatsApp messages would pop up and you

19:18 click like uh I think you want to like transcribe something,

19:22 you could feel yourself clicking it and then going

19:25 into the app instead of it just like happening

19:27 there like that needs to be smoothed over so

19:30 everything is sort of in place and in time.

19:31 You don't want to feel like you're going in like every time you you could go

19:34 into a list of apps and go into it and then you had to get out of it.

19:37 All that sucked.

19:37 I that's what I didn't understand when

19:40 they first debuted this product a year ago.

19:42 I didn't really understand the potential utility of like

19:47 being able to chat on Facebook and Instagram

19:50 direct messages and WhatsApp and whatnot on your glasses

19:54 like that doesn't really appeal to me.

19:56 The phone is a better surface area for that.

19:58 Yeah, but even with a phone, so I am the world's biggest like lock screen user,

20:02 like so I I use the lock screen all the time to like just catch

20:06 see what's going on, like make sure I

20:08 have an awareness of things that's going on.

20:10 Um, so that's almost like a completely different UI

20:14 than actually going into an app and interacting with stuff.

20:17 The other thing though is this this may be a meta advantage.

20:21 It's obviously AI is the answer for everything.

20:24 You like the AI should generate stuff.

20:25 The AI it should all be just in time.

20:27 It should be showing stuff when you need it, as you need it.

20:30 It should be having all context of your life so it

20:32 can bring up stuff at the right time, the right place.

20:35 This is where, yes, knowing what you're seeing, I can see the value there.

20:38 And I think it's to their advantage that they

20:41 do not have an app platform that they're

20:43 even tempted to try to leverage into this because

20:47 apps are the wrong paradigm that okay, using an app is a deliberative action.

20:52 You go there, you click the app, you go into it, you do something.

20:56 That's fine in a VR experience where you're using your eyes and you're focused.

21:00 You're going into something.

21:01 So, I have no problem with apps in VR.

21:02 Like, I think that that can make sense.

21:04 For AR, it should all be ambient.

21:07 It should just all be there as you need it,

21:09 when you need it, and not there when you don't.

21:11 That is going that is going to mean AI.

21:14 And that means that's why it's important they rebuilt their AI team.

21:20 They need to rediscover the passion for glasses because they need

21:24 to be tied into that AI team in a meaningful way.

21:28 And it's a reason to be worried about Apple.

21:30 Not just that they've focused on this eye thing, which I think is great for VR,

21:35 but not for AR, but also they're not building their own AI.

21:40 Um, and I think that's going to be critical for this going forward

21:43 to the extent that's sort of elemental

21:45 to succeeding in this particular paradigm.

21:48 Can you just like do Gemini and and and do stuff on top or do you

21:51 need to have like the model at a deep

21:54 level being trained for these particular use cases?

21:56 We'll see.

21:57 Uh but I think by and large it feels

21:59 like over time models more and more stuff's getting

22:02 baked into the models first in pre in post-

22:04 trainining and RL but also even back into pre-training.

22:07 Is Apple going to be able to like the whole the world

22:10 is moving towards more integration in terms of harness and model?

22:15 Apple's bet is the opposite that we can use any model or use Gemini for now.

22:19 We can use something else later and we

22:20 can build all our infrastructure on top of that.

22:23 I'm not sure that's that's going to provide the best experience.

22:27 We'll see.

22:27 Well, and in terms of the best experience and the best use cases

22:31 for a product like this, my question is I mean my I guess

22:36 my assumption would be that it's going to take 5 to 10 years

22:40 to pull some of the most useful apps and sort of use cases into existence.

22:47 Like people aren't just going to use these for directions and say, "Oh god,

22:50 it's definitely worth walking around every day with $1,000

22:54 glasses on that look kind of stupid." Um,

22:57 but I am fully open to the possibility that there

23:00 are use cases that just don't exist yet and [snorts]

23:02 that will exist as AI becomes more performant and we

23:06 all kind of interact with these on a more regular basis.

23:09 My question is do you think Meta is committed to waiting it out for the if

23:15 it takes 5 to 10 years like how committed are they to the platform?

23:21 I mean, they're still they lost another

23:24 gazillion dollars last quarter, so there's that.

23:26 They've also talked about laying off teams.

23:28 They I think they're I think they're very focused to this angle, the AR angle.

23:33 I'd be much more nervous about Quest and sort of like the VR angle.

23:37 That seems to be what they're shifting away from, which I think is is okay.

23:42 I think these my feeling is VR and AR are less adjacent than everyone assumed.

23:48 I think people sort of thought they're converging.

23:51 I feel like they're diverging.

23:53 I think they're meaningfully different experiences and Apple has a better VR

23:58 experience in my estimation and I think the VR experience on AR

24:04 it the VR experience is very isolating like I'm

24:07 watched the Masters a couple weeks ago in my living

24:09 room surrounded by my family in the Vision Pro

24:12 and the Masters app in the Vision Pro was phenomenal.

24:16 [laughter] But I felt I felt horrible cuz I cuz

24:19 I could see with Apple's cameras looking at my family members

24:22 and I'm like look I can't do this for more

24:24 than 5 minutes because I feel like a horrible family member.

24:27 Um so some sort of AR solution where

24:30 you're actually present in the physical world and not

24:32 like completely isolated from everyone around you makes

24:36 me feel better about the future of humanity.

24:38 So, I hope the distinction is real and I hope the AR wins