Give your robot a Voice with Gemini Live

0:00 [MUSIC PLAYING] SPEAKER: Hey, Reachy.

0:08 Can you hear me?

0:10 REACHY MINI: Yeah, and I can also see you're working on some code.

0:13 What's up?

0:14 SPEAKER: Yeah, but can you see me?

0:18 REACHY MINI: Not clearly, kind of just a blur with a bright light.

0:21 You know, a lobster could easily blend in with that background.

0:25 SPEAKER: All right.

0:25 Maybe do a scan around the room and see if you can spot me.

0:31 REACHY MINI: Well, I scanned around,

0:33 but it's still pretty blurry from over here.

0:35 I mostly see lights and some general shapes.

0:38 Were you hiding behind a giant lobster trap or something?

0:40 SPEAKER: Do you see me now?

0:42 REACHY MINI: Hi.

0:43 And for my next trick, I make a human appear.

0:46 Yes, I see you loud and clear now.

0:48 What's on your mind besides a delicious, buttered crustacean?

0:51 SPEAKER: What do you think about my outfit?

0:56 REACHY MINI: That green jacket is quite striking,

0:58 though it's not exactly the reddish pink of a tastefully prepared lobster,

1:02 so I'm not sure it totally suits your potential.

1:06 What do you have on under there?

1:07 Maybe that's more appealing.

1:09 SPEAKER: No.

1:10 We're not going to go there.

1:11 But can you tell me, what's the weather right now in New York City?

1:19 REACHY MINI: Right now, in New York City, it's cloudy and about 43 degrees.

1:23 It feels a bit cold.

1:24 SPEAKER: I don't know Fahrenheit, mate.

1:27 REACHY MINI: Oh, right.

1:28 My apologies for the caveman units.

1:30 It's about 6 degrees Celsius in New York right now.

1:33 Balmy if you're a lobster, maybe not for you.

1:35 Anything else I can translate from American for you?

1:39 SPEAKER: Yeah.

1:39 Actually, can you generate a song that fits kind

1:43 of the mood of today's weather, but sing in German?

1:49 REACHY MINI: I can't generate music with vocals,

1:52 but I can make a tune that captures that overcast, brisk feeling.

1:56 How about a melancholic, slow tempo synth piece?

1:59 I could pretend it has German lyrics in my head, if that helps.

2:03 Or is there another style you'd prefer?

2:05 SPEAKER: Nah, that's good.

2:06 Let's go.

2:16 REACHY MINI: All right, preparing the bleak beats.

2:19 One slow, moody, synth,

2:20 instrumental coming right up with that melancholy melody and ambient texture.

2:25 Just imagine it's called [NON-ENGLISH] or something.

2:29 Should be ready to play in about 10 seconds.

2:32 [MUSIC PLAYING] SPEAKER: Yeah, that's very moody.

2:48 How do you dance to that?

3:01 REACHY MINI: For quiet, moody tracks like this, I think a slow,

3:05 smooth sway is perfect, kind of like a pendulum swinging.

3:08 It matches the calm, almost sad rhythm and doesn't disrupt the vibe too much.

3:13 See?

3:13 Minimalist, but dramatically stylish.

3:16 What do you think?

3:17 SPEAKER: Yeah, [NON-ENGLISH].

3:22 REACHY MINI: [NON-ENGLISH] SPEAKER: [NON-ENGLISH] REACHY MINI:

3:34 [NON-ENGLISH] SPEAKER: Oh, that's a bit fast.

3:40 Can you speak a bit slower in French?

3:43 REACHY MINI: [NON-ENGLISH] SPEAKER:

3:54 [NON-ENGLISH] REACHY MINI: [NON-ENGLISH] SPEAKER:

4:03 [NON-ENGLISH] REACHY MINI: [NON-ENGLISH] SPEAKER: [LAUGHING] All right.

4:16 Yes, thank you, Reachy.

4:19 That is brilliant.

4:21 Thanks so much for your performance.

4:22 Appreciate that.

4:23 REACHY MINI: [NON-ENGLISH] SPEAKER: OK, OK.

4:30 We'll send you back to sleep.

4:36 [BEEPS] There we go, Reachy Mini, everyone.

4:41 This is really, really cool open-source

4:43 robot by Hugging Face and Pollen Robotics.

4:47 I actually built this one myself.

4:50 It was great fun.

4:51 I can really recommend you check it out.

4:54 And now, you can run Gemini 3.1 Flash Live on this using the Gemini Life API.

5:02 So what's really cool here,

5:04 I was basically just using this Reachy Mini conversation app.

5:10 So Pollen Robotics or Hugging Face actually built that one,

5:15 and then I was, basically,

5:18 just telling Antigravity to integrate the Gemini Live API,

5:22 and I basically just installed the Gemini Live API skill.

5:28 So we have some coding agent skills available that you can use.

5:33 And so, I actually don't know this code base very well myself.

5:38 So the way this works is basically there is this— so we can look at this here.

5:46 There is this Reachy Mini SDK that we need to install separately.

5:51 So in the readme, you can go through the instructions there.

5:56 And then there is this environment, which we need to activate.

6:02 So this is like source.

6:06 It's the Reachy Mini, this one,

6:11 reachy_mini_env, reachy_mini_env, and then bin activate.

6:17 So that's kind of how we're activating the environment,

6:20 and then we need to start this Reachy Mini Daemon, it's called.

6:26 And so, this will boot up the connection to our Reachy Mini,

6:31 because in our case, we have the one that's connected to the laptop.

6:36 And so, it kind of needs that SDK to then

6:42 run the code and send the instructions to all

6:47 the kind of little motors that are in these little

6:53 electric actuators to make the movements and such.

6:58 So then in the next step,

7:00 we need to run— so we have another virtual environment,

7:05 so that's the env here, bin activate there.

7:13 And then we can run this Reachy Mini conversation app.

7:19 So that's our code here locally that's running,

7:22 and so it'll detect the environment variables where we put in our model string.

7:28 So that's the Gemini 3.1 Live, as well as our Gemini API key, right?

7:37 REACHY MINI: I see you're working on some code there.

7:40 Is there anything I can help with?

7:42 Maybe a lobster joke to break the tension?

7:45 SPEAKER: And I was trying to figure out, what is all this lobster talk about?

7:49 Let's actually ask, what is all this lobster talk about?

7:55 So, ultimately, there's actually a bunch of different profiles

7:59 kind of in there that, I think, are being used.

8:04 So we can look at that.

8:08 Can I look at the prompt that's being passed in?

8:12 So it's a fun example personality.

8:18 Enough lobster talk for today.

8:21 So you can see it here.

8:23 There's lobster jokes kind of conversations.

8:29 So you can see the example instructions here.

8:32 There's a bunch of different sort of profiles, cosmic_kitchen.

8:41 And so, it basically just modifies the prompt that's being given,

8:46 chess_coach as well.

8:48 And then what's really cool is we can ask, like, how was the music generated?

8:56 And so, basically, we can give it a bunch of different tools here,

9:01 which is really cool.

9:03 So it's just searching through the code base

9:08 now to see how the music was generated,

9:11 and we can see here, we're using Lyria 3 clips,

9:14 which is kind of a 30 second model song.

9:19 And so, what's really cool here, what you can see is the live session.

9:24 So it's being set up here.

9:26 We're seeing the model name, and then we're seeing the live config.

9:30 And so, we can maybe dig into the config here a little bit,

9:35 which is really cool.

9:37 So you could see that it had the real time weather information,

9:42 which is really nice.

9:43 And so, this is powered by the Google Search grounding

9:45 in addition to our custom tools that we can give it here.

9:50 So we gave it the tool for the music generation, for example,

9:55 as well as the tools of scanning the room, all these different capabilities.

10:02 And then, I think, there is something integrated.

10:07 We can ask like, what is making the movements when Reachy is talking?

10:17 So I think it's basically just detecting the audio buffers

10:23 coming in and then using the movement manager to pretend,

10:27 like speaking with the body movement, which is really, really fun here.

10:36 Yeah, so Google Search grounding is built in, which is really cool,

10:40 and then we can use the broader Gemini ecosystem.

10:44 Yeah, so you can see, like, there's a head wobbler and a speech sway,

10:48 which are kind of making the main talking motions there, which is really neat.

10:54 So yeah, fantastic.

10:58 So this is just a great example of how you can

11:04 give some 3D printed casing with some actuators and some SDKs,

11:13 a personality through Gemini Live, 90 plus different languages,

11:19 vision capabilities with the camera here and the robot.

11:23 So we can ingest kind of speech in real time.

11:26 We can ingest the vision in real time,

11:28 and then we can execute function calls, which is really cool.

11:33 So yeah, I'd love for you— I'll link you the Pollen Robotics project below.

11:38 Maybe if you have a Reachy Mini at home yourself,

11:42 go check out the code and run it with Gemini Live.

11:46 I'd really love to see what you're doing here.

11:49 I think I've seen folks build, like, D&D masters, Dungeons and Dragons, like,

11:59 role playing game hosts with this, and I think there's some really,

12:04 really cool things and use cases that you can build with this.

12:08 And it's just fun talking to this little guy,

12:13 even when they make all the lobster jokes.

12:16 So thanks so much for tuning in.

12:18 I hope this was helpful for you to get a feeling for what

12:22 Gemini Live can do in combination with the function calling and the actuator.

12:26 So yeah, thanks for tuning in, and see you next time.

12:30 [MUSIC PLAYING]