NVIDIA GTC Studio with Insights from Schneider Electric

0:10 1 Hi everyone, welcome to the NVIDIA GTC studio.

0:13 2 My name is Tiffany Janzen and today I'm 3 joined with Pankaj Sharma,

0:16 who is the Executive 4 VP of Software and Services at Schneider Electric.

0:21 5 Pankaj, welcome.

0:22 6 Thank you, Tiffany.

0:23 7 How's your GTC been going so far?

0:26 8 Excellent.

0:26 9 It's been the last two days, super busy,

0:28 10 a lot of information, but that's really good.

0:31 11 That's really good to hear.

0:32 12 When I saw that we'd be having a 13 conversation today,

0:35 I was really looking forward to 14 it

0:37 because I think what Schneider Electric is 15 working

0:39 on and some of the challenges that 16 you are

0:41 solving are some of the biggest 17 challenges we are,

0:43 frankly, facing right now when 18 it comes to AI infrastructure and scaling.

0:47 19 And on that note, we are seeing incredible 20 momentum around AI factories.

0:52 21 We're seeing a ton of them starting to 22 pop up.

0:54 23 But in order for them to be successful, 24 they need a ton of power.

0:59 25 Can you share with me a little bit 26

1:01 about some of the challenges and also opportunities 27 that brings?

1:04 28 It brings, first of all, a lot of 29 challenges and opportunities.

1:09 30 So maybe just take a step back.

1:10 31 If you heard the keynote when Jansen was 32 sharing the five-layered cake.

1:14 33 Yes.

1:15 34 And we play at many layers, obviously,

1:17 but 35 the primary role we play is at the 36 bottom,

1:20 where the energy infrastructure has to come 37 in.

1:23 38 Now, the challenge at hand is that the 39

1:26 utilization and applications of AI are starting to 40 increase dramatically.

1:30 41 I mean, we see it in our own 42 lives, right,

1:32 as consumers or businesses and how 43 it

1:35 is making everything more efficient and so 44 on.

1:37 45 But to build all of that, you need 46 the AI factories, for which,

1:41 of course, you 47 need physical infrastructure

1:43 and also you need available 48 energy.

1:46 49 Now, when we think about the overall

1:47 planet 50 and the amount of energy available,

1:49 and we 51 keep in mind that at the end of 52 the day,

1:51 it has to be sustainable too, 53 right?

1:52 54 So available energy and green energy availability is 55 overall a challenge.

1:57 56 And that's why the big push right now,

2:00 57 and when Jansen also shares the bottom layer,

2:02 58 it's so important for us to be able 59 to solve for that.

2:04 60 So yeah, that's a huge issue at hand.

2:06 61 So it's related to great infrastructures,

2:09 it's related 62 to clean generation of energy,

2:11 it's related to 63 availability of physical infrastructure, which is efficient,

2:14 which 64 is where companies like us come in.

2:17 65 So that's the overall challenge.

2:18 66 But like you said, it's also a huge 67 opportunity.

2:21 68 It's a huge opportunity because while

2:24 you think 69 about availability of energy,

2:26 you also have to 70 think about how utilizing AI,

2:30 you can make 71 the availability of energy better.

2:33 72 So at Schneider, for example,

2:35 what we talk 73 about is AI for energy and energy for 74 AI, right?

2:39 75 Everybody talks about energy for AI, oh, energy, 76 so much of energy needed.

2:42 77 But AI can actually enable by simplifying through 78 digital and so on.

2:47 79 Isn't that so funny how it works where 80 it's, you know,

2:50 we need so much energy 81 for AI and AI is going to be

2:54 82 the enabler to solve that challenge at the 83 end of the day.

2:57 84 What are some examples of AI being able 85 to do

3:02 that, that you have seen being 86 able to solve some of these grid challenges?

3:04 87 Because the grid really isn't set up for 88

3:07 this level of scaling when it comes to 89 AI.

3:10 90 Multiple examples I can give you.

3:12 91 So one of the examples I give you 92 is

3:15 when you think about in a large 93 data center, right?

3:19 94 There's so much of heat getting generated because 95 of compute.

3:22 96 So you need to take out the heat, 97 which means you need cooling and you

3:25 need 98 cooling systems and HVACs and so on.

3:27 99 Just by utilizing AI algorithms

3:31 on how historically 100 the air conditioning system, HVAC systems are working,

3:36 101 we are able to save almost 10% 102 energy being utilized, right?

3:40 103 So this is when you turn them on, 104 turn them off, et cetera.

3:42 105 So this is where the genetic comes into 106 play, right?

3:44 107 I mean, it automates the whole performance.

3:47 108 Other examples of where AI can help is, 109 for example,

3:53 overall in a large data center 110 environment,

3:58 on the physical layer, you have compute.

4:00 111 So when you apply AI to think about 112

4:03 when is compute needed and when it is 113 not needed,

4:06 let's say in a large training 114 factory,

4:08 that can then help utilization of energy

4:10 115 and bringing the need for energy down.

4:12 116 So AI for energy actually is a very 117 important tool.

4:16 118 And I try to say that whenever somebody 119 asks me this question,

4:19 because it's very easy 120 to just get

4:21 overly perplexed by the fact 121 that, oh, there's so much of energy needed.

4:25 122 But unless we do AI, we can't even 123 solve for that.

4:27 124 Of course, we need to generate, but we 125 can't solve for what is

4:30 available and how 126 much is getting utilized.

4:32 127 Absolutely.

4:33 128 And I think that's on a personal level, 129 I think, to be honest with you,

4:36 that's 130 where I am currently at in my research 131 journey, if you will,

4:40 when it comes to 132 AI and the energy required to keep

4:44 on 133 scaling is I'm in that stuck phase where 134 it's,

4:48 well, how are we going to solve 135 this?

4:50 136 Where is it going to go?

4:51 137 So hearing from you of those use

4:54 cases 138 and examples with AI is really exciting,

4:56 especially 139 that stat 10% for cooling.

4:59 140 Yeah.

5:00 141 Yeah.

5:00 142 I mean, there are other examples I'll give 143 you.

5:01 144 Like when you think about software, we have 145 one of our businesses,

5:04 which is called the 146 digital grid business,

5:05 which is how do you 147 optimize the grid?

5:08 148 Just by utilizing software on how the grid 149 is managed,

5:12 you're able to bring down the 150 technical losses by a few percentage points.

5:17 151 Now, that means that much available energy,

5:20 which 152 can be utilized for all the other things 153 we're trying to do.

5:23 154 Now, if you put that in AI, AI 155 helps again, right?

5:26 156 So it's almost circular if you think about 157 it.

5:28 158 You know, have you at Schneider been

5:31 working 159 on this with AI for, I'm curious

5:34 because 160 right now I feel like it's

5:36 a topic 161 of conversation and I'm sure you've,

5:38 you know, 162 been bombarded with so many questions around

5:41 this, 163 but have you been working on this way 164 before?

5:44 165 I'm sure that people were even starting to 166 really think about it,

5:48 like the general public.

5:49 167 Yeah.

5:49 168 And, and, and look, the, the need for 169 energy is something

5:54 which has been a challenge 170 for many years.

5:56 171 It's not something new, right?

5:58 172 The only thing is because now we have 173 stronger

6:02 LLMs and we have much stronger compute 174 capability in NVIDIA,

6:07 the, the application of AI 175 has become rapidly faster.

6:12 176 So hence the need of energy has gone 177 up dramatically,

6:14 but the need of energy for 178 data centers has always been a challenge.

6:19 179 For a company like us, we look at 180 it from two points of view.

6:23 181 First, because we create the physical infrastructure.

6:26 182 So our goal always is to make it

6:28 183 more and more efficient and efficiency then helps.

6:31 184 Efficiency is through design of the product,

6:34 but 185 efficiency is also through digital,

6:37 which is how 186 you manage those parts,

6:39 the entire digital part, 187 the software part of it.

6:42 188 So we've always, as a company, been,

6:44 been 189 working on bringing out those kinds

6:47 of technologies 190 so that the input energy,

6:49 which is going 191 into a data center

6:51 and how much is 192 actually getting utilized by compute,

6:53 you start to 193 reduce that gap dramatically.

6:55 194 So that's been an effort of Schneider Electric 195 for a very long time.

6:59 196 We've also been also focused on thinking about

7:02 197 how you utilize that in a sustainable fashion.

7:05 198 It's just that today with the demand going 199 up so much,

7:08 everything needs to be accelerated, 200 right?

7:10 201 So it's not just accelerated compute.

7:12 202 It's accelerated generation of energy,

7:14 the accelerated infrastructure, 203 which is needed.

7:16 204 It's accelerated digital, which is all softwares,

7:19 which 205 are going to make it easier.

7:20 206 So all of that has to be accelerated.

7:22 207 It really can't be done in silos.

7:24 208 It has to be a solution altogether almost.

7:27 209 Exactly.

7:28 210 I mean, that's such a good point you're 211 making because when

7:30 you also think about the 212 entire build out of AI factory, each system,

7:36 213 each component of the system is

7:39 important because 214 you get the best optimization

7:42 of both the 215 physical layer and the software when it works 216 as a system.

7:47 217 So the system play is absolutely critical.

7:49 218 So great point.

7:50 219 You know, I know Schneider does both, as 220 you mentioned,

7:53 the physical aspect, like the physical 221 side of things,

7:56 and then also the software.

7:57 222 Are there any scenarios where you are solving 223 a problem,

8:01 working with a company who already 224 has one or the other?

8:04 225 And even though both are so important,

8:06 you 226 have to kind of really adjust based on 227 that.

8:09 228 So maybe it's an existing AI factory or 229

8:13 they have pretty archaic software that they're dealing 230 with.

8:16 231 Are there any cases of that?

8:18 232 So your question really is on, let's say, 233 Brownfield.

8:21 234 Yeah.

8:21 235 You will, right?

8:22 236 And there are not many existing AI factories.

8:26 237 Yeah, that's true.

8:26 238 That's true.

8:27 239 Just kind of getting started.

8:28 240 But we've been building data centers,

8:30 the world's 241 been building data centers for many decades now.

8:33 242 And typical utilization levels in data

8:37 centers are 243 extremely high right now,

8:39 as you can imagine, 244 because the demand is so high.

8:41 245 But some of the builds which happened

8:43 in 246 the past have reached a limit of performance,

8:48 247 simply because they were never built for this 248 level of compute.

8:52 249 So we have those kinds of examples, and 250 to your point,

8:55 both on the physical layer 251 and the software which is utilized there,

8:57 there 252 are times when you can have

9:00 a Brownfield 253 environment and you can apply software.

9:02 254 And when we say software, we talk about 255 energy intelligence.

9:05 256 And I'll maybe take a second here to 257 explain to you what that means.

9:08 258 Energy intelligence is where,

9:11 when you have performance 259 data of data centers,

9:14 as one example, and 260 any other application for that matter,

9:18 for the 261 longest period of time, for decades,

9:20 you utilize 262 that data and then put AI

9:23 on top 263 of the data with the help of software, 264 come up with insights,

9:28 which you can either 265 be AI augmented or AI native,

9:33 more so 266 AI native now, when it was heard that 267 as a conversation,

9:37 so that those insights can 268 help

9:38 improve the performance of the physical layer.

9:42 269 Now in a Brownfield environment,

9:45 sometimes with energy 270 intelligence, this can be done.

9:48 271 The challenge always is, if it is a 272 much older environment,

9:53 the physical layer is not 273 digital.

9:57 274 So either you attach something like an appliance

10:00 275 on the physical layer to make it digital, 276 to be able to gather the data

10:03 and 277 perform what I said as energy intelligence,

10:06 or 278 you really need to swipe it out.

10:08 279 So it's a whole process.

10:09 280 Like as a company, we do that for 281 our customers.

10:11 282 First of all, we have ourselves been building 283

10:13 data centers for our customers for many decades 284 now, right?

10:16 285 So we work with them to think about 286

10:19 their physical layer and do a complete asset 287 analysis,

10:22 et cetera, try to bring in digital 288 software, et cetera, to simplify.

10:26 289 But if there are times where the physical

10:28 290 layer is just not digital at all,

10:31 then 291 all you have to do is you have 292 to swap it out.

10:33 293 Absolutely.

10:34 294 So there are those kinds of scenarios.

10:35 295 That's really interesting.

10:36 296 It's exciting to me, or interesting to me,

10:40 297 because every scenario is so different and every 298 challenge is...

10:43 299 I mean, there's a lot of similarities,

10:45 but 300 the way you have to think slightly about 301 each one is different.

10:49 302 Yeah.

10:49 303 I mean, it's different also because it

10:51 depends 304 on where the data center is located.

10:52 305 It depends on the environment around it.

10:56 306 It depends on availability of power, how much 307 more power is needed.

10:58 308 Like for example, if you're already maxed out 309

11:00 on power availability in a certain geographical location,

11:04 310 even if you can build more, there is 311 no point of that, right?

11:08 312 Yeah.

11:09 313 I didn't think about that.

11:10 314 That's a very good point.

11:11 315 Why is it important?

11:13 316 We kind of spoke about this earlier, but 317 why is it important to manage

11:17 all of 318 these systems together for AI factories?

11:19 319 Can you expand on that a bit more?

11:21 320 Yeah.

11:21 321 You know, when you think about an AI 322 factory,

11:23 I mean, think about everything that goes 323 into the physical layer.

11:26 324 So, first of all, you have energy coming 325 in.

11:28 326 And even in an AI factory, which,

11:31 by 327 the way, is some of the most, let's 328 say, well utilized environments,

11:36 also because there's a 329 lot of training

11:39 which is happening in those 330 AI factories, right?

11:40 331 So it's kind of similar type of loads, 332 right?

11:43 333 Even in those environments, there is standard capacity 334 of energy,

11:47 which means if you're getting a 335 hundred points

11:49 of energy coming in, you're utilizing 336 only 64,

11:52 as an example, for compute, which 337 is the whole point,

11:55 which means you still 338 have 40, which is standard somewhere.

11:57 339 Yeah.

11:58 340 So that's one example.

12:00 341 Then, so what do you do to make 342 it better?

12:03 343 So this is where software, AI,

12:04 energy intelligence 344 kind of things come into play.

12:06 345 The second point is, when you think about 346 the physical layer there,

12:10 so you have different 347 parts of physical layer.

12:13 348 So you have the power infrastructure,

12:14 you have 349 the cooling infrastructure,

12:15 you have cabling, you have 350 data, you have all of those things, right?

12:19 351 So each one of them individually has to 352 be the most efficient,

12:25 but collectively is where 353 the handovers happen.

12:28 354 So as an example, the amount of compute 355 getting generated leads

12:32 to the utilization of the 356 power infrastructure.

12:35 357 And when that's linked to the cooling infrastructure,

12:39 358 that handover is absolutely critical.

12:41 359 If that handover is not seamless, then you're 360 losing a lot of energy.

12:45 361 You talk about the standard capacity, that's where 362 you're losing.

12:48 363 So that's why the play of the system 364 is absolutely critical.

12:53 365 Even if each one of those individual

12:55 components 366 are the most efficient components, software defined components.

13:00 367 Yes.

13:00 368 But the system play is absolutely critical.

13:02 369 Absolutely.

13:02 370 That makes sense.

13:03 371 So much energy and heat is lost if 372 it's not,

13:06 for example, going back to liquid 373 cooling or properly system put in place.

13:12 374 Exactly.

13:13 375 So liquid cooling now, of course, I mean,

13:16 376 for us is a material part of what 377 we do, right?

13:19 378 About two years ago,

13:21 we had Motivair come 379 in and join our company and they're

13:23 the 380 number one leader in the world in that 381 space.

13:26 382 But it's important that not just, so back 383 to the system play,

13:31 it's not just important 384 to have the best

13:33 liquid cooling technology from 385 an offer standpoint,

13:35 whether the cooling distribution units 386 or the heat exchangers, et cetera.

13:38 387 It's also important how the flow of fluid 388 is managed

13:43 inside the data center and at 389 what temperatures it's getting managed.

13:47 390 What's the optimum temperature?

13:49 391 Because all of that eventually is leading

13:52 into 392 the amount of energy we are consuming.

13:54 393 And if you're wasting energy when you're consuming 394 it,

13:57 then that's something that makes it inefficient.

14:00 395 And again, we are talking about the fact 396 we

14:02 need more and more and more and 397 more energy, right?

14:04 398 Absolutely.

14:05 399 That's such a good point.

14:06 400 I never thought of it from that perspective,

14:08 401 especially around with liquid cooling.

14:10 402 I'm like, oh, liquid cooling, it's going to 403 solve so many problems,

14:12 but there's so much 404 more that goes into it,

14:14 even the way 405 it's flowing, the way it's set up.

14:17 406 If it's not done properly, then wasting energy.

14:20 407 There is, you know, throughout our conversation,

14:25 we've 408 addressed that there is a bottleneck when it 409 comes to AI scaling.

14:28 410 And one of those bottlenecks is around power.

14:31 411 Other than, for example, liquid cooling,

14:33 what are 412 some other ways to unlock this?

14:37 413 So liquid cooling is a part of the 414 physical layer, right?

14:42 415 So everything on the physical layer, whether it's 416 power infrastructure,

14:46 cooling infrastructure has to become more 417 and more efficient, right?

14:49 418 Which is what we spoke about already.

14:51 419 On top of that, one way to unlock

14:54 420 it now is with the energy intelligence, right?

14:58 421 So with energy intelligence, the ability to capture 422 data,

15:02 to be able to design better, you 423 know,

15:06 we talk a lot about SimReady in 424 this conference with NVIDIA,

15:09 there's a reason for 425 that, right?

15:10 426 Because everything which you put in, if it 427 is SimReady,

15:14 SimReady, SimReady, then it's able to 428 do the digital twin,

15:19 where even before building 429 the AI factory,

15:22 you have a design which 430 is the most optimum design, right?

15:25 431 So that's why this entire energy intelligence layer 432 comes into play.

15:29 433 So now what you can do is,

15:31 you 434 can build a digital twin in advance before 435 you make the AI factory,

15:35 but with the 436 intelligence of all the years of data which 437 you have,

15:39 in functioning of each one of 438 those offers,

15:42 those physical layer offers, you create 439 AI algorithms,

15:45 which tells you how to now 440 build so much better

15:48 than the way you 441 would have built in the past.

15:51 442 So to unlock, to your point, the utilization 443 of energy,

15:57 it's both on the efficiency of 444 the physical layer,

15:59 but it's also on the 445 energy intelligence which comes,

16:01 which is again AI 446 for energy.

16:03 447 I love that.

16:04 448 Energy intelligence.

16:06 449 Energy intelligence.

16:06 450 Yes.

16:06 451 That's why I learned a lot through this 452 conversation,

16:08 but that was, for me, that's the 453 take home that I...

16:11 454 That's what we do.

16:13 455 That's what we do.

16:13 456 I mean, as a company, if you think 457 about Schneider Electric, again,

16:16 we've been in the, 458 not just data center,

16:19 many other businesses, but 459 we are an energy management company.

16:21 460 But more and more energy needed

16:24 for AI 461 cannot happen without energy intelligence,

16:29 because of the 462 learning what we have collectively for so many

16:33 463 decades on how the performance of energy is 464 and utilization is,

16:36 you have to apply AI 465 on that, and that's energy intelligence.

16:39 466 And of course, there are other things that 467 you need more grids,

16:41 you need better grids, 468 and there are many other things we need 469 to do.

16:44 470 Pankaj, thank you so much for your time 471 today.

16:46 472 This was a really insightful conversation.

16:49 473 Thank you.

16:49 474 And thank you all for tuning in.

16:51 475 Make sure to go check out more sessions 476 on demand in the GTC catalog.