Why Every Company is Building Their Own AI Chips
Tiff In Tech
0:00 For the last few decades,
0:01 we've lived in an era of generalpurpose [music] computing.
0:04 Whether you're editing a video, writing an email, or training a simple model,
0:08 you used the [music] same basic chip,
0:10 which were designed to be good enough for everything.
0:13 But recently, [music] we've hit a wall.
0:14 As our AI models have scaled,
0:17 they're starting to run into what is called hard limits of physics.
0:20 We've seen servers overheating, [music] energy grid straining,
0:23 and I mean chips that are physically incapable
0:25 of moving data [music] fast enough to keep up.
0:28 Now, in the past, we just waited for the next faster chip to come out.
0:32 But today, the world's biggest tech [music]
0:34 companies have realized that faster isn't enough anymore.
0:38 Now, according to the latest data from 2026,
0:40 [music] the industry is seeing a massive pivot.
0:43 We are entering what for this video we
0:45 are going to call the [music] silicon renaissance.
0:47 Basically, it's a period where companies like Apple, Google,
0:50 Amazon [music] are abandoning off-the-shelf parts
0:53 to build their own custom chips, their own brains, if you [music] will.
0:57 And this shift is about to change,
0:59 I mean everything from your phone's battery life
1:01 to the very cost of the internet itself.
1:04 [music] But first, I want to talk to you about Meter.
1:06 Most enterprise networking is a massive headache
1:09 of fragmented hardware and weird ISP contracts.
1:13 If you're building in a physical space,
1:14 whether it's a startup office or a huge warehouse,
1:17 now trying to scale your Wi-Fi and wired infra usually
1:21 involves a ton of manual configuration that frankly doesn't hold up.
1:27 And this is where Meter comes in.
1:29 They've built a full stack approach to networking.
1:31 Meaning they design their own hardware from APs, switches,
1:36 I mean the entire works, including the software that runs it.
1:41 Now what I find interesting is their model.
1:43 Essentially, you get the hardware, the installation,
1:46 and the maintenance for one predictable monthly rate.
1:48 Now, for the engineers watching, you get a centralized dashboard with actual
1:53 visibility into the network health without
1:56 having to deal with a massive capex project or a dozen different vendors.
2:00 I mean, we've all been there.
2:01 We've all felt that pain.
2:03 Now, if you want to see how they're modernizing network deployments,
2:06 head over to meter.com/tifftech.
2:09 It's a much cleaner way to handle infra.
2:11 Now, back to the video.
2:12 Now, to understand why this is happening,
2:14 think of a standard GPU like a professional athlete.
2:17 They are incredibly versatile and strong.
2:20 But if you ask that athlete to spend 10
2:23 hours a day doing nothing but one specific thing,
2:27 a highly repetitive movement like a specific type of AI math,
2:31 they're going to waste a massive amount
2:33 of energy on muscles they aren't even using.
2:36 This is what is called the general purpose tax.
2:38 Basically, a chip is built for everyone.
2:40 [music] It's optimized for no one.
2:42 For years, we just paid that tax.
2:44 But as AI models become the most power- hungry software in well human history,
2:50 that tax became too expensive to ignore.
2:52 Recent benchmarks show that moving from a general
2:55 purpose GPU to a specialized [music] ASIC can lead to up to 90% reduction
3:00 in power consumption for specific [music] training tasks.
3:03 So the solution isn't a bigger athlete,
3:05 it's building a digital specialist [music] designed
3:07 from the atom up for one specific job.
3:11 This is where the first major player
3:12 in the renaissance comes in, which [music] is Google.
3:15 Now, while everyone else was fighting over Nvidia GPUs,
3:18 Google was quietly working in the background
3:20 perfecting the TPU or tensor processing unit.
3:23 First though, to understand why the TPU is so different,
3:26 we have to look at the logic tax.
3:27 [music] So, think of a GPU like a high-end professional kitchen.
3:31 Earlier, it was an athlete.
3:32 Now, we're talking kitchen.
3:33 I mean, it's flexible.
3:34 You can bake a cake or sear a steak at the same
3:37 time using SINT [music] architecture which
3:39 stands for single instruction multiple threads.
3:42 But the thing is all of that flexibility requires extra space
3:45 and energy just to manage the chaos of all those different orders.
3:50 Now the TPU is a specialized assembly line.
3:53 I feel like always in tech we are
3:54 using these kitchen examples but it really hits.
3:58 I don't know.
3:58 My first one was API.
3:59 What about you?
4:00 Okay, but in all seriousness a specialized assembly.
4:02 Basically, it uses a design called a systolic array.
4:05 In a normal chip, the processor has to constantly reach out to memory,
4:09 grab data, and put it back, which is a massive energy drain.
4:13 But in a systolic array, data flows through the chip like a wave.
4:16 It passes from one processing element to another
4:20 without ever stopping to talk to the memory bus.
4:23 Once the data starts moving, it just keeps flowing through thousands
4:26 of gates until the final calculation is done.
4:29 So by removing the need to constantly check back with memory,
4:32 the TPU eliminates essentially that logic tax.
4:35 [music] Now Google didn't just stop at their chip.
4:38 Their latest TPU v7 code named
4:41 Ironwood represents a fundamental shift in infrastructure.
4:44 [music] In a traditional data center,
4:46 you have thousands of individual servers connected by copper cables.
4:50 But copper [music] has a problem, a big problem.
4:53 It resists electricity.
4:54 I mean, it gets hot, it loses data, and it creates what's called a latency wall.
4:58 [music] To bypass this, Google built optical circuit switching.
5:02 Instead of sending basically electrical signals through wires,
5:05 they are using tiny [music] moving mirrors
5:07 to route beams of light between chips.
5:10 By using light, they can link over
5:12 9,000 ironwood chips into a single seamless fabric.
5:16 In a normal setup, the more chips you add, the slower the communication gets.
5:20 But with OCS, the network is almost well flat.
5:24 Data moves at the speed of light with near zero power loss.
5:28 So this isn't just a chip.
5:29 It's a planetary scale brain essentially.
5:32 [music] Okay, but let's take a step back.
5:34 Why does this matter for the economy for you and I?
5:37 Well, for years, Nvidia held a de
5:39 facto monopoly because of their software stack, CUDA.
5:42 It was the only way to talk to [music] AI hardware.
5:45 But by building their own silicon, companies like Google and Amazon are creating
5:48 their own software stacks [music] like XLA.
5:51 They're effectively cutting out the middleman.
5:54 By [music] 2027, it is predicted that over 40% 40%
5:58 of all AI workloads will now run on custom non-Invidia hardware.
6:03 We are basically seeing a decentralization of power.
6:06 The silicon renaissance is essentially the breaking
6:09 of the monopolies and that is going
6:11 to lead to a massive explosion in what we can actually afford to build.
6:16 But this isn't just a story about giant data centers.
6:19 [music] It's also about the devices you use on a daily [music] basis.
6:25 Apple iPhones.
6:26 I mean, Apple is the architect that proved custom silicon [music] could change
6:29 our daily lives by moving to their own M series and A series chips.
6:34 They essentially took control of their own destiny.
6:36 Now what they did by using a 2nanmter process at TSMC,
6:40 [music] Apple can now pack billions of transistors into a tiny space.
6:44 But more importantly, they've [music] carved out a dedicated section
6:48 of that chip called the neural engine.
6:51 Because Apple builds a software [music] and the silicon,
6:53 they can optimize for privacy and battery life in a way that no one else can.
6:57 When your phone recognizes a [music] face
6:59 or translates speech locally without touching the cloud,
7:02 that is a silicon renaissance in action.
7:05 It's silicon [music] sovereignty,
7:07 the power to build features that were once considered physically impossible.
7:11 And then there's Amazon.
7:13 So while Google is chasing the physics of light, literally,
7:17 Amazon is chasing the [music] physics of the balance sheet.
7:20 So for years, the biggest barrier to entry
7:22 in AI hasn't been [music] talent or ideas.
7:25 It's been the NVIDIA tax.
7:27 Basically, if you want to train a world-class model,
7:29 you have to wait months for GPU [music] allocations and pay a massive premium.
7:33 Now, Amazon's Tranium [music] 3 chips
7:35 are designed to essentially break that monopoly.
7:37 They are built for one singular goal, unit economics.
7:41 By optimizing the silicon [music] specifically for the AWS neuron compiler,
7:46 Amazon has stripped away every unnecessary feature,
7:49 which allows [music] them to offer compute at a fraction of the cost.
7:54 They are essentially commoditizing AI,
7:56 turning it from a luxury [music] resource into a public utility.
8:00 All right, now let's bring this back down to [music]
8:02 earth to how does this impact you and I?
8:04 I know some of you are thinking exactly that, Tiff.
8:06 [music] This is really cool tech, but I'm a front-end developer.
8:09 I'm a data scientist.
8:11 Why does this matter to me if Amazon builds their own chips?
8:14 It matters because we are seeing the end of the abstraction [music] era.
8:18 I mean for 15 plus years our job was
8:21 to be as far away from the hardware as [music] possible.
8:24 We used highlevel frameworks so we never had to think about the CPU.
8:28 But in this next era this silicon
8:30 renaissance that [music] distance is becoming a disadvantage.
8:34 If you want to stay ahead you need to transition [music]
8:36 from being a user of these systems to an architect of them.
8:40 Now how do we do that?
8:41 Step one embrace hardware awareness.
8:43 You don't need to be an electrical engineer,
8:45 but [music] you do need to understand compute in memory and vectorization.
8:48 When you're choosing a library,
8:49 ask is [music] this optimized for the silicon it's running on.
8:52 The developers who can save a company, say 30% on their cloud bill just
8:57 by choosing the right hardware [music] aware library,
9:01 they are the ones who well I mean,
9:03 knock on wood or rock in this case, they are the ones who will be unlayoffable.
9:09 Step two, pivot [music] to the edge.
9:11 The renaissance is moving AI onto the edge.
9:14 Your phone, your watch, even your smart glasses.
9:16 [music] If you can build models that run locally
9:19 on an Apple neural engine or a Google Tensor chip,
9:22 you are solving the [music] biggest problem in tech, privacy and cost.
9:26 Start looking at frameworks like CoreML or TensorFlow [music] light.
9:30 And step three, master the orchestration.
9:33 We've spoke about this before.
9:34 As chips get more specialized, software becomes more fragmented.
9:37 [music] We aren't just writing one big app anymore.
9:39 We are writing what is called orchestrations that send some tasks to a GPU,
9:44 some to a TPU, [music] and some to a local your local MPU.
9:49 Your new job description [music] isn't coder, it's system conductor.
9:53 So the silicon renaissance isn't just about faster chips.
9:56 It's about tech independence.
9:57 [music] It's about building a future where AI is faster,
10:01 more private, and significantly more efficient.
10:04 [music] We aren't just using computers anymore.
10:06 We are using systems that were built from the [music]
10:09 atom up specifically to solve the problems of tomorrow.
10:13 I don't know about [music] you,
10:14 but the more I dive into the hardware aspect of AI and how we're building it,
10:19 which is really shaping our future, [music] I just keep on getting blown away
10:23 and just keeping on needing to peel back the layers.
10:26 I'm curious to get your thoughts on this.
10:28 What other questions do you have around this?
10:30 We're [music] doing a really big deep dive in tech explainers on these topics
10:34 and I want to hear what you want to learn because [music] just like yourself,
10:38 I'm very curious about this.
10:40 I mean, I spend hours and hours
10:41 with my team researching these subjects and [music] topics,
10:44 condensing them down into, you know, 12, 13 minute videos.
10:49 Then [music] there's so much more to it.
10:50 We always try and highlight and really bring to the forefront what you need
10:54 to know and hopefully in a way that is easy to digest and understand it.
10:57 All right, I will see you in the next video.
10:59 Leave in your comments what other topics,
11:01 tech explainers you want me to dive into.