Azure Blob Storage for AI

0:00 (bright electronic music)- Hi, everyone, and thank you for tuning in.

0:17 My name is Saurabh Sensharma and I'm a principal product

0:21 manager on the Azure Blob Storage team focused on AI workloads.

0:25 Today I'm excited to share the amazing scale and performance capabilities

0:30 of Azure Blob that enable AI scenarios across AI training and AI inferencing.

0:37 We will demonstrate how Azure Blob Storage

0:40 can deliver high storage throughput at exabyte scales,

0:44 which is important to keep compute resources busy during training,

0:48 and how Azure Blob Storage enables customers to not

0:53 just store and manage hundreds of petabytes of enterprise data,

0:57 but also achieve low latency retrieval with security and freshness,

1:02 that is critical for agentic inference scenarios.

1:05 Alright, let's start with AI training and fine-tuning.

1:10 The AI training and fine-tuning pipeline follows roughly the same pattern,

1:15 whether it's Microsoft AI, OpenAI, or your own training workloads.

1:20 Only, the scalability requirements might differ slightly.

1:25 As you can see on the slide from left to right,

1:28 it starts with bringing raw data into Azure Blob Storage.

1:33 Now this could be terabytes or even hundreds of petabytes of data.

1:38 Once this data is ingested in Azure, the data is validated,

1:42 cleansed, and annotated,

1:44 typically using open-source Spark or Databricks engines.

1:48 Once the training data and base model weights are ready,

1:51 the model enters the training phase,

1:54 which is highly compute and storage intensive.

1:57 It requires terabits per second read bandwidth to load the training

2:02 data and terabits of write bandwidth to checkpoint the training process.

2:07 Checkpoints capture the full state of the GPU cluster

2:11 and store it in durable storage to enable restarts after failures.

2:16 They must also be retained for traceability and model lineage.

2:20 At scale, checkpoints can generate exabytes of data.

2:25 Once the training process is complete, the final step is to deploy the model

2:30 to the environment where AI inferencing takes place.

2:35 To adequately support AI training,

2:37 the storage system must be able to store exabytes of data and provide tens

2:42 of terabytes of throughput to maximize utilization

2:45 across large-scale distributed CPU and GPU clusters.

2:50 Now everything that you're going to see

2:52 in this video today is publicly available,

2:55 but it was established over the years through iterative

2:58 development and practical lessons learned from operating at scale.

3:03 Let's look at one of the features that enable

3:05 customers to meet new scale and performance requirements: scaled accounts.

3:10 Scaled accounts are applicable to high-intensity

3:14 workloads like training and inferencing.

3:17 Scaled accounts are what kicks in underneath the covers whenever we

3:21 detect that a regular storage account needs additional performance and scale.

3:26 We scale the account across an entire Azure region, not just a single cluster.

3:32 So capacity is bound only by the scale of the region itself.

3:37 Scaled accounts is a capability you automatically get whenever needed.

3:41 There are no account changes, no API changes, and no incremental API charges.

3:48 The other helpful capability you can use for your AI workloads is BlobFuse.

3:53 When your applications and tooling are built to use POSIX-based file systems.

3:58 With BlobFuse, you can mount a blob container into a VM's file system,

4:03 enabling training code and other applications to access Blob Storage

4:08 as if it were a local file system with intelligence streaming.

4:13 Combined with the VMs and local NVME drives,

4:16 it can be used to preload data or push

4:19 out checkpoints to avoid blocking GPUs longer than necessary.

4:23 BlobFuse is open-sourced and is designed to be easy to use with helpful

4:29 default configurations that you can of course tweak for your unique scenarios.

4:34 So what happens when you put a scaled account to the test?

4:38 Let me share with you a brief demo using,

4:41 a Standard Blob Storage account in a US region.

4:45 We stood up a compute cluster with hundred thousand vCPUs and use

4:51 BlobFuse to write 1.5 pebibytes of data into a scaled account,

4:57 simulating checkpoint writes,

4:58 and then configured another separate 100K core compute cluster to read

5:05 the 1.5 pebibytes of dataset out from the scaled account,

5:09 simulating large data sets or checkpoint loads for training.

5:13 We use separate compute clusters to avoid any caching effects.

5:17 So what you see here is the first compute cluster

5:21 writing data into the account at 22 terabits per second,

5:25 or almost 3 terabytes a second.

5:28 And for the read performance,

5:29 you see just north of 51 terabits per second of read bandwidth,

5:34 or just over 6 terabytes a second.

5:37 This demo illustrates the scale and performance of Azure Blob Storage,

5:42 and that is used by our highest scale AI customers and partners today.

5:48 Now that we've covered training, let's talk about inference.

5:52 When you move from training trillion parameter models

5:55 to running them in production, the focus shifts.

5:59 Instead of exabytes of raw or checkpoint data,

6:02 you're dealing with terabytes of enterprise domain data,

6:06 but you still need hundreds of gigabits per second throughput

6:10 and sub-10 millisecond latency to keep your AI agents responsive.

6:15 And Azure Blob Storage delivers exactly that.

6:19 Let's look at the storage centric view of the pipeline

6:22 to enable agentic inference and reasoning over enterprise data.

6:26 We are ingesting multimodal, unstructured enterprise data,

6:31 cleaning it, chunking it, and then generating embeddings,

6:35 and then indexing them into a vector

6:38 database for future retrieval by agentic applications.

6:43 Blob Storage integrates with Azure AI

6:46 Search and other third-party vector databases, so those indices stay fresh.

6:51 And when a user query hits an agentic app,

6:55 the agent triggers a vector similarity

6:57 search and retrieves the write context instantly.

7:01 Single-digit millisecond latency matters here

7:04 because every millisecond saved over several agentic loops mean faster agentic

7:11 reasoning and better end-to-end user experience.

7:15 Now, assuming enterprise data is not already on Blob Storage,

7:18 you would need to bulk move heterogeneous

7:20 enterprise data to Azure to activate this pipeline.

7:25 Now, Azure Storage offers both offline solutions

7:28 like Azure Data Box and online solutions

7:31 like Azure Storage Mover to achieve petabyte

7:34 scale fast data ingestion into Azure Blob Storage.

7:39 Azure Storage Mover is fully managed, no agents, no scripts.

7:43 Just high-speed parallel transfers that move billions

7:47 of objects across clouds from on-premises environments.

7:51 We are talking petabyte scale migrations

7:54 with built-in encryption and incremental syncs,

7:57 so your data is ready for AI workloads without any downtime.

8:02 Once your data is in Azure,

8:04 Premium Blob Storage takes the performance to the next level.

8:09 Unlike Standard Storage, Premium Storage is SSD-backed object storage,

8:14 designed for highly interactive workloads such as AI inference.

8:19 You get up to 40x faster performance with up

8:23 to 65% savings on transactions compared to Standard Blob Storage,

8:27 and more importantly,

8:29 consistent low latency IO in single-digit millisecond ranges.

8:35 That means models load quickly, enterprises data retrieval is snappy,

8:41 and GPUs stay busy instead of waiting.

8:45 It's not just storage speeds and feeds

8:47 that matter for inferencing with enterprise data,

8:50 it is integration with AI services and tools that just works at scale.

8:55 Blob Storage serves as the petabyte scale

8:58 data and knowledge source for Microsoft Foundry IQ,

9:01 Azure AI Search, Azure OpenAI, and Azure Machine Learning.

9:06 Now, we are also aware that a lot of you use third-party

9:10 open-source tools and frameworks like PyTorch

9:13 and LangChain to build agentic application.

9:17 Blob Storage does not only integrate with these tools for developer ease,

9:21 but we have been optimizing the most

9:23 adopted integration for high throughput pipelines

9:27 so your apps can scale to cover

9:30 your enterprise data context without any bottlenecks.

9:33 Now let's dive into a few of these that are valuable

9:36 to a growing community of enterprise AI developers building with Python.

9:42 Data professionals building with Python face a constant challenge,

9:47 bridging agent application code and cloud storage, no matter the scale.

9:52 Python's fsspec standard is the universal file adapter and ADLFS,

9:57 or Azure Data Lake file system, is its specialized high-performance gateway

10:03 to Azure's Blob and Data Lake Storage.

10:07 With our latest enhancements to ADLFS, you get parallel uploads and faster,

10:13 more resilient block-level writes, reducing timeouts for large files.

10:18 By making Azure Storage faster and more reliable in ADLFS,

10:24 every tool from PyTorch to Pandas to Ray

10:27 that uses fsspec gets a huge boost on Azure.

10:33 Best part, there is no custom integration required.

10:37 Switching from local files or other cloud files to Azure

10:41 is often just as simple as specifying a file path,

10:45 as you can see in the example on this slide.

10:48 Taking this a step further to agent orchestration frameworks,

10:52 I am really excited to share our official

10:55 support for the Azure Blob Storage loader for LangChain.

10:59 LangChain can now stream billions of documents from Blob Storage with 10 times

11:05 faster ingestion and 90 times lower peak

11:09 memory usage compared to prior community versions.

11:12 These improvements mean your AI pipelines

11:15 can handle massive data volumes efficiently.

11:20 This is critical for enterprise-scale RAG, or Retrieval Augmented Generation,

11:25 and agent inference where speed and efficiency over

11:30 terabyte to petabyte-scale knowledge sources directly impact user experience.

11:36 Alright, now it's time to bring everything that we spoke about together.

11:41 As we saw for training, Blob Storage delivers tens of terabits per second

11:46 throughput at exabyte scales using scaled accounts and BlobFuse.

11:51 So, checkpoints complete in seconds, not minutes.

11:55 For AI inference, Premium Blob ensures

11:58 single-digit millisecond latency for enterprise data retrieval.

12:02 And across the board,

12:03 our optimizations for open-source frameworks and tools make data prep

12:08 and ingestion 5 to 10 times faster and more resource-efficient.

12:13 Azure Blob Storage isn't just storage,

12:16 it is the performance backbone for AI at scale.

12:21 As we come to the close of this video,

12:24 here are a few helpful resources to supercharge

12:27 your enterprise AI journey with Azure Blob Storage.

12:30 (bright music) We are eager to hear your feedback on how we can

12:34 help you scale what you do with Blob Storage for your AI training,

12:39 fine-tuning, and inference workloads.

12:42 Thank you for watching.