Connected managed and complete agentic apps with Microsoft Foundry | BRK113

0:00 Mike Hulme: All right.

0:00 Welcome everyone and welcome to our session, welcome to Ignite.

0:08 Thank you all for coming to join us here.

0:10 And it's actually great to be back in San Francisco.

0:12 This is a part of the country that I called home for most of my life up

0:17 until about two years ago when I made

0:18 my way over to the Seattle area and joined Microsoft.

0:25 Today has actually been a great day.

0:27 It's great to see so many people coming out.

0:29 This is just the second year that we went back in in-person events.

0:33 And I think it's really fantastic that we're here in San

0:35 Francisco where we see just so much great AI innovation happening.

0:39 And across this event, if you watched the keynote,

0:41 if you attended some of the sessions, you might have heard us talking a little

0:44 bit about frontier organizations or frontier transformation.

0:48 And at the core of that is a wave

0:50 of new applications that are just coming online.

0:52 We expect that to be somewhere in the neighborhood of 1.3 billion agents

0:55 that are going to move to production in just the next two years.

0:59 And as we've discussed,

1:00 these applications really distinguish themselves by their ability to reason,

1:05 by their ability to act on our behalf

1:07 with both intelligence and with agency and to serve

1:10 as an extension of our teams and really

1:12 an extension of ourselves in our everyday lives.

1:14 And the organizations that are using these agents and they're being successful

1:19 and they're driving deep impact for their business is what we call frontier.

1:22 And in this session we're actually going

1:24 to shift from the concept of what a frontier

1:27 application is into how you can actually

1:29 bring these AI applications and agents to production,

1:32 how you can ensure that every agent delivers on the expectations

1:36 and the results that you're looking for, and that every one

1:38 of those applications can also be governed and managed and operated

1:41 in the way that you need to to protect your business.

1:44 And so if we look at just some great examples here,

1:47 the impact of AI is already undeniable.

1:50 We're seeing direct, measurable,

1:52 and very tangible improvements in things like efficiency and productivity,

1:56 cost and time savings for key business services.

2:00 And just take, for example,

2:01 Standard Charter Bank who used the app modernization agents in GitHub

2:05 Copilot to transform their legacy stock grading systems in just four weeks.

2:10 The developers talked about an increase in productivity of around 40%.

2:15 And Java version upgrades were accelerated by around 70%.

2:18 And all of that included 95% test coverage by AI-generated tests.

2:23 Or look at SoftBank who's now using AI agents

2:26 to cover about 2/3 of their enterprise customer support issues.

2:30 This is reducing customer support costs by over $150 million.

2:34 This is a significant improvement in all of these business services.

2:38 But the challenges that we hear

2:40 from customers like yourselves are also quite common.

2:43 We hear that you're under massive pressure to deliver

2:46 agentic applications for your business as fast as possible.

2:49 That you're navigating a complex set of tools and services,

2:52 that new models are arriving every day.

2:55 We had some arrive just this week.

2:57 And building the services means that developers

2:59 are working across a very diverse,

3:02 very complex set of multiple, often unknown or foreign environments.

3:06 And that you're having to integrate all

3:08 of these disconnected systems just to be successful.

3:10 Those that move to production can often find

3:13 that the agent actually falls short of expectations.

3:16 Without the right grounding, the right data,

3:18 the right context for all of your users in your business,

3:21 you're finding that some of those services

3:23 just aren't giving you what you expected.

3:25 And with the rise of multi-agent systems,

3:27 the security threat service and the need for governance over

3:31 agent behavior for usage and for cost just continues to expand.

3:35 But we need to define a model for security

3:38 and governance that protects your business while also

3:40 giving you the freedom to innovate at the speed

3:43 and the impact that you need for your business.

3:45 And that's incredibly complex, but to make it even more complex is

3:49 that these applications themselves actually have unique needs.

3:52 They need access to the best models for various use cases.

3:56 They have to connect those agents and applications

3:59 to the right knowledge sources to contextualize AI responses.

4:03 You have to orchestrate across multiagent systems.

4:06 And you have to continuously evaluate if your solutions at every stage

4:10 of the AI lifecycle have the proper content safety and security measures.

4:15 All of that is making this entire process

4:17 of bringing these applications to life very, very complex.

4:21 And these are the exact challenges that we've taken on with Microsoft Foundry.

4:25 And as we've evolved this platform,

4:26 we've thought through the kinds of things that we need to do.

4:29 It's the most widely used AI platform in the world,

4:32 and it gives us great insights into what you need to be successful.

4:36 And so it's why we continue to expand our model catalog to bring

4:40 you the greatest diversity and range of models for every use case,

4:43 including those from OpenAI and from Anthropic,

4:45 and why we're making it easier than ever to connect

4:48 with the diverse data sources that bring deep context to your agents

4:53 that allow you to dynamically reason over your business systems

4:56 and your data and ensure that every agent is grounded, reliable, and relevant.

5:01 It's also why we're bringing together

5:03 this single control plane that we announced today.

5:05 All of the services you need to manage:

5:08 Identity and policy, observability and security.

5:10 And we've built this platform to be integrated so that you

5:14 have great access to a complete set of Azure services.

5:17 But we also want it to be modular so

5:19 that you have the flexibility to use any model, any tool, open frameworks.

5:23 This is a complete foundation for you to define the next stage of your AI

5:27 strategy and to really push forward

5:29 to deliver the results that you're looking for.

5:32 Now, today we took a very significant step forward.

5:36 We've augmented Microsoft Foundry with a curated

5:39 set of application platform capabilities.

5:41 And that's actually what we're going to show today through our demos.

5:44 We'll bring those to life and show you exactly how they work.

5:46 But it includes things like Logic Apps connectors that give

5:49 you access to 1,400 systems like SAP and ServiceNow, HubSpot, all as MCP tools.

5:57 And it gives you the power to quickly integrate into your core business systems

6:00 but also to act on the real-time

6:02 business data and events that those systems surface.

6:05 And through API management,

6:07 any API or function can now be securely exposed as an MCP tool.

6:11 But also through the integration of AI

6:14 gateway into the new Foundry Control Plane,

6:16 you have augmented governance and visibility for every

6:20 agent and MCP tool with built-in authentication, cost control, and logging.

6:24 Now, this is just the beginning.

6:26 These are great new integrations, and we're excited to showcase those for you.

6:29 But you should expect that this will be a continual strategy

6:32 and we'll just continue to bring out more ways that we can

6:35 bring services from our application platform together with our AI platform

6:39 as well to help you bring these agents and AI applications to life.

6:43 Now, before we dive into the new capabilities,

6:45 I actually want to highlight a specific customer.

6:49 KPMG is doing some great things with agents.

6:51 They're realizing great results.

6:53 And in just a minute, I'm going to bring up Robert Finlayson,

6:56 who's going to detail their journey working with us.

6:59 They began their generative AI journey back in 2023.

7:04 They built a new service called KymChat, which is really an assistant.

7:07 But they quickly advanced into building agents and have

7:11 become one of the early adopters of Microsoft Foundry.

7:13 They now have over 50 AI assistants and chatbots in production.

7:18 But they're actually working on 1,000 more agents

7:21 and chatbots that will be helping them across their system.

7:23 And Robert Finlayson is the Principal Product Manager at KPMG.

7:27 He's going to share where they started,

7:29 some of the things that they experienced along the way,

7:31 and then also what's next.

7:33 Before we bring up Robert, I'm going to run a short video.

7:35 And when we're done with that, Robert will join us on stage.

7:40 Music] Speaker 1: KPMG Workbench is our global AI platform

7:46 built with Microsoft to amplify human expertise through trusted AI agents,

7:51 accelerating research, innovation, and client delivery.

7:55 Early agents built with AutoGen on infrastructure

7:58 as a service worked well but lacked orchestration at scale.

8:02 Think of them as talented musicians without a conductor,

8:04 brilliant but not in harmony.

8:07 Enter Foundry Agent Service, integrated into Workbench,

8:10 bringing intelligent agents into KPMG's AI ecosystem.

8:15 When a KPMG app triggers an agent execution,

8:17 the request first passes through Azure API Management.

8:21 This secure facade enforces role-based access control

8:24 and logs every interaction for complete observability.

8:28 From there, the request flows into the agent service.

8:30 Inside Microsoft Foundry,

8:32 workloads are orchestrated across multiple Foundry projects.

8:35 Each project hosts a network of agents and connects seamlessly to KPMG tools,

8:40 enabling complex, agentic workflows.

8:43 The data layer is powered by Azure Cosmos DB.

8:46 Telemetry streams into App Insights, Azure Monitor, and Log Analytics.

8:50 Now let's see this architecture perform.

8:53 KPMG CaseCraft is an AI-powered storytelling assistant,

8:57 with agents working together to create,

8:59 find, and elevate KPMG's most compelling client delivery stories.

9:04 Picture the reviewer, research, and editor agents working in parallel to craft

9:08 a case study for a major European airport.

9:11 Multiagent workflows run seamlessly, recovering gracefully from errors,

9:15 while telemetry ensures compliance and performance.

9:18 Just as a conductor brings out the best in every section of an orchestra,

9:22 this integration unlocks capabilities that turn

9:25 individual agent skills into a seamless, high-performing ensemble.

9:29 Benefits of using Microsoft Foundry: Complex workflow logic.

9:33 [Music] Parallel execution.

9:35 [Music] Reliability.

9:37 [Music] Deep observability.

9:40 [Music] Our experience showed that orchestrating

9:44 agents isn't just about waving the baton.

9:47 It's about designing the entire score before the first note is played.

9:51 Here's what made the harmony possible.

9:54 Security context propagation.

9:55 [Music] Early architecture planning.

9:58 [Music] Built-in connectors.

10:01 [Music] Telemetry.

10:03 [Music] With Microsoft Foundry as the conductor and APIM as the stage manager,

10:10 KPMG Workbench transforms isolated agent

10:13 brilliance into a symphony of innovation,

10:16 delivering outcomes that resonate with clients and redefine how we work.

10:22 Music] All right, welcome to the stage, Robert Finlayson from KPMG.

10:26 Thank you, Robert.

10:27 Robert Finlayson: Thank you.

10:28 Thank you.

10:30 Applause] What an intro.

10:36 So how do we get started?

10:38 Our journey with Microsoft Foundry started over 18 months ago.

10:43 We wanted to be our client zero.

10:46 So we immediately deployed models,

10:47 put APIM in front of those models for cost recovery, observability.

10:53 Then we started to create agents.

10:55 We use AutoGen and Semantic Kernel.

11:00 In May of this year, when agent service went GA,

11:04 we transitioned our agentic workloads into agent service.

11:09 All net-new agents are now in agent service.

11:14 And we continue with our transition.

11:19 As we've scaled into our agentic fleet,

11:23 we've had to standardize how we work with Foundry.

11:27 So we leverage Foundry's projects, templates, the incredible tools from Foundry,

11:35 KPMG tools, all to build once and use many times.

11:42 We've learned a lot in our journey.

11:46 And I'll just highlight two things.

11:49 One thing is we were expecting Foundry to be a dev tool.

11:54 And for us, Foundry has extended to an agentic SELC platform.

12:00 So we have product managers,

12:01 model managers who are looking at the model catalogue.

12:05 We have data scientists doing experiments.

12:07 We have our trusted AI professionals looking at the responsible AI tools.

12:12 We have our testing and operations team look at the evaluations.

12:18 So Foundry has extended to our full SELC.

12:22 Our other learning has been about the project architecture in Foundry.

12:29 Don't use one single project for all your agents.

12:33 Use multiple projects.

12:35 It segments the agent workloads and it's helped us

12:39 put role space access control onto our agentic apps.

12:45 So two things we've learned along the way.

12:49 What we're building next with Foundry is

12:52 we're transitioning our agentic workloads, hosted agents.

12:58 We have a lot of container-based agents from AutoGen.

13:03 And you'll hear later in this session today

13:06 about how we now have hosted agents in Foundry.

13:10 So that's something which we'll be doing next.

13:13 The other super exciting thing is the Foundry Control Plane.

13:21 And that's going to provide us with deep

13:25 observability to put guardrails on our tools.

13:29 And a lot of our teams are excited to get working with that.

13:33 So that's what we'll be doing next with Foundry and agents.

13:40 Balan Subramanian: Thank you, Robert.

13:41 Robert Finlayson: Cheers.

13:43 Applause] Balan Subramanian: So that was really exciting to hear what KPMG has

13:47 done to build their own platform for their apps and agents.

13:51 But I'm even more excited about what we're doing in Foundry

13:54 to make it your turnkey platform for building your apps and your agents.

13:58 So to take us through that, I want to show you

14:01 a few demos of some of the capabilities that we've added into Foundry.

14:05 But first, going back to what Mike showed you earlier,

14:10 these are the areas that we focused

14:11 on, equipping your agents with tools and knowledge,

14:15 giving you better ways to build agents in Foundry

14:18 so you can do them wishfully or through code,

14:21 and also giving you this centralized control plane for managing

14:24 your agents and all the data sources and tools that they use.

14:30 So to take us through the capabilities,

14:32 we thought it would be nice to kind of use a real scenario.

14:35 So the scenario that we're going to use

14:37 today is about enhancing the customer return process.

14:39 This is a pretty common process in many retail and other industries.

14:43 And the things that we're going to show, firstly,

14:46 we're going to talk about how agents can replace much

14:49 of the human reasoning that's needed in these business processes.

14:53 Then we want to show you how these agents can access different business systems,

14:57 both for getting the data that they need to make these decisions,

15:01 but also to implement some of the actions

15:03 so that they can function autonomously.

15:06 Then we'll talk about hosted agents.

15:08 Like, we'll add more and more agents that can all now work together.

15:12 And then we'll show how these agents can enhance your existing

15:16 business processes and also help you build new agentic process implementations.

15:20 And finally, we'll show you how you can observe and govern

15:23 all aspects of such a solution through Foundry and the application platform.

15:30 So first, let's talk about enriching agents with real-time context.

15:34 So agents are only powerful and useful

15:36 as the tools that they have available to them.

15:39 So tools help agents deal with multiple modes,

15:42 whether it's text, images, video, or audio.

15:47 They help you connect to different business systems.

15:49 These could be your back-end business systems.

15:51 It could be partner systems.

15:53 It could be third-party SaaS that you're using.

15:55 And they help you integrate custom business logic.

15:59 So this could be existing business processes that you want your agent to use.

16:02 So all of that is possible through tools.

16:08 Now looking at what Foundry provides, Foundry provides a whole bunch of built-in

16:12 tools that help you deal with different modes,

16:15 whether that's speech, vision, or language.

16:18 As Mike announced, we are also announcing more than,

16:23 it's actually 1,500 plus now, not 1,400— it's 100 more that we found

16:28 this morning— that are now available in Foundry.

16:32 And these are enterprise-grade connectors that have been available

16:36 in our process automation platform for a really long time.

16:38 They are used by thousands, like hundreds and thousands of enterprises.

16:43 And what we have done is made it super easy to make

16:46 them available as MCP tools so that your agents can access them.

16:49 And I'll show you next how you can take advantage of that in Foundry.

16:53 And finally, we give you many different ways to build your own custom tools,

16:58 whether these are existing APIs that you want to make available as MCP tools,

17:02 or these are new MCP tools that you want to code to support your agents.

17:06 We make that possible in Foundry.

17:09 So let's jump into a quick demo.

17:18 Balan Subramanian: This is the new Foundry portal,

17:20 and we can start by creating an agent here.

17:22 But to make this process simpler,

17:24 I've already created some agents where we can look at it

17:26 step by step on how to get to the full returns agent.

17:30 So let's start with this.

17:32 To start with, we have some very simple system instructions for the model,

17:37 and we're going to add some tools.

17:40 The first tool we want is a tool to get our order details from Shopify.

17:44 To do this, we'll go here, add a new tool.

17:47 And as you can see, Foundry has a lot of built-in tools,

17:51 but it also has this catalog where you can find remote MCP servers,

17:57 local MCP servers, and also Logic Apps connectors available as MCP

18:01 servers that you can customize and use in your agents.

18:05 So we'll go ahead and search for Shopify.

18:09 There you go.

18:10 Click that.

18:13 To run these MCP servers, Foundry uses Logic Apps.

18:17 So the first time you create a tool in a Foundry project,

18:20 a Logic Apps resource is created.

18:22 But after that, the same Logic Apps resource is used for other tools.

18:32 So now we are in the Azure portal.

18:36 And we'll start by giving this a name.

18:45 The more descriptive we can make our tool and also the actions available in it,

18:50 the easier it is for the agent to start using them.

18:57 So we need to provide a connection to Shopify.

19:00 And to do that, you need the API key from Shopify and also the store URL.

19:06 Let's go ahead and use a connection that I've already created.

19:09 And now we can add the actions that are available through this tool.

19:12 So these are the actions that are available through the Shopify connector.

19:16 And let's pick this one.

19:22 And we can also configure the parameters

19:25 that are required for this tool to function.

19:28 In this case, we need the order ID.

19:32 And we can specify if the order ID is going to be provided

19:35 by the model dynamically or if it's going to be hardcoded at this point.

19:40 So you can imagine the use case where

19:42 a tool is going to be used by different agents,

19:45 and it always has to behave the same way.

19:48 So you might want to hardcode some of these parameters.

19:50 But in this case, we're going to let the model provide the parameters.

19:54 Save the changes and register.

19:56 This will go ahead and create the Logic App connector to create

20:01 the corresponding MCP server and make it available as a tool in Foundry.

20:06 The nice thing about this is you didn't

20:08 have to know the full API signature of Shopify.

20:11 And you can set it up in a way that the agent can dynamically figure

20:17 out what parameters are needed and what the shape of the request needs to be,

20:22 and send that request over to this tool.

20:25 So we can now go ahead and add this to any of our agents,

20:27 but let's switch over to an agent that I've already created that has this tool.

20:32 So now, this agent has more instructions.

20:36 So we're going to ask the user for the order number.

20:38 We're going to use the tool that we just added to get the details of that order.

20:44 And we're going to ask the user which

20:46 product from the order they want to return.

20:51 So now, let's go ahead and add the File Search tool.

20:55 This is going to help us add all

20:57 the policy documents from the different suppliers for our store.

21:03 So we'll add all these documents.

21:08 And what this is going to do is create a new vector

21:11 index based on all the policies that are in these supplier-provided documents.

21:18 And it's going to make that index available to all the agents in this project.

21:25 So you can create a new index if you want,

21:27 or you can use one of the available indices.

21:31 So we'll go ahead and switch to one

21:32 of our agents that already has this tool added.

21:41 So as you can see, now we have more instructions.

21:46 Basically, the agent is going to use

21:49 the PDFs to reason about the return and make

21:52 a decision on whether the return is allowed

21:55 or not based on that particular supplier's policy.

21:59 And it's going to let the user know about its decision.

22:03 So now we have two tools.

22:05 Let's add another tool.

22:07 This one is going to be a function that I've already created.

22:11 The function is pretty simple.

22:13 It basically takes in package dimensions and weight and decides if

22:18 a package is overweight or it's a package that requires pickup.

22:24 And then the agent can use that information to offer pickup to the user.

22:28 So this is an example of a scenario where you don't have an existing MCP server,

22:34 and you have to create a new MCP server.

22:37 It's very easy to create these MCP servers with functions.

22:40 Basically, in your functions code, you add the MCP tool trigger,

22:44 and that automatically makes it available as an MCP server.

22:48 And you can simply use the URL

22:54 of the function together with the MCP server app key,

23:02 and this will allow your agent to now talk to this function through MCP.

23:10 And to do that, we'll go here, we will basically provide a name for this tool,

23:17 provide the URL that we just looked

23:19 at, and provide the key from the Azure portal.

23:24 Again, we'll go back to an agent where I already have this tool.

23:30 So now we've added more instructions that basically

23:35 tell the agent to, in some conditions,

23:41 use this particular tool to figure out if a pickup should be offered.

23:44 Now, the next tool we want to add is going to be the Shipping Rates tool.

23:51 To do this, let's assume I want to get shipping rates,

23:55 but I don't have a tool in mind.

23:57 So I can just go to the catalog and search for shipping.

24:04 And there are some options that are available here,

24:07 and we can go ahead and pick "Easyship." Now,

24:10 you do have to create an account with each

24:13 of these providers and get the API key and provide it.

24:16 But the really nice thing here is all of these APIs,

24:21 you don't have to really know the shape of the API.

24:24 You can let the agent figure out the shape of the API.

24:28 And also, it provides you a good starting point.

24:31 So if you're ever looking for a tool,

24:33 you can go to the catalog and see if the thing that you're trying to accomplish

24:40 is possible through any of these MCP

24:43 servers that are already available in the catalog.

24:46 Cancel out of that.

24:47 We'll go back to our agent that already has this tool as well.

24:52 So now we have all of these tools.

24:54 And we've also added some new instructions that basically say

25:00 if the customer is going to pay for the return shipping,

25:03 then we can provide them with a quote from the Shipping Rate tool.

25:11 And then what we want to do is

25:14 call our existing business process that— let's say

25:17 the organization already has a business process where

25:20 once the process is— once a return is approved,

25:23 we can generate— the supplier needs to be notified,

25:27 some databases need to be updated,

25:29 and the shipping label needs to be generated and emailed out to the customer.

25:35 Let's say all of this is an existing business process.

25:39 So this is a Logic App that implements this business process.

25:45 And what we want to do is use this Logic App as an MCP tool.

25:52 To do that, we go to our AI gateway,

25:56 and we simply import that Logic App—— as an API.

26:04 And once we've done that, we can go to MCP

26:07 servers and create a new MCP server using that API.

26:13 I've already created one here.

26:15 This is the Return Execution tool.

26:17 And I can go back to Foundry, and just the same way I added the function,

26:24 I can add this by providing the URL to the MCP tool,

26:30 and also the key to talk to it that I can get from the AI gateway.

26:36 One interesting point here is now the URL is actually the AI gateway.

26:41 So all the interactions between the agent and the tool

26:44 are now managed and governed by the AI gateway.

26:50 So let's switch to one of our agents here.

27:04 And we now have the final return agent.

27:07 So as you can see, this has a whole bunch of steps that it's executing.

27:10 And it has a whole bunch of tools

27:13 that the agent can use as needed for these steps.

27:19 Now, one thing I want to note is this is not a best

27:22 practice in terms of having one agent have so many different instructions.

27:27 What we found to be a better practice is splitting this up

27:31 into independent agents and using workflows to put these agents together.

27:38 And now this entire workflow can be made available as one agent

27:43 that can be accessed from your chat

27:46 application or from another business process.

27:49 For now, we'll continue to use the return agent.

27:56 And then run our agent.

27:57 I'm going to do this in the Foundry chat playground.

28:00 Typically, you would integrate this agent into a chat application,

28:03 but using the playground allows us to see

28:06 all the different steps that are happening.

28:28 So in the playground, we show you all the different MCP tool invocations.

28:33 In a real chat application, you wouldn't be showing this to the end

28:36 user and asking them to approve each invocation.

28:39 But here in the playground, it allows us to kind of look at all

28:42 the different parameters and the responses that we're getting.

28:47 So now the agent is using the Shopify tool,

28:50 and it's fetched some of the details from the order.

28:55 And it wants to know both the item to be returned and the reason.

28:58 Let's go ahead and just first give it the item number.

29:04 So at this point, the agent has found more information about the item,

29:09 but it still needs information about why the item is being returned

29:13 before it can do its reasoning about whether the return is eligible.

29:29 So now, the agent has done its reasoning

29:34 using the policy document of the relevant supplier,

29:38 and it has figured that it can be returned.

29:42 And now it's checking— it's going to use

29:44 our function to check if the package is oversized.

29:47 So the nice thing about agents is that they can sometimes

29:53 use their global knowledge when the specific knowledge is not available.

29:59 So in this case, the details about the dimensions are

30:03 not available as part of our inventory database in Shopify.

30:07 So it's just using some typical values for such items.

30:11 Now, this is not appropriate in some use cases,

30:14 but in a lot of use cases, it is okay to assume these typical values.

30:19 So we'll go ahead and use that.

30:26 While that's happening, I also— well, it finished.

30:30 So now, it's decided that the item is oversized,

30:34 and a pickup will be offered by TechNova.

30:38 And now it wants my email.

30:40 Now, the interesting thing is, you know,

30:43 sometimes this data is already available to the agent,

30:46 and you just need to give it a hint.

30:57 Okay.

30:57 So that's been helped, and the email address was part of the order

31:03 details that the agent pulled from Shopify.

31:07 And—— we will basically tell it to go ahead.

31:15 And now, it's going to invoke our business process in Logic Apps

31:21 that handles the creation of the shipping label and emailing it to the user.

31:26 I'll go ahead and approve that as well.

31:35 Gmail account, we would see that we've now received

31:38 that email with the return details for this particular order.

31:43 And going back to Foundry, as a developer,

31:45 you can see the trace of each conversation.

31:49 Let's pick the last one that was completed.

31:51 So you can see all the calls that were made, all the interactions with the user,

31:57 and the corresponding MCP tool invocation and the response of that invocation.

32:05 So that was just a simple example of how you can use the power of the LLM

32:10 to do reasoning and equip your agents

32:13 with tools and basically implement an end-to-end business process,

32:18 or an agentic business process now that leverages existing APIs,

32:23 existing functions, and existing business processes,

32:26 but brings them all together to provide a great experience for your end user.

32:33 Shawn Henry: So where we are is we've kind of built our agent,

32:36 and now we're going to move it and see

32:38 how we can add agents and capabilities in code agents.

32:43 And what Balan was showing you is how you can very quickly

32:45 and easily build really powerful agents

32:48 directly in Foundry by creating your instructions,

32:51 connecting your tools, testing them out, evaluating them.

32:54 But sometimes there's no substitute for the expressiveness

32:56 and the complexity that you can create building in code.

33:02 So in order to help with that, in Azure AI Foundry and Agent Service,

33:08 we have the Microsoft Agent Framework,

33:11 which allows you to build agents in Python and C#,

33:16 connect them together into multiagent systems and workflows,

33:19 and then deploy those into Foundry using our new hosted agent service.

33:24 Then once they're in Foundry, you can orchestrate them just like you can

33:26 orchestrate any of your agents together in Foundry.

33:28 You can use them, test them, evaluate them, connect them into M365, into Teams.

33:33 All the things that you can do with your declarative agents in Foundry,

33:36 you can do with code agents.

33:39 And all this relies on a new feature

33:41 we announced today for Foundry called Hosted Agents.

33:44 And what Hosted Agents does,

33:46 it allows you to take agents built with Microsoft Agent Framework

33:49 or LangGraph and then use our developer tools or Azure developer tools,

33:53 wrap those up in a container.

33:55 We'll do all that work for you.

33:56 We'll host it up in Foundry so that anything that you've

33:58 built in code can now be managed as an agent inside Foundry.

34:05 So let's take a look at that.

34:06 I will head over here.

34:11 Oh, man, someone has been messing with my machine.

34:14 All right.

34:17 So we can take a look.

34:18 If we're building in Microsoft Agent Framework or LangGraph,

34:21 we can specify our agents in code.

34:23 We can pull them from templates.

34:25 We can do all the things that we could do in the Foundry portal.

34:28 So you can see here I have an agent I've created.

34:31 It's got a name.

34:36 It's got a prompt.

34:37 We've connected to some tools.

34:38 These tools are built in code.

34:40 We can access anything that we can access in code.

34:42 And it's got custom structured inputs and outputs that we can use so we can make

34:47 sure that our agents always output data

34:50 that can interface with the rest of our systems.

34:53 We can create a lot of agents.

34:55 So this workflow I'm going to create

34:56 is going to generate coupons for our customers.

34:59 So it's going to look at what loyalty tier are customers in.

35:03 It's going to look at what they're returning.

35:04 It's going to go look at social networks, see,

35:06 evaluate the product and some of the reviews of the product.

35:09 And it's going to decide based on that if it should give a coupon

35:13 for that specific product or if it should

35:15 give a more generic coupon for the entire store.

35:17 So we have, what, six agents here,

35:18 and we're going to orchestrate these together into a workflow.

35:21 And we do that again in code.

35:24 In the same way, you can kind of imagine we have

35:26 this graph and we're connecting these edges of all these agents together.

35:29 And we can even do things like map reduce and fan in, fan out,

35:33 and all the kind of things you would expect from a workflow engine in code.

35:40 And so we go ahead and run this, and actually I have it running,

35:43 hopefully, over here.

35:44 And we can see it.

35:45 So this is the workflow we've generated.

35:49 You can see we have these agents strung together.

35:50 They're passing data between them.

35:52 Our orchestrator agent makes a decision and then

35:54 decides what kind of coupon we want to create.

35:56 So we're using our dev UI here as part of agent

35:59 framework to actually run this, and it's asking us for our inputs.

36:02 Most of these have some defaults, but I'll put my name in here.

36:05 And then we'll run it.

36:06 And we can see our agents actually running.

36:09 We're starting off by taking that input data,

36:12 looking at the loyalty program that was given to us.

36:14 Let's see here.

36:15 I think we— are we going?

36:19 There we go.

36:19 Maybe?

36:20 Yes.

36:21 Okay.

36:21 So loyalty agent running.

36:22 Now the social agent is going out, checking all the social networks,

36:25 using all our tools and connectors that we built directly in code,

36:28 gathering that data, and then sending that off to our orchestrator agent,

36:31 which looks at the data, decides what type of coupon, a product code,

36:34 or a store discount and then generates an email

36:37 that we can now send to our customer with that information.

36:40 All that happening in code.

36:42 But what we really want to do is we want to integrate

36:44 that with the agents that Balan showed

36:47 you earlier that he wrote declaratively in Foundry.

36:50 So very easy if you're familiar with Azure developer tools,

36:54 and even if you're not, if we've set up our project correctly.

36:57 So I have over here, I've set up my project.

36:59 We have tools to generate all the Bicep, and JSON, and Docker files for you.

37:07 And so all you need to do is you need to type a— oh, not up here.

37:12 All you need to do down at the bottom, you guys can see,

37:14 I am typing azd up, which will package all this stuff up together,

37:20 send it up into Foundry.

37:22 But that takes a little while.

37:23 It takes a couple minutes to do that.

37:24 So what I'll do is I'll pop right back over into Foundry.

37:28 And this is Balan's agent here that he was showing you earlier.

37:31 But we can see earlier I created a hosted agent here.

37:34 So this is an agent that is using that code that I deployed up into Foundry.

37:38 We can't modify it once it's in Foundry because that was all done in code.

37:43 All the instructions and tools were done in code.

37:45 But I can talk to it.

37:46 I can have conversations with it.

37:47 I can look at the traces.

37:49 I can monitor it.

37:50 I can evaluate it.

37:51 I can see all the things it's doing.

37:52 And more importantly, I can connect it to my other agents using workflows.

37:58 So I've created here a workflow using Balan's agent, which is the return agent.

38:04 And it loops through.

38:05 But Balan, if his agent had been working,

38:07 he would have gone through the loops and it would have gone through all

38:10 the steps that he showed and determined

38:12 if the customer was eligible for a return.

38:16 And once it had decided that it was going to process the return,

38:19 what we want to do is we want to call our code agent.

38:21 So just like I've created the return agent here in our workflow,

38:25 if we decide we want to process a return,

38:27 this is saying if the response is not process return.

38:30 So if we're processing a return, we'll go over here.

38:33 We'll invoke an agent.

38:34 I'll pick my agent from the dropdown.

38:36 I've got my list of agents that are from my "Agents" tab.

38:39 I'll connect that.

38:40 And now I can run this workflow within Foundry

38:43 with a combination of all the agents that I've built in Foundry.

38:47 We have code agents.

38:48 We have declarative agents working

38:51 together and orchestrating themselves within Foundry.

38:54 Now, what I really want to do, though,

38:55 is I want to take these agents orchestrating together

38:58 and I want to use them with my business process automation.

39:01 So I'm going to have Balan come back up here with his freshly refreshed laptop,

39:05 and he's going to show us how we can

39:08 connect this workflow to some existing business process automation.

39:14 Balan Subramanian: All right.

39:14 As soon as I walked back there, the network started working.

39:17 So hopefully it stays.

39:21 But let's switch to slides first for a minute.

39:30 So what Shawn showed us was how you could build a team of agents.

39:33 But now in many organizations you already have existing

39:37 business processes that you want to enhance using agents.

39:42 So Microsoft's platform for business process automation is Azure Logic Apps.

39:46 So in addition to the integrations that we showed

39:49 earlier that are now also made available in Foundry,

39:52 Logic Apps also gives you a battle-tested

39:55 platform for running critical business processes.

39:58 That includes different kinds of workflows.

40:00 So these are not just workflows among agents but also

40:03 workflows that you would typically encounter in an enterprise,

40:06 right, things like updating databases,

40:09 things like implementing transactions across multiple systems.

40:13 These are some of the things that Azure Logic Apps does.

40:16 Now, Azure Logic Apps is very mature technology.

40:19 It's used by over 100,000 enterprises worldwide.

40:24 It's running about 80 million executions every month.

40:29 And now what we are enabling is the opportunity

40:32 to bring some of the agents that you've built,

40:34 whether you built them using code or visually, into Logic Apps.

40:40 And the way we're doing that is through the Logic Apps Agent Service Connector.

40:46 We're excited that this is going GA.

40:48 Today we've learned a lot from the public preview,

40:51 and we have incorporated that into the product.

40:53 So what the agent service connector does is lets

40:55 you seamlessly call agents that are in Foundry or elsewhere.

41:00 It uses the A2A protocol.

41:01 So you can also have agents running in GCP or in AWS,

41:06 though we hope you do them all in Azure,

41:09 and incorporate them into your existing business processes

41:13 or new business process implementations that you're building.

41:17 So with that, let's switch back to my machine and pray it works.

41:28 All right, so this is an existing Logic App that I have,

41:32 and this one is the same return process.

41:34 But assume that this was the original

41:36 return process where the company basically uses ServiceNow.

41:40 A user calls into the customer service center,

41:43 and a human basically takes in all

41:46 the information about the order to be returned, why it's being returned,

41:49 things like that, and creates a ServiceNow ticket for it.

41:54 And at some point, another human picks up that ServiceNow ticket.

41:59 Logic Apps provides integrations into Teams

42:02 and various other communication tools.

42:05 So you can assume that a Teams notification popped up,

42:08 and a human basically now looked at this tool.

42:12 And then they had to go figure out who is the supplier,

42:14 look up their PDF documents.

42:16 And you can also imagine the complexity

42:18 when these documents could keep changing,

42:20 like maybe the supplier changes policies every six months.

42:24 And ultimately, they update the ServiceNow ticket again with the addition.

42:30 And this Logic Apps workflow basically is

42:34 listening for updates to that ServiceNow ticket,

42:36 and it's able to send an email to the customer.

42:39 Now, this is a process that could take hours or even days.

42:43 And now with agents, we can make that much faster.

42:48 So this is the same process or the same workflow.

42:52 But now, instead of sending the ticket to a human and waiting for a response,

42:58 it's basically using the agent service connector

43:01 to start a conversation with the agent, just like a human user would do.

43:06 And it basically awaits the response and then it

43:09 continues the same process of sending the email out.

43:12 So now let's see how this would actually work in action.

43:15 This is ServiceNow.

43:16 So we'll go ahead and create a new ticket.

43:18 And let's say Nathan is the person who called

43:23 into the call center and asked for a return.

43:31 I have an order number here.

43:42 Inaudible] (Laughs) thank you, kid.

43:50 All right.

43:50 Now we can go back to our Logic App.

43:54 Let's go look at the run history.

43:58 And that looks like the one that just ran.

44:01 So let's take a look at that.

44:05 And you can see how it kind of picked up the ticket,

44:09 took all the details from it.

44:11 It called the agent.

44:14 And basically we can look at all the inputs and the outputs.

44:25 Well, I'll have to look at the agent response.

44:28 So basically this is the email that's going to be sent back to the user.

44:32 And assume this is Nathan Miller.

44:37 And now this is the retention coupon that was sent out automatically.

44:42 And what we just did was process

44:44 the return and also generated the retention coupon.

44:47 So just zooming out a little bit,

44:48 if you look at this from a perspective of a business process,

44:51 we made the process faster for the end user, who is the customer in this case.

44:56 We made it less human intensive.

44:59 So there is less things that a human has to do to execute this business process.

45:05 And we also enhanced the business process.

45:06 So in addition to not only processing the return,

45:10 we are also able to do things like generate

45:12 retention offers and send it to the end customer.

45:18 So there's more to agentic process automation.

45:23 And later on I have some links to some

45:25 of the other sessions that go deeper into that.

45:28 We saw a couple of ways to build agents and use them in Logic Apps.

45:32 But you can also build these agents through Logic Apps,

45:35 just like we built them with code and built them using the Foundry portal.

45:40 Logic Apps is another way where, as part of your business process,

45:44 you can start creating agents that ultimately end up running in Foundry.

45:49 So this is kind of what we went through so far.

45:52 The last one left is governance.

45:54 And as you can imagine, as developers build more and more agents and start using

46:00 more and more tools and start sharing them across the organization,

46:04 organizations are going to need a way to get

46:08 visibility and control over these agents and tools.

46:10 And that's what the Foundry Control Plane provides.

46:14 Now, a big part of the Foundry Control Plane is the AI gateway.

46:18 This is APIM, Azure API Management.

46:21 This is the same gateway that Robert mentioned

46:24 that KPMG uses to orchestrate the different agents and tools.

46:28 What's exciting about what's happening now

46:30 is we are incorporating this into Foundry.

46:34 So right from within Foundry, you can create an AI gateway.

46:37 You can use it to manage how your agents use different models.

46:41 You can even switch between models and model

46:44 versions without changing any of your agent code.

46:47 You can take existing APIs and make them available as MCP servers.

46:51 You can also put other MCP servers behind API management so that you get another

46:56 point of visibility and control into how

46:58 those MCP servers are being used by your agents.

47:01 And not only that, the AI gateway also

47:04 helps you incorporate agents that are running elsewhere.

47:07 Like maybe you have, like I said before,

47:09 unfortunately agents running in GCP or on AWS.

47:12 You can register them into the Foundry Control Plane using the AI gateway.

47:16 And now all traffic to those agents flow through the Foundry Control Plane,

47:21 giving you better visibility and control.

47:25 So we'll switch back real quick.

47:27 I just want to show you the Foundry Control Plane and how the AI gateway works.

47:32 So under "Operate" in Foundry,

47:35 if you go under "Admin," there's the AI gateway section.

47:40 You can add new AI gateways.

47:46 Well, it's loading.

47:48 Well, while that's loading, let's go and look at the actual AI gateway.

47:52 This is Azure API Management.

47:54 And as you can see, some of the MCP

47:57 servers that we built previously now show up here.

48:00 And you can put additional policies in place.

48:03 So this is a simple example of a rate-limiting policy

48:07 for the Azure function that we converted into a MCP tool.

48:11 If you remember, this was the tool to check for oversized packages.

48:16 Maybe this is an expensive operation for you to run, so you want to throttle it.

48:20 And that's easy to do simply by adding a policy.

48:23 And the nice thing about this is this policy

48:25 applies to all usage of this tool within your organization.

48:29 So rather than trying to hunt down how

48:31 each agent uses different data sources and different tools,

48:34 now you can manage that all centrally through

48:37 the Foundry Control Plane using the AI gateway.

48:42 And as you can see, the same AI

48:44 gateway can be used against multiple Foundry projects.

48:48 And it will apply all the content safety policies, all the API access policies,

48:53 and data access policies uniformly across all of those projects.

49:02 So that kind of brings us towards the end.

49:05 And just going over what we did today,

49:08 we created some agents that can do reasoning on their own.

49:13 We equipped those agents to talk to different business systems.

49:17 We showed how to create agents through code and how

49:19 to put all these agents together to create teams of agents,

49:22 and then incorporate these teams of agents into existing business processes,

49:27 and finally get management and governance

49:29 across the entire application landscape.

49:33 So these are some of the related sessions that I would invite you to go to.

49:37 Shawn, our expert, will be talking about AI-powered automation and going deeper

49:43 into the Microsoft Agent Framework and Foundry

49:46 workflows in this deep dive session.

49:49 The API Management Team will be talking about the AI gateway in more detail,

49:53 showing you how to apply different policies and how to manage both models,

49:59 and tools, and agents using the AI gateway.

50:02 And finally, the Logic Apps Team will be

50:04 talking about agentic process automation and how you can,

50:08 whether it's existing business processes that you're

50:11 updating or new business processes you're building,

50:13 how you can bring the power of agents to both

50:16 of those through Logic Apps and its integration with Foundry.

50:20 And we would love to hear from you.

50:22 We are early in the agentic process automation space,

50:27 and we would love to hear from you about what

50:29 you plan to do with agents and what challenges you have.

50:33 We'll be at the Hub.

50:34 We have a couple of booths, and the entire team will be there.

50:38 So thank you, and I look forward to seeing you at the Hub.

50:45 Applause]