Nasdaq Boardvantage: AI-driven governance on PostgreSQL and Microsoft Foundry | BRK137

0:01 All right, let's make a start.

0:03 We've got a relatively large amount to cover.

0:05 So hi everybody.

0:06 Thanks for for joining the session today.

0:07 My name's Charles Ferguson.

0:09 I lead the product management team for Postgres at Microsoft

0:12 and I'm joined by Motion who will come on stage

0:15 shortly from NASDAQ to talk about how they've built

0:18 their board Vantage solution on a variety of Azure services.

0:22 So I was going to kick us off here.

0:25 It's a relatively short agenda.

0:26 Their motion's going to expand upon that in a little

0:29 bit more depth when he gets here.

0:31 But the basic content of the session today is going to cover

0:34 everything that you can fundamentally see on on the screen here.

0:38 So NASDAQ are deploying these solutions using two of our open

0:42 source databases in in Azure today, Postgres and MySQL,

0:45 but then complementing that with a number of other key services,

0:49 most notably Foundry, where they're leveraging their AI models.

0:53 And Motion's going to walk you through, you know,

0:56 how they're using AI in the board Vantage solution for doing a lot

0:59 of work on documents and text to bring a lot of efficiencies and optimization.

1:04 We'll talk about the models, talk about document intelligence and how

1:07 they implemented that over a previous way.

1:09 They were doing things to find a lot

1:11 of efficiencies and improvements in quality as well.

1:14 We'll talk a little bit about API management and then Kubernetes,

1:17 which is the the underlying application infrastructure for for all of it.

1:21 Before we get into that, however,

1:23 I just want to give a few minutes here to talk about what we've done

1:26 specifically in Postgres at Microsoft over the last

1:28 couple of over the last couple of years.

1:31 Many people are often unaware actually that Microsoft is one

1:35 of the largest committers to the open source Postgres project.

1:40 And you know, Postgres has found itself in this position,

1:42 particularly over the last couple of years.

1:45 And I think you know,

1:46 by virtue of the fact that when the ChatGPT tidal wave broke,

1:49 Postgres was one of the only databases that have had

1:53 a native vector index directly within the database at that time.

1:56 And if you take a look at the popularity of PG vector,

1:59 it basically hockey sticked upwards from that point in time,

2:03 February about 2023, it was already had strong developer preference

2:06 and we've seen that continue to grow.

2:09 But amongst the hyperscalers,

2:10 Microsoft has been the largest committer over the last three years

2:14 and it's part of really how we approach working in the Postgres ecosystem.

2:19 We build and ship Postgres services.

2:21 We ship Azure Database for Postgres and we've

2:24 announced Horizon DB at Ignite as well.

2:27 But part of being a responsible member of that community

2:30 is taking what we learn from open source software,

2:32 running it at scale in the cloud and committing

2:35 it back upstream into the core open source project.

2:38 And that is exactly what we do.

2:39 It is a virtuous loop that just continues around.

2:42 It makes Postgres better for for everybody and it's

2:44 how we participate in that in that community.

2:48 Now at Ignite, we've had a couple of announcements specific to flexible server.

2:52 I'm not going to drain the entire list here,

2:54 but two things that I really want to call out is how we've

2:57 lit up the latest generation of skews directly into the flexible server service.

3:01 We've basically doubled the maximum number

3:02 of E course that are available for you.

3:04 We've made improvements to our storage as well,

3:07 which are not mentioned here as well,

3:09 but that's enabling high availability on a more performant version of SSD.

3:12 The other one that I just want to talk

3:14 about is in the lower left hand corner here,

3:17 the post Chris extension for VS Code.

3:19 So we shipped this actually,

3:21 we announced this actually at Pycon just before build

3:23 and then re announced it at build earlier this year.

3:26 I think we're close to about 280,000 installations right now.

3:29 We've seen a ton of positive feedback,

3:30 a lot of interaction with that community who have asked us to fix little things.

3:35 We've been able to turn fixes incredibly

3:36 quickly and we're seeing really strong adoption.

3:39 And our vision with this extension in Postgres is to offer

3:42 effectively the full life cycle from provisioning to managing to developing,

3:47 leveraging Copilot for more productive,

3:50 productive development and then also debugging,

3:52 monitoring and then migration as well.

3:55 And you've seen a lot more of that come

3:58 to light at Ignite this year in other sessions.

4:01 But if you do use VS Code and you're interested,

4:03 go take a look for the VS Code extension from, from Microsoft.

4:06 It's got about, as I said, 270,

4:08 two 180,000 installations today and you should be able to to spot that one.

4:14 Now the big announcement that we made Ignite this year was around Horizon DB.

4:17 Horizon to be is our new cloud native Postgres service where we've

4:21 basically built a new storage capability that is zonal resilient by default.

4:25 There was a great session given earlier this morning by Denzel and Adam

4:29 that went into a lot of depth on how this works technically.

4:32 But basically, it's designed to deliver

4:34 much higher transactional performance throughput,

4:36 much higher vector search performance

4:38 with a full zonal resiliency configuration so

4:40 that you can run these mission critical

4:43 workloads with the highest levels of availability.

4:45 And then very simple failovers that sort of make it a little bit more,

4:50 improve the availability,

4:51 reduce the downtime relative to perhaps self managed Postgres as well.

4:56 So this is in private preview right now.

4:58 There is a sign up form that you can get started on.

5:00 I think we've got a slide for that towards the end of the session.

5:03 But really what I want to do is hand it over to Mohsen.

5:05 So before he comes up here, we've got a short video we're going to play

5:08 and then I'm going to give it to Mohsen

5:09 for the rest of the sessions to walk you through

5:11 how they built board Vantage on all of this Azure technology.

5:14 So thank you.

5:17 Where's my video and hit play?

5:29 This milestone marks the culmination of our future of the boardroom floor,

5:33 which is a celebration of bold ideas, transformative technologies,

5:36 and strategic partnerships that are shaping the future of governance.

5:42 It's been such a thrill to be building AI driven products with NASDAQ.

5:47 AI changes the way we work, it changes the way we live,

5:50 and now it's reimagining the way that we can

5:53 make the most important decisions at the highest level.

5:57 We are reimagining how boards operate, empowering boards to lead with clarity

6:01 and confidence in a world that demands agility, insight, and precision.

6:11 Thank you very much Charles.

6:12 It is really fantastic to be here.

6:14 Hello everyone.

6:15 My name is Mohsen.

6:16 I am a senior manager of engineering at NASDAQ

6:19 and I work with the NASDAQ board Vantage platform.

6:22 I'm here today to talk about our AI

6:24 driven governance on Azure Foundry and postcard Sequel.

6:26 But before we go there,

6:27 I'd like to walk you through the agenda so you kind of know what you're in for.

6:31 The picture Microsoft has used here is from a few years ago.

6:34 A lot of things have changed since then.

6:37 One thing that has really changed is how we do governance in in 2025.

6:42 That's exactly what we're going to be talking about.

6:44 We will talk about Nasdaq's cloud journey and some of the challenges that we've

6:48 had with that and how Azure helped us to to solve those challenges.

6:52 We will then talk about enabling AI on our platforms and some of the features

6:55 that we've put into the hands of our customers over the last two to three years.

7:00 We will be doing a technical deep dive and I'll

7:02 be demoing a couple of features for you guys.

7:05 As a tech guy, I know sometimes in events like

7:07 these, the demo gods are not very kind to us.

7:10 So in case something goes wrong, Fred,

7:11 not we have backup videos, so you'll be switching to them.

7:14 But hopefully everything should be fine.

7:16 And then we'll be doing some architectural deep deep

7:18 dive to explain some of the architectural decisions we made.

7:22 And then eventually, if there's any learnings and evolution on our platforms,

7:25 we'll be talking about that as well.

7:29 So why does governance matter in 2025?

7:33 Governance is no longer about oversight.

7:35 It's actually about foresight because we are living

7:38 in a technical revolution that's being driven by AI.

7:41 The decisions need to be made really

7:44 fast because stakes are really, really high.

7:46 Governance is what is keeping leadership aligned, agile and accountable.

7:52 And now at NASDAQ we want to solve a problem.

7:55 That problem is where traditional ways

7:57 of working are becoming less and less relevant,

7:59 where decisions have to be faster,

8:01 smarter, and be made with greater transparency.

8:04 This is where NASDAQ Board Vantage steps in and really shines.

8:07 With our AI driven workflows like smart document summarization,

8:11 AI generated meeting minutes, contextual insights into your upcoming meetings.

8:15 We are transforming governance from a manual

8:18 reactive process to a proactive strategic advantage.

8:21 I'm happy to to say that since we've released our AI features to production,

8:26 our customers report more than 25% of time savings annually.

8:31 That is more than 100 plus hours saved on manual workflows

8:34 like executive summaries of board

8:37 documents and manually creating meeting minutes.

8:40 So that's 100 plus hours saved.

8:42 It's it's, it's fantastic savings for for our customers.

8:46 However, it started with something slightly different.

8:49 It started with our move to the cloud.

8:51 Now I know this session is is mainly about

8:53 our AI foundry and postcard sequel in our AI workloads,

8:57 but I think it's really important to understand

9:00 how we did what we did and the role

9:03 that our partnership with Microsoft and particularly Microsoft

9:06 Azure played in, in driving our innovation forward.

9:11 Today, NASDAQ Board Vantage serves more than 4500 companies worldwide.

9:17 We are serving almost half of Fortune 500 organizations.

9:21 It is really important for us to speak the same language that our customers do

9:25 as most of our customers are already in the cloud or are on their way there.

9:29 As I'm sure is the case with all the organisations represented in this room,

9:33 you are in some way, shape or form towards your journey in the cloud.

9:37 So when we are onboarding our customers,

9:40 Azure really simplifies that onboarding process because we can speak the same

9:44 language when it comes to cloud standards such as data privacy,

9:47 security, cross region deployments and everything that comes

9:50 with the with being on the cloud.

9:53 Now, NASDAQ is also a, like I mentioned, a global platform,

9:56 which means we operate in different parts of the world.

10:01 And that while amazing,

10:02 means that we have to comply with regulations pertaining to data

10:07 residency and data processing in different parts of the world.

10:11 When we were in the data centre,

10:13 we were managing this by, you know, having data centres in different regions.

10:17 So being on Azure meant that that had to, to, to meet that requirement as well.

10:22 And now you might be thinking, what is this guy talking about?

10:24 Of course, being on the cloud means that we

10:26 will be meeting all these requirements, but let me explain.

10:30 NASDAQ Board Vantage today is made-up of many moving parts from Azure

10:36 Kubernetes Services to multiple databases

10:38 to to BLOB storages in Azure key walls.

10:42 And now with Azure AI Foundry where we are deploying different models,

10:46 all of these components have to follow that exact same standard

10:50 when it comes to a regional data residency and and processing.

10:54 So for example, if I'm going to deploy a 4.1 mini model in Australia E,

10:59 it has to be processing it's data in Australia E for that customer.

11:05 And again I would I'm happy to say

11:07 that Azure has has really delivered in, in that aspect.

11:10 NASDAQ board Vantage is also a multi tenanted platform.

11:13 What that means is that for each

11:15 of our 4600 customers or actually more than 4600 customers,

11:20 we are segregating them based on on databases and file storages.

11:24 And then on top of that, we are providing them a extra

11:28 layer of security by encrypting each customer's data with their own key.

11:32 Now when we wanted to move to the cloud,

11:35 it had to to to satisfy these primary requirements of our backbone architecture.

11:40 And I'm happy to say again that moving to Azure,

11:43 we found a one to one fit for pretty much everything that we had.

11:46 So for example, when we were migrating our database,

11:50 the Azure database for my sequel server was

11:52 a perfect fit for our multi tenant system.

11:56 We we were we could migrate everything 1:00 to 1:00.

11:58 And then on top of that, we got high availability, auto scaling,

12:01 auto grow and we didn't have to worry

12:04 about managing our own infrastructure over there.

12:07 Then when it came to customer security keys,

12:10 we were using HSMS in the data centre.

12:13 Moving to the cloud,

12:14 Azure offers AKV which is doing exactly the same thing, but again,

12:17 now we don't have to worry about high availability

12:20 and everything that comes with maintaining the the hardware.

12:24 And then for for for file storages.

12:27 The Azure BLOB storage was a perfect fit as well because now we could,

12:31 in the BLOB store, store each customer's

12:33 data based on their own storage accounts.

12:37 But I would say the greatest benefit that we

12:39 get by being on the cloud is our engineers

12:42 can focus on writing and delivering business value

12:44 without having to worry about maintaining our own infrastructures.

12:48 I don't have to get into those late night calls anymore to worry

12:51 about oh we need to patch this server or a hardware module fail.

12:55 Sometimes I still get called into those calls,

12:57 but those are usually because of the bugs I've introduced into the system.

13:01 But I would say now with the Azure AI Foundry,

13:04 we have access to all the AI services as well,

13:07 which has really helped to drive our innovation forward

13:10 and help to stay relevant in this ever changing technological landscape.

13:16 I'm not going to bore you by explaining each part of this document.

13:19 This is just to explain the gravity of of our infrastructure.

13:24 You can see we have many, many moving parts and components

13:26 from Azure Kubernetes services to multiple databases, event driven architecture.

13:31 Now AI, foundry, BLOB storage.

13:33 And obviously we operate in a high available, highly available environment.

13:37 So each of these regions have a primary and a secondary and all

13:41 of these are replicated across all the the the regions that we operate in.

13:45 So you can see it's quite an extensive architecture

13:48 that Microsoft Azure is helping us to to maintain, right.

13:52 Let's pivot.

13:55 As soon as we had moved to the cloud, I think this was sometime in mid 2022.

14:00 Later that year we all know chat GPD came and it

14:03 changed the way that we are going to be engineering permanently.

14:07 In my opinion now NASDAQ being the transformative company that it is

14:11 recognized the transformative potential of this technology and we got to work.

14:15 But then again, we didn't want to just, you know,

14:17 go to market with the first feature that we could

14:19 have just for the sakes of being the first one.

14:22 So NASDAQ poured a lot of energy, time,

14:25 resources and a few dollars to analyse what

14:28 would make the most sense for our customers, our product people, our business.

14:34 They spoke to our customers,

14:36 identified the the business flows that would make the most sense.

14:39 And this is where we decided that smart document summarization and the AI

14:44 meeting minutes would be the the best strategic advantage for our customers.

14:48 And they've been living in production for, for quite some time.

14:52 And the next step that we wanted to focus on was driving customer adoption.

14:55 I, I'm happy to say that, you know, out of more than 4500 customers worldwide,

15:00 more than 1000 have enabled AI on their platforms already.

15:04 And you might be thinking that's less than 30%.

15:06 You guys need to do better.

15:08 But let me explain.

15:09 We have, we operate in a very sensitive segment of the market.

15:13 Our customers trust us with some of their most strategic sensitive data,

15:17 data that is used to drive business decisions.

15:20 When AI was becoming mainstream,

15:22 there was a lot of challenges that we had to go through.

15:26 We not only had to solve those challenges,

15:27 but we also had to build that confidence and pass it down to our customers.

15:31 So that's exactly where Azure and PostgreSQL has been helping us.

15:35 But before I go there, I'd like to talk about some of those challenges

15:39 that we that we faced and we have solved.

15:42 AI is non deterministic.

15:44 Now as engineers, we have worked with non determinism before.

15:48 We have dealt with large distributed systems where certain

15:51 risk conditions do not execute in the order that you

15:54 expect them to, but they are usually edge cases

15:56 and they can be mitigated with a significant amount of effort.

16:01 AI is inherently non deterministic.

16:03 That's just how it is.

16:04 We have to accept that, work with that and work around that.

16:08 I don't need to impress upon anybody how bad hallucinations can be,

16:12 but in our line of work,

16:13 we are customers are again using data to drive decisions.

16:17 Hallucination can be catastrophic for both us and our our customers.

16:22 So that's also something we have to understand and work with.

16:25 Now, the medium that we use to communicate

16:27 with the AI today is natural language and prompt engineering.

16:31 And we know that natural language is ambiguous.

16:34 Again, as tech people, we work with programming languages.

16:39 Programming languages are built to remove ambiguity,

16:41 unless it's JavaScript, then it's just ambiguity.

16:44 But I digress.

16:45 The point I'm trying to make is that natural

16:47 language is inherently weak when it comes to removing ambiguity.

16:50 Also something we have to to understand and work with.

16:53 Now AI has come a long way since 2022,

16:55 but it is still slow and expensive to deploy at scale.

17:00 And organizations like NASDAQ with products like Board Vantage,

17:03 which are trying to mainstream that we have to be aware

17:06 of that and work with the with the limitations that we have.

17:10 Obviously AI also introduced uses a new way

17:12 of a new challenge of testing and validating the outputs.

17:17 Again, as programmers, we are used to determinism in our unit tests,

17:21 but you can't do a assert equals on the response from your, your AI output.

17:25 So we either have to build new tools or work with tools that somebody else is,

17:31 is building to validate and test AI in this in this changing paradigm.

17:37 Well, I think that's enough for the theory.

17:39 Now I'd like to talk about some

17:41 of the, the AI features and show you some, some demos.

17:45 Now since 2022, we have put a few features in the hands of our customers.

17:49 The ones that I'd like to talk

17:51 about are meeting minutes and the board assistant.

17:54 The meeting minutes is AAI feature that has we,

17:57 we deployed it in March earlier this year.

18:01 I do want to impress upon everyone that a board meeting minute is

18:04 very different from the meeting minutes

18:05 that you would see from a regular meeting.

18:08 These are legal doc records which can be used in court proceedings.

18:13 These these contain important information about deliberations,

18:16 how decisions were made or anything and and basically strategic word exchange.

18:22 Corporate secretaries, after the meeting has happened,

18:24 spend a lot of time trying to formulate

18:27 these meeting minutes to ascertain their accuracy.

18:30 In fact, I was on a call with one

18:32 of the corporate secretaries a couple of weeks ago doing

18:35 a demo for this feature and she told me they

18:38 spend more than two weeks to to finalize these meeting minutes.

18:42 Our tool can do it in less than 5 minutes.

18:45 But before we go any further,

18:47 I'd like to show you guys how this tool actually works.

18:52 So I'll switch over to a demo.

18:55 So there's a few ingredients that goes into to building these meeting minutes.

19:00 You have an agenda document.

19:01 I will upload that.

19:03 An agenda document, as the name indicates,

19:05 will basically outline whatever is going to happen

19:07 in the meeting and who's going to talk about that.

19:10 It's also going to mention any participants in the meeting,

19:13 whether they are joining online or they are joining in person.

19:18 Then you have a meeting material.

19:19 This is basically a board packet,

19:21 and these can be massive documents, up to 400 to 600 pages.

19:24 Sometimes I've seen even more meeting notes.

19:29 Basically is the the the note taker in the room,

19:32 they will record any word exchanges that happen,

19:35 voting decisions, all of those things

19:37 that are not obviously in the board packet.

19:40 And then we allow you to upload an example of the previous meeting

19:43 minutes because we understand that each company has its own way of writing,

19:48 formulating their meeting minutes document.

19:50 We don't want to assume that you want in a certain way.

19:53 We let you upload an example and then we use AI

19:56 to apply that style and formatting into your your linguistic formatting,

20:00 basically into the actual document.

20:03 So I'm going to go ahead and attach an example as well.

20:07 And then we'll press generate.

20:08 Obviously, we're not going to wait for this because it

20:10 can take a while and we don't have a while.

20:12 So I have previously generated a document that I'd like to show you guys.

20:16 And you can see that this is

20:18 a meeting minutes from a Massachusetts for authority.

20:20 Priority meeting and I'm not going to go through the whole thing,

20:24 but you can see how the AI formulated the the entire document.

20:27 Let me just make it a bit bigger.

20:31 And it will based on the agenda that you've

20:34 uploaded is going to Add all the information.

20:37 And then in the end, if there's any key decisions, it will add those as well.

20:40 And then basically it's formatted according to the, the example

20:45 of the meeting minutes that was provided, right.

20:48 So let's talk about what happened here.

20:53 I uploaded a few documents, some magic happened and then you had

20:56 a draft of the the meeting minutes generated.

21:00 Let's try to unravel that magic as I uploaded the agenda.

21:03 We use document intelligence and I'm going to talk about

21:05 document intelligence and the importance of that in just a second.

21:08 We extracted all the information from from the agenda

21:11 and then we applied the agenda processing prompt.

21:15 Obviously, this is a special prompt that parses the agenda

21:17 in a in a way that we're going to use it further.

21:20 We process that with 4.1 mini and then we store

21:22 that in the in the memory for for the processing.

21:26 As a next step, we then extract meeting materials.

21:28 And this is where document is just

21:30 becomes really important because as I mentioned,

21:33 these can be documents of up to 400 to 600 pages.

21:36 When we did not have document intelligence, we wrote our own semantic chunker.

21:40 The idea was that you want to create small chunks of these documents,

21:43 create embeddings and then store them into the into the database.

21:47 That chunker was doing quite well,

21:48 but it was always a guessing game where a section starts,

21:51 where a section ends and sometimes it would not be very accurate.

21:55 When we started using document intelligence,

21:57 basically what it does is it gives you

22:00 a well defined layout of the entire document.

22:03 So you know where it's the heading,

22:04 you know where this a section starts, a section ends.

22:07 And it's particularly very good when it

22:10 comes to recognizing figures and, and other

22:13 diagrams because sometimes a lot of context is missed in those things.

22:17 So once we've done that, we still perform the chunking,

22:20 but now it's easier to to chunk this this document.

22:24 And then we create embeddings with the text embedding

22:27 ADA and persist that into the the Postgres sequel.

22:30 We will be talking a lot more about the Postgres

22:32 sequel with PG vector enabled in just one second.

22:34 The exact same thing happens with the with the meeting notes.

22:38 We process that with document intelligence, semantically chunk it,

22:41 create embeddings stored into the Postgres SQL database.

22:45 And then obviously if you have an example of the document,

22:48 we will process that with the document

22:50 intelligence and we will apply a style guide

22:52 prompt to it and we will keep that on the on the side as well.

22:57 Now that we have processed all

22:59 the ingredients of this this massive board context,

23:02 it's time to start putting it together.

23:06 So how do we put it together?

23:08 We take this processed agenda and then we

23:10 match each of the headings to the stored embeddings.

23:13 And you can see that with Postgre SQL with PG vector enabled,

23:17 it is so easy to run a hybrid search.

23:20 What you can see down here is a Java code snippet

23:23 with a SQL query that has a vector similarity search as well.

23:27 You don't need to.

23:29 There's no rocket science here.

23:30 It's just simple SQL with hybrid search.

23:32 And then you can do a look up based on the cosine similarity.

23:37 And then you can limit up to the number of embeddings that you are looking for.

23:42 The exact same thing happens now that we've matched the embeddings,

23:46 we can apply the prompt because remember,

23:48 what we've selected right now is basically chunks of text, right?

23:52 So when you apply the, now we need to process it

23:54 with natural language so it makes sense to the, the reader.

23:57 So we apply the prompt, process that with the chat completions.

24:01 And now you have the agenda that has been enriched with the meeting material.

24:05 And now the exact same thing happens

24:07 with the with the meeting notes, same process,

24:09 refine it and then add that to the enriched

24:12 draft that you had with the with the meeting material.

24:16 And now if you have a style example optional,

24:18 but if you wanted, we are going to apply that style,

24:21 apply the prompt process that which had completions

24:24 and you have a final drafted meeting minutes.

24:27 I must impress upon each of you that this tool

24:29 is not built to replace the corporate secretary.

24:32 It's built to make their lives easier.

24:35 So instead of spending two weeks on drafting this manually,

24:38 now they spend less than 5 minutes and they can,

24:40 you know, be more productive with their time.

24:43 And then obviously you want to have the human in the loop.

24:45 That's why you get a Word document downloaded.

24:48 The corporate secretary can go back and edit that if she wants,

24:52 or he wants to add more information to it.

24:54 Or if anything doesn't make sense, they could change that as well.

24:58 Now with AI meeting minutes, we proved that we could use PG vector enabled

25:04 in in Postgres Azure database which is natively available.

25:08 You all can go and do that right now.

25:11 We prove that it works for production workloads

25:13 and it was working really well for us, for all our customers.

25:16 So then the next natural step was OK,

25:18 how do we use it in our next feature, which is the the board assistant.

25:23 Now the board assistant is infinitely more complicated than than the meeting

25:27 minutes and it will be fun demoing it to to you guys.

25:30 But the idea with the board assistant is that we

25:33 want to make life easier for for our users, particularly our directors.

25:37 So if you are a, if you're a director heading to a meeting,

25:41 you're probably busy with a lot of things.

25:43 What we want to do is provide

25:45 you contextual insights into your upcoming meetings.

25:47 So for example, it'll tell you what you should be preparing

25:50 for based on whatever has happened in the last two to three meetings.

25:54 And then it gives you a chat assistant that you

25:57 can use to, to talk to and get more information.

26:00 But let me show you how that works.

26:06 I know I have to switch the screen.

26:07 Just give me one second, all right.

26:12 So imagine that I am a director.

26:14 I come into my board portal, I have a meeting on the 20th of November

26:18 and then I can see an AI icon over here.

26:21 I click on this guy and then it tells me,

26:23 it shows me the different insights that will be

26:26 that I should be preparing for my upcoming meeting.

26:28 So for example, strategy, risk, finance.

26:31 So let's go ahead and click on risk.

26:33 It has selected the top 6 insights for Risk

26:35 that I should be focusing on in my next upcoming meeting.

26:39 So let's say I go ahead and choose cybersecurity because why not in Signite.

26:44 So what it's going to do is it's going to give me

26:46 a comparative analysis of what we have talked about in the previous minutes,

26:50 previous meetings and what should we be

26:53 focusing on, on the upcoming meeting, right?

26:55 So for example, I'm not really happy with what it has told me.

26:57 I could say, I could start talking to it and say, could you expand on Queue 3?

27:06 And just like that, it's going to go and look up all the information stored

27:09 in the in the back process that with AI

27:12 and give me what I should be focusing on.

27:14 You can also see that it gives you follow up prompts.

27:17 So based on the conversation you're having,

27:19 it will tell you what other questions you should be asking.

27:22 So you can ask that question as well.

27:25 Then on top of that, if you want to know something else about the meeting,

27:30 for example, let's say who else is attending,

27:33 could you tell me who else is attending?

27:39 So it's going to give me information about who else is attending.

27:41 So if I want to avoid somebody, I will not go to this meeting.

27:46 And then obviously you can switch to to the other insights

27:49 and you can do the exact same process over there as well.

27:52 So let's switch back and see what has happened.

28:00 Now, as I said, our board assistant is much more complicated.

28:03 What's happening here is we are

28:06 just introducing building this, this new feature,

28:08 but we have had customers who've been adding data

28:11 to our platform for the last 8 to 10 years.

28:14 And this is a lot of data, right?

28:16 So if you are, if you have a, a board packet,

28:18 as I mentioned, this could be like 400 to 600 pages.

28:21 And then multiply that by the amount of board meetings that you're having.

28:24 It's a ton of data that needs to be processed.

28:26 So what we did is we built a, the board assistant in two parts.

28:30 1 is your data processing pipeline.

28:32 So as a customer enables AI,

28:34 you're going to go and we we look up this data process that with a pipeline.

28:38 We'll just get to that in a second and then store that into the the vector

28:42 database and also the my sequel for for non non vector information.

28:46 And then the board assistant part is basically

28:48 the interface that lets you interact with the assistant.

28:51 As you just saw in the demo for the data processing pipeline,

28:56 we decided to go with a serverless approach.

29:01 The reason for that is because as I said,

29:03 we are processing an immense amount of data.

29:06 I don't want to worry about CPU, memory limits or anything like that.

29:10 I will get what I need to, to, to, to, to get to process this.

29:15 And I only pay for what I have to pay for.

29:17 And if I'm not using it, I'm not going to pay for it.

29:20 And then I don't want to, to put stress on our existing

29:23 infrastructure by processing all this ton

29:25 of data with the existing micro services.

29:28 It just it's, it's not meant for that kind of workloads.

29:33 And then So what we're doing is we are using a fan in, fan out approach.

29:37 There's an orchestrator that fires up per client.

29:39 I'm not going to bore you with all the activities that happen over here.

29:42 But then for each of the meetings, it processes each of the the, the the steps

29:46 that are in part of your meeting as activity functions.

29:50 Let's take a look at what happens in one activity function.

29:53 So what happens is, let's say I'm processing the metadata,

29:56 so any information about the title,

29:58 where the meeting is happening, any other tags that might be important,

30:01 I take them, I chunk them, I create the embeddings,

30:04 we tag the embeddings so that we can find them later in a in a easier way.

30:09 And then we persist them into the the vectorized database.

30:12 The insights generation that you saw is a preprocessed activity.

30:16 We have a research agent that runs in the background

30:19 that's going to pick up all your documents.

30:21 It's going to run a comparative analysis,

30:23 it's going to do a, it's going to refine

30:25 that and it's it's going to run a comparative analysis again.

30:29 And it does that over many passes until it's

30:31 reached the state where now you can say, OK,

30:34 I have a very good comparative analysis of how a trend

30:37 in my insights has changed over the last three to four meetings.

30:41 And then obviously this is not a vector.

30:43 It's basically text that will persist that into the My Sequel database.

30:48 So you guys saw me interacting with the with the assistant.

30:51 What happens is I write a prompt, it gets converted into an embedding,

30:55 we match that with the existing context

30:57 and then basically it fetches all the existing data.

31:00 The the reason why I put this piece of snippet over here is thisis.net Core.

31:04 This is again to demonstrate how easy it is

31:06 to write a hybrid search query in any platform basically.

31:10 So all this is doing is employing EF Core

31:13 link and simple vector search and now you can

31:16 basically limit this with cosine similarity in any

31:19 of the semantic rankings that you want to do as well.

31:25 So once we found the the matching embeddings,

31:27 we will add the historical context so the AI knows the flow

31:31 of information and the context that the chat was being held in.

31:35 We will then apply the prompt because obviously this is a board assistant.

31:38 We don't want you asking about the weather or anything like that.

31:41 We will then process that with a chat completions model and then we are using

31:46 the Azure service for Signal R to basically

31:48 stream this back to the back to the client.

31:52 This is another view of how the the the chat assistant works.

31:56 You write a query, the orchestrator kicks in, is going to go

31:59 look up all the information that it needs from the vector database,

32:03 combine that with the historical context and then

32:05 pass it to the LLM to process it.

32:07 And then the response is streamed back to the the user.

32:10 If you look at the architecture,

32:12 it does look quite daunting, but it's actually basic Azure services.

32:17 We're using Azure Kubernetes Services.

32:20 This is where we pull all the information

32:22 from the into the into the functions app and then

32:25 the board assistant micro service basically is the one

32:28 that is doing all the the chat work.

32:31 Then we're using Azure Messaging which is basically we're

32:34 using Service bus to drive our event driven architecture.

32:38 Signal R is used to stream data back to the client.

32:41 We're using Postgres SQL and my SQL to persist data as you just saw.

32:46 And then we are using durable functions so

32:49 that we can keep track of what is happening.

32:52 And then if an activity fails, we can re trigger that activity.

32:55 And then obviously Azure,

32:57 the AI foundry is the backbone of our entire AI infrastructure.

33:01 We are using multiple models.

33:03 We're using the 4.1 for chat completions,

33:05 we're using the O3 Mini for reasoning and then

33:08 we're using the the text ADA embedding 002 model.

33:11 And then obviously document intelligence, as I explained earlier,

33:15 is used heavily to process all these these documents.

33:20 In the beginning, when I was talking about enabling,

33:22 AII mentioned some of the challenges that we are facing,

33:26 especially related to determinism and hallucinations and all of those things.

33:30 I do want to talk about how Azure AI Foundry has helped us to solve those.

33:35 If I don't know if you've used the prompt flow,

33:37 but I've found it extremely helpful

33:39 to breakdown complex problems into smaller steps.

33:42 You can write a prompt, you can add context,

33:43 you can write more prompt, you can add more context.

33:45 So it it helps you tackle determinism.

33:47 Now I am by no means saying that it's going to remove determinism,

33:51 non determinism, that's probably never going to happen,

33:54 but it helps you tackle it and helps you engineer

33:56 your, your, your, your systems in a way that can be a bit more predictable.

34:01 Document intelligence,

34:02 as I explained it has really changed the way that we provide context to the AI.

34:07 Our context is much richer.

34:09 It makes gives us much more accurate responses.

34:12 Model deployment helps us to deploy any models

34:14 that we find relevant into our data zone deployments.

34:18 As I mentioned earlier,

34:20 our requirement is that we will deploy the model wherever our customer is based.

34:25 And then obviously we can configure our consumption plans as well.

34:28 Now, as prompt flow has helped us in to tackle complex problems.

34:34 Chat playground helps us tackle less complex problems.

34:38 But if you're looking for, Oh my God, what's the output going to be?

34:41 If I, if I write a small prompt and this is my context,

34:44 you can easily get the response there without writing a whole program

34:48 that communicates with, with, with open AI using the, using the API.

34:51 So it makes your life really, really easy.

34:53 And I use that actually all the time.

34:56 Then with Postgres and PG vector, as I mentioned earlier,

34:59 this was really a game changer for us when it

35:02 came to vector databases because we are using Postgres today already.

35:06 So and Azure supports natively enabling PG vector.

35:10 So we enabled that.

35:11 It really drove our innovation because we could quickly start

35:14 building PO CS and then start building production workloads as well.

35:18 Since it's already part of our infrastructure.

35:20 We have it in high availability,

35:22 cross region replication and everything that we have today.

35:25 We don't have to worry about the, the, the maintainability

35:27 and all of those things that come with again, maintaining your infrastructure.

35:31 PG vector again game changer.

35:33 As I mentioned with with meeting minutes, we were,

35:37 we are storing data in a temporary way.

35:42 So it's, it's all once the the the flow is completed,

35:45 we delete all the embeddings with chat.

35:48 So I mean, in in this case,

35:49 we didn't have to worry about indexing because it's, you know, you,

35:52 you save it once you look it up and you delete it with board assistant,

35:56 it is much more complicated because now

35:58 you're adding millions of embeddings per customer.

36:01 So the H and SW indexing really comes in handy over here.

36:05 And then combined with the the sequel search that we do it,

36:10 it really makes the queries perform much, much better.

36:15 I think you would be thinking, wow, this is cool, but show me the code.

36:18 Unfortunately, I won't be able to show you the code,

36:21 but I will be able to show you the next best thing.

36:25 I'd like to show you a slide from or a dashboard

36:28 from how our AI systems perform in production today.

36:33 So the and and how these help us to basically build confidence in our customers.

36:37 What we are doing is we are using

36:39 deep eval to evaluate our AI responses over many,

36:43 many publicly available government documents.

36:46 So for summarization flows and AI meeting minutes,

36:49 we take publicly available documents,

36:51 we churn them through our systems and we, we,

36:55 we have thousands of different data samples that we then process.

36:59 And I'd like to show you guys

37:01 in terms of different metrics for coherence correctness,

37:03 you can see how well these these systems are performing.

37:08 We check for bias, we check for toxicity and then for answer relevancy.

37:13 You can run it per dock document.

37:14 We can do document cosine similarity score comparison and then obviously you

37:19 can do a a simple cosine similarity score comparison over multiple models.

37:23 So we can switch the model,

37:25 we can run the whole thing again and it it basically will give you the will give

37:32 you the results and then you can compare

37:35 them to other models that that are performing right.

37:39 I mean, I would like to take this opportunity to at the end,

37:43 thank Microsoft very much for inviting us here

37:45 and having us share our experiences with you.

37:48 I think this partnership that we've had has

37:50 really helped to drive our AI innovation forward.

37:54 I've been working with Charles and Bart over here over

37:56 the last one year and they have been extremely helpful.

37:59 So I mean, this was a great experience for me to come

38:03 here and show you guys how our tools we've built our AI tools.

38:06 And if you have any questions,

38:08 I'll obviously be be outside in the in the expert area.

38:12 Would love to hear from you guys.

38:13 And I would say thank you very much

38:14 for your attention and hope to see you guys again.