What happens now that AI is good at math? — the OpenAI Podcast Ep. 17

0:00 Hello, I'm Andrew Mayne, and this is the OpenAI podcast.

0:03 Today, our guests are researchers Sebastian Bubeck and Ernest Ryu,

0:07 and we're going to talk about math,

0:09 how it went from almost laughable to Olympiad level,

0:11 and why you need math to reach AGI.

0:14 The progress of the last few years has been nothing short of miraculous.

0:18 We will be able to have LLMs be able

0:21 to solve problems that require more than 50 pages of thinking.

0:24 Mathematics was just the perfect benchmark to see

0:28 the model making progress during the last four years.

0:34 Sebastian, Ernest, I'd love to know more about you.

0:36 So how would you explain your roles?

0:38 Yeah, sure.

0:39 So I have been working in mathematics for almost 20 years now.

0:44 I used to work in optimization and theory of machine learning.

0:49 I was a professor at Princeton for a few years before moving to Microsoft.

0:53 And now I'm a researcher at OpenAI.

0:56 And in the last few years,

0:58 I've been really trying to understand how AI can help mathematics and to really

1:03 evaluate the progress that we're making in terms

1:06 of solving difficult math problems with AI.

1:08 Ernest, how about you?

1:09 Yeah, so I've recently joined OpenAI as a researcher.

1:13 But before that, I was an applied

1:16 mathematician working on optimization and machine learning theory.

1:20 And in my previous job,

1:21 I worked as a professor of mathematics at the UCLA math department.

1:26 So I think a lot of people have

1:28 this perception that these models aren't good at math,

1:30 literally called language models.

1:32 And how has that changed?

1:34 What's gone on?

1:35 Yeah, I think the progress of the last

1:38 few years has been nothing short of miraculous.

1:42 It's important to remember that two years ago,

1:44 we didn't even have reasoning models,

1:46 let alone models that could prove, you know, difficult mathematical theorems.

1:50 Today, two years later,

1:52 the models are able to help Fields Medalists in their day-to-day work.

1:57 So really the jump is just simply astounding.

2:00 And maybe if I can build a little bit more on that, something which

2:05 is important to understand is that everybody

2:08 has been surprised by this progress, including us.

2:11 So to tell you a story, a year and a half ago,

2:15 I was at a workshop at a conference with other fellow mathematicians,

2:19 and there was a debate that I participated in on whether LLMs,

2:24 scaling LLMs, will help us resolve major open problems.

2:27 So this was a debate, you know, a year and a half ago.

2:30 And the room was very divided.

2:32 In fact, they did a poll at the beginning,

2:34 and I think it was like 80% said no, impossible that this would happen.

2:38 So then the debate unfolded.

2:39 And by the end of the debate, it was more like 50-50.

2:42 So pretty good progress during that hour.

2:45 This obviously was just so wrong in hindsight.

2:47 Like just mere eight months later,

2:49 the model was starting to be able to do research level mathematics.

2:53 What was the breakthrough moment for you realizing that there

2:57 was a really good intersection between AI and mathematics?

3:00 So summer of '25, the big news was ChatGPT was able

3:05 to achieve a top human level performance at the International Math Olympiad,

3:10 a gold medal performance.

3:12 So that was amazing news.

3:13 And that demonstrated that, well,

3:15 at least for the competition level mathematics,

3:17 the models are very highly capable,

3:20 only on par with the top human high school contestants.

3:26 But, well, competition problems are canned problems.

3:29 They have relatively short solutions because they are

3:31 meant to be solved within a few hours.

3:34 And they're not novel because, well,

3:35 somebody came up with it, there's a solution.

3:38 So it's not research-level math.

3:39 So then I got curious,

3:41 and a lot of people got curious, can ChatGPT do research-level mathematics?

3:45 And there was a lot of debate online.

3:48 And then I thought to myself, I should try it on my own problems.

3:51 Maybe I'll try it for myself and make up my own

3:54 mind as opposed to listening to what other people say,

3:57 because a mathematician myself.

4:00 So I took a classical open problem in optimization theory,

4:05 which is a branch of applied mathematics that I work in.

4:10 And the question specifically is there's a famous

4:14 algorithm called the Nesterov accelerated gradient method.

4:16 And does this have this convergent behavior

4:19 or is it possible that in certain bad cases,

4:23 can there be a certain divergent behavior?

4:26 This question was genuinely open in the sense

4:29 that people know that in most cases,

4:32 the algorithm behaves well, it's convergent,

4:34 but people really did not know, is there a bad instance?

4:38 Does it, in the worst case, could it diverge?

4:41 The answer turned out to be yes.

4:43 And the way I discovered it is I remember it distinctly.

4:47 So my bedtime for my son is 8 p.m.

4:51 and then I try not to stay awake after midnight.

4:54 So I had four hours of usually evening hours

4:57 to myself if I want to focus on something.

4:59 So I decided, okay, I'm going to spend a few days working on this.

5:03 So over the course of three days,

5:04 so that's 12 hours total, I interacted with ChatGPT on this question.

5:08 It wasn't as simple as me just putting in the prompt and getting a solution.

5:12 I played the role of the verifier.

5:14 I told whenever the model made a mistake, I corrected it.

5:19 I also tried to point the conversation into areas that I felt,

5:24 approaches that I felt were novel.

5:26 And after a while, the proof, there was a proof and I checked it.

5:31 I also asked ChatGPT to double check it and it was correct.

5:36 And that's how this 42-year-old open problem got resolved.

5:40 And once I got this solution, I thought to myself,

5:46 what would be the most fun thing for me,

5:49 fun way for me to publicize this because I

5:51 could just write a paper and that would be, but that'd be less fun.

5:55 So I decided, let me go to Twitter and talk about this.

6:02 but, well, I had a lot of fun.

6:04 Yeah.

6:04 So it was, I think one of the earliest instances of a genuinely open problem,

6:10 mathematical open problem being solved by AI and yeah, I mean,

6:16 people ate it up and it was a lot of fun.

6:21 It is interesting as you brought that up that we've seen sometimes people said,

6:24 hey, I found something cool or novel.

6:25 And then sometimes it gets torn apart.

6:27 Sometimes it stands up.

6:28 And going into social media can be kind of scary,

6:30 but it sounds like we do need these kind of feedback cycles.

6:33 I think part of the challenge for a lot of us is we hear terms, you know,

6:36 we hear like the International Math Olympiad

6:37 and we're trying to figure out like,

6:39 okay, what does that mean from like a scale of a problem?

6:43 You know, I can understand addition, subtraction, multiplication.

6:46 Could you give me an example of understanding, like,

6:48 where we went from, from, like, you know, first ChatGPT,

6:50 which could kind of sort of use it, then it could do math,

6:53 it could use a tool, but then the model sort of implicitly understanding that.

6:57 When ChatGPT, you know,

6:59 just entered the scene in early '23, I started testing them.

7:05 I was very curious about how the model

7:08 would perform fair on sort of common math problems.

7:12 So these would include math problems that you

7:14 would see in the high school level, but also day-to-day math-ish problems.

7:19 So for example, imagine a scenario where the three of us went camping

7:23 together and then I paid for this, I paid for this, and then Andrew,

7:26 you paid for whatever.

7:28 And then we want to clear the ledger

7:29 and we want to split things evenly at the end.

7:32 Can ChatGPT do the calculations for us?

7:35 And this is moderately complicated if you

7:37 have like 17 items that we've purchased.

7:39 In 23, 24, and also in early 25, I remember, the models couldn't do this.

7:45 Another example would be, I'm in, let's say, in Korea, Seb's in Paris,

7:51 Andrew, you're in California, and want to set up a Zoom meeting.

7:54 What would be a good hour to do so?

7:57 Again, in early 25, the models couldn't do this.

8:01 But then just suddenly things just changed.

8:04 And I wasn't in OpenAI at the time, so I'm not at all,

8:07 I'm not quite privy to what exactly you did,

8:11 but suddenly the models started solving IMO problems.

8:14 And then furthermore, it started solving research problems.

8:17 And the way I sort of calibrate this right now is

8:21 that unless you are a professional

8:24 mathematician trying to discover new mathematics,

8:27 If you are somebody who's, let's say,

8:30 a physicist or a chemist who uses relatively complicated mathematics,

8:34 like differential equations, differential geometry,

8:36 things like this, but you're not inventing new math,

8:40 then ChatGPT can do all of the math that you would need.

8:45 So any basically user of high-level mathematics from STEM can

8:50 now use ChatGPT to basically have their math taken care of.

8:55 You want to exercise some degree of caution, you know,

8:58 check whether things are right, you know, run simulations just to double check.

9:01 The models can make mistakes.

9:03 But now any math problem that you would want to solve,

9:06 most people, for 99% of the population, the models can do it.

9:10 When I worked on the release of GPT-4,

9:11 I used scheduling as one of those examples.

9:13 And I could put three people into a schedule and have it figure out time slots.

9:17 But pushing it beyond that, that was really hard.

9:20 Why was there a change?

9:22 So Ernest just talked about noticing all of a sudden it got better.

9:24 Now we know one thing was tool used.

9:26 You could let the model use a calculator.

9:28 But something else happened with the models themselves.

9:31 So going back to the debate that I just told you about,

9:35 the framing was really about can scaling alone, scaling of LLMs alone bring you

9:41 to solving research breakthroughs in mathematics.

9:45 And this is a wrong framing.

9:47 What we do at OpenAI, we do a lot of research, innovative research.

9:52 It's not just about scaling the model.

9:54 So when you ask what happened,

9:56 or when you're asking what happened middle of last year

9:58 when suddenly the model were able to solve math problems,

10:01 well, a lot of things happen.

10:03 We do a lot of research and all of this has to progress at the same time.

10:06 So I can't really point to a single element.

10:10 But it was able to do it itself, though, without the tools.

10:13 Yeah.

10:14 So I think it's really, really important to just double down

10:18 on what Ernest was saying about the progress

10:21 and the scheduling problems that the model wasn't able to do back then.

10:26 I said that two years ago, we didn't have reasoning models.

10:29 Well, think about four years ago.

10:31 Four years ago, so this is pre-ChatGPT.

10:33 And I remember Google came out

10:35 with a mathematics model called Minerva at the time.

10:38 And I fell from my chair.

10:40 I was so impressed.

10:41 What was I impressed by?

10:43 That the model, I could give it the coordinates of points in the plane,

10:46 and it would give me a line that goes through those points.

10:50 Like when I say that, you know, now it's almost hard to understand.

10:53 What are you talking about?

10:54 Obviously, a model can do that.

10:56 So I think we have kind of forgotten how quickly things have happened.

11:01 And now, yeah, you know, Ernest was saying that it's basically at the point

11:05 where unless you're trying to invent new mathematics,

11:07 it's kind of at the right level already.

11:10 I would say we're already seeing glimmers

11:12 that even to invent new mathematics, it's getting there.

11:16 Could you break down, though,

11:18 aside from somebody who's interested in developing new

11:21 fields of mathematics or just making new proofs,

11:24 what does this affect everything else?

11:26 What is the impact of this going to be on science?

11:29 What is the impact of the rest of what you're working on?

11:33 Why is this really important and not just, oh, cool, it does math?

11:35 So I think the, oh, cool, it does math part,

11:39 what did matter as we were developing those models

11:42 as a good way to benchmark the progress?

11:45 The nice thing about mathematics is

11:47 that the questions are very clear, non-ambiguous.

11:50 You know, everybody agrees on what the question is asking.

11:52 So that's point number one.

11:53 Point number two, you can verify the answer.

11:56 So once a model can give an answer, everybody will agree.

11:58 Was it correct or was it not correct?

12:00 Although you can put a pin on that because we will talk about,

12:03 you know, in research level, it's not that simple anymore to evaluate.

12:07 But before research level, it's very easy to evaluate.

12:09 So mathematics was just the perfect benchmark to see

12:12 the model making progress during the last four years.

12:16 Now, we'd say we have kind of saturated that aspect.

12:20 And you can ask, okay, now, okay, fine.

12:22 The models do mathematics.

12:23 We have understood.

12:25 What about the next steps?

12:27 And for the next step, I would say that having our models be good at mathematics

12:31 is going to be good for many, many other things.

12:33 And let me explain why.

12:35 A key feature of mathematics is that to resolve a problem,

12:37 you have to think for a long time, be it days, weeks, sometimes years.

12:44 So this long thinking, not only do you have to think for a long time,

12:48 but you also have to think consistently for a long time.

12:51 If at some point in your chain of reasoning,

12:53 there is a mistake, this will kill the entire argument.

12:55 It doesn't matter if everything after that is correct.

12:58 If there is one single failure point, the entire argument is destroyed.

13:03 So this property makes it that this is what you want out of reasoning models,

13:07 that if they make mistakes, they will be able to correct themselves.

13:11 So we are hoping that this property that they

13:13 acquire through mathematics will generalize to other domain,

13:17 which by the way, is exactly the same thing with human beings.

13:20 Why do we train human beings in mathematics?

13:22 I mean, it's a very fun topic.

13:23 I love it.

13:24 We did it professionally.

13:25 Maybe we still do some of it a little bit.

13:29 But why do we train humans in mathematics?

13:31 Exactly for the same reason.

13:32 It gives you this kind of very logical thinking.

13:35 Do we need to think about new ways to talk about these discoveries?

13:39 Yeah.

13:39 So I personally view it a little bit as part of my role

13:44 to try to educate the research community about the recent advances.

13:49 because I have this dual background of both being

13:53 a former mathematician and now working on the frontier of AI.

13:56 And indeed, like Twitter and social media is a great place to try

13:59 to explain what is the progress in particular because this progress is so fast.

14:04 So, you know, for example,

14:07 maybe we can talk a little bit about the Erdos problems,

14:09 you know, and some of the controversies that happen around that.

14:14 So there was a first example.

14:16 So there was first, you know,

14:18 Ernest example And there were a few other problems that were solved.

14:20 You just want to explain Paul Erdos though too,

14:22 just so I think people would love to know who

14:23 he is and why his problems are sort of interesting.

14:26 Yeah, of course.

14:26 So Paul Erdos is one of the most prolific mathematicians of the last century.

14:32 He has written, I think, 1,500 research paper.

14:35 He was a very iconoclastic figure.

14:38 You know, he didn't have a house or an apartment.

14:40 He was just traveling from one university to the next,

14:43 trying to find new collaborators.

14:45 And every time he would go to a place and basically ask questions.

14:48 He was very, very, very gifted at asking questions.

14:51 Not all the questions that he asked were interesting.

14:53 Let me just say that right away.

14:55 But still, it was very productive.

14:57 And the research community wrote a lot of papers with him.

15:00 There is even this concept of an Erdos number,

15:03 which is how far away are you in the chain

15:06 of collaborators from having also a paper with Erdos.

15:10 My Erdos number is two.

15:12 I have co-authored a paper with someone who co-authored with Erdos.

15:15 Wow.

15:16 Yeah, I'm pretty happy about that.

15:18 My number's three.

15:20 The joke was, you know, you could be on a train ride with him.

15:23 And then by the end of the train ride,

15:25 you'd maybe work on a paper with him and have your name.

15:27 Absolutely.

15:28 Absolutely.

15:28 I think the two versus three basically says something about our respective age.

15:33 That's actually what it says.

15:34 So anyway, so Erdos has, you know, all of this problem.

15:37 And there is a very nice website by Thomas Bloom,

15:40 who is keeping track of all the Erdos problems that are still open.

15:46 So I think there is like a thousand

15:47 problem or something like that on that website.

15:50 And Thomas himself has done the work of trying to find,

15:53 he's an expert in combinatorics.

15:54 So he can kind of say, okay, this is open.

15:57 This is resolved.

15:59 This has some complicated status for every problem.

16:04 Of course, he doesn't necessarily know the answer to all of them.

16:07 So if there is a paper which is marked open,

16:10 it is not necessarily true that nobody knows how to solve it.

16:13 But it is also a very interactive website

16:16 where people can go on it and, you know,

16:18 add comments to every problem and explain

16:20 whether there is a solution, et cetera.

16:22 So it's a very dynamic, great website.

16:25 So, of course, once we started to have

16:27 GPT be able to solve research math problems,

16:29 this sounded like a treasure trove of problems to try our models on.

16:33 And we tried a couple.

16:36 And to our great surprise,

16:37 the model came back with answers to some of them that were marked as open.

16:42 So we got really excited about this.

16:44 The first one, you know, that I tweeted about, I don't remember when it was,

16:49 maybe it was in October or something like that last year.

16:53 It was a deep literature search result.

16:55 So let me explain what that means.

16:57 It means that what GPT did is that it did a vast literature search,

17:02 trying to scan, you know, thousands of papers.

17:04 And it found in some unrelated field, the answer to the question.

17:09 Now, it's really important to understand that it's not like in that, you know,

17:13 unrelated field, the person said, okay, I'm solving an Erdos problem.

17:16 It was written in a completely different language.

17:18 It was different mathematics.

17:19 You have to do work to connect the two pieces and GPT did that.

17:23 So that was kind of amazing.

17:25 And this was very ad hoc.

17:27 Like, you know, we just tried by hand basically in the ChatGPT interface.

17:31 Once we saw that, Mark Selke, who is in our team also,

17:36 decided to have a more systematic approach of trying all of the problems.

17:41 And he tried that, and the model came back with solutions to 10 Erdos problems.

17:46 And this was, you have to remember, at that point, there was still, I think,

17:49 a very dynamic discussion about whether those models could

17:51 go beyond the state of the art and discover, invent new mathematics.

17:56 So I got very excited about this result and I tweeted about it.

18:01 And, you know, it's kind of an infamous

18:04 tweet because people misunderstood it as kind

18:07 of saying it really found the solution to 10 open problems that are very hard.

18:12 And the solution is completely new and did not exist in the literature.

18:15 But that's not what happened.

18:16 It was connected, of course,

18:17 to the previous case where it is a deep literature search.

18:20 So there was some, you know, feud with Google about, you know,

18:24 with Demis about whether, you know,

18:26 this is the right way to talk about such results.

18:30 But now the punchline is kind of amazing, which is a few months later.

18:35 So again, I said 10 solutions to open

18:38 problems and these were solutions in the literature.

18:40 And then the question is, can you find solutions that are not in the literature?

18:44 By now, we have more than 10 actual solutions that are completely new,

18:49 that are publishable in top journals in combinatorics,

18:52 completely obtained by, you know,

18:54 some by ChatGPT and some by our internal models.

18:58 So just within, again, this really speaks to the acceleration.

19:03 In the span of just a few months,

19:05 we went to, it's kind of a ridiculous statement

19:07 to say that there would be 10 solutions to Erdos problems.

19:11 it's actually happening for real and it's accelerating.

19:15 Yeah, it's interesting because it seems like step one is

19:17 have models be able to do really good literature research.

19:20 And there have been major papers and awards done,

19:23 given to people who've just done literature searches and found

19:26 the solution was solved here and it actually applies elsewhere.

19:29 So it's neat that it does that as the first step,

19:32 but now that it's actually doing original.

19:33 I mean, you know, one thing that I really like about AI research

19:37 is that it forces us to confront big questions about intelligence and about,

19:41 you know, research and progress and how do we discover new things.

19:46 In particular, there is this question of whether

19:49 the progress that we're seeing in science,

19:51 is it just putting together different pieces and, you know,

19:55 doing a little bit of reasoning on top of it?

19:58 Or are there those brilliant, you know, sparks of insight?

20:00 Everybody, of course, points to Einstein's, you know, relativity.

20:04 I'm not even sure that really counts, to be honest.

20:07 So I think the jury is still out on whether

20:10 this process of just recombination and a little bit of thinking,

20:14 whether you can kind of increase, you know, human knowledge with no limit,

20:18 or do you really need the sparks of genius that would be somehow only human?

20:23 Well, even he credited, I forgot who it was, but who came up with the, you know,

20:26 the analogy, the visualization method, you know,

20:27 he said it wasn't his, we pointed out who did it,

20:29 and he kind of took it to the next step further, obviously.

20:31 And I think that we sometimes,

20:33 we love these tiny little stories when it's a lot more complex than that.

20:37 Yeah, absolutely.

20:38 What will it mean for scientists in general

20:40 if we have better mathematical tools in AI?

20:43 How does it affect other things, biology, material science?

20:46 Yeah, so again, how it affects the rest of science?

20:49 Well, the point is, I think it's really important for everybody to understand.

20:54 It's not like we're doing something very, very special for mathematics.

20:57 Our techniques, our training techniques are very general.

21:00 They are applied to everything.

21:01 So our expectation is that we are seeing more progress Well,

21:06 one reason is because it's very easy to benchmark.

21:09 It's very easy to see that progress.

21:12 But we have full expectation that this is going to happen in all sciences.

21:16 It's not going to be limited to mathematics.

21:18 Yeah, it seems like something that's very good at going,

21:20 if this is true, and then this is true,

21:21 and going through a long sequence of those kinds

21:24 of statements has a lot of applications elsewhere.

21:27 We've heard the term auto-researcher.

21:29 Do you want to unpack that a bit?

21:31 Right now, the way we work is exactly what Ernest described.

21:35 which is really an interaction.

21:36 It's kind of a professor-student interaction where ChatGPT is

21:40 a student and the professor is kind of, you know,

21:42 giving a first problem and the student comes

21:45 back and then they talk a little bit.

21:47 The student goes away for another week, comes back.

21:50 One point, of course, is that it's compressing those timelines greatly.

21:54 In the story, you know, of solving this problem in 12 hours.

21:58 I mean, I don't know, without ChatGPT, how long would it have taken you?

22:01 Well, I have spent more than 40 hours failing without AI.

22:04 And I don't know, maybe a month.

22:08 Right.

22:09 So exactly.

22:09 So, you know, there is this thing of just compressing timelines.

22:13 Now, when we talk about the automated researcher,

22:16 that's a slightly different vision where the model or maybe a collection

22:20 of model would work autonomously for a long period of time.

22:25 This is kind of needed if we want to go beyond the current level.

22:29 The current level of interaction,

22:30 the professor-student interaction where the student comes back after a week,

22:33 it's going to be very hard

22:35 with that mode of interaction to do real breakthroughs,

22:38 to solve actually longstanding research problems or to make

22:43 progress in very difficult fields in biology

22:46 where you need to interact with the wet lab and do all kinds of experiments.

22:49 So once you want to go towards the real breakthrough,

22:53 we will need to work over longer timelines.

22:56 And this is where the automated researcher comes in.

22:59 Maybe let me say it in a slightly different way.

23:01 One concept that I'm a big fan of is this concept of AGI time.

23:06 So you can have AGI seconds, minutes, hours, days, and so on.

23:10 So that really means you have an AI

23:12 and it can mimic human thinking, but for how long?

23:16 So as Ernest was saying, two years ago,

23:19 maybe models were mimicking a high school student

23:22 who thinks for a few minutes on a problem.

23:24 Now we can mimic a researcher who can think for hours, maybe a few days.

23:29 We really want to go towards, and this progress has been going on for now,

23:33 very consistently for four years,

23:35 where we went literally from seconds to minutes to hours to days.

23:40 And now we are roughly at days slash one week.

23:43 We want to go to weeks, if not months.

23:47 This is open research.

23:49 I don't think anyone on the planet knows exactly how to do it.

23:54 But this goes back to, we are doing a lot of research, a lot of innovation.

23:58 And I think once everything will be put together,

24:00 we're just seeing this arc of progress

24:02 where we keep making progress in AGI time.

24:04 But this is the direction of the automated researcher.

24:07 So the people, the other mathematicians that I talk to, their mode of using AI

24:12 is they open up ChatGPT and then

24:14 they talk to ChatGPT within that context window.

24:17 And you can have multiple sessions,

24:20 but each session has a finite context length.

24:23 And roughly on the order of like 50 pages of a math paper.

24:28 And that's not long enough to make true deep math,

24:34 groundbreaking math breakthroughs,

24:36 because a lot of math papers are longer than 50 pages.

24:39 And also, the human thought that went into to produce, let's say,

24:43 a 10 or 30 page paper is usually, well,

24:45 much orders of magnitude longer than the final output.

24:49 So there's a limitation with the limited context window.

24:54 But people who've use Codex will know that you

24:58 can actually have very long work sessions with Codex.

25:01 So you just keep giving instructions as to what kind of code you want to write.

25:05 And then the code itself that you're working on, the repository of your code,

25:09 which in the math sense, the analogy would be that would be analogous

25:13 to like math notes that you write down.

25:15 That can be very, very, very long.

25:18 But Codex is pretty good at dealing with that.

25:21 Once in a while, it compactifies its conversations

25:25 And it has its way of becoming this really amazing

25:29 agent that can do really complex jobs over huge

25:33 repositories of code over a really long context of conversation.

25:40 And this, I believe, is going to happen with mathematics research as well.

25:45 So we will be able to have LLMs be

25:48 able to solve problems that are longer than just,

25:50 you know, that require more than 50 pages of thinking.

25:53 And that's what human mathematicians do.

25:55 People think for a day on a certain problem,

25:58 and then we kind of summarize our ideas and then put it into notes.

26:03 The next day or the next week, we come back to it.

26:07 And then over several months, we've thought for so long,

26:10 but it's sort of summarized,

26:12 it's sort of organized in a way that becomes manageable.

26:15 And in the end, the final output becomes

26:18 a 30-page paper summarizing the thoughts over many, many months or even years.

26:23 so yeah i think that's gonna happen i was working on a very very

26:27 laughable problem to you guys over the weekend and using an LLM to try

26:31 to do it to figure out like how to use a really small LLM

26:34 to do math in the middle of it i needed a benchmark and i

26:37 came across easy math which is a benchmark for small LLMs and problems just

26:40 a paper on it there wasn't really like a lot of data and I

26:43 just in the middle of codex ago can you create my own benchmark here

26:47 and just generate the data for that And five minutes later, I had it.

26:52 And that was magical to me because I'm in the middle of working

26:55 on the tool that would have involved me all of a sudden,

26:58 okay, I got to spend a few hours,

26:59 go do a generator, go produce this sort of stuff.

27:02 Absolutely.

27:02 And it runs in the background.

27:03 I can't imagine what it's like for you guys doing grown-up problems.

27:07 Yeah.

27:07 I mean, what you describe is really, you know,

27:11 what we went after when we published the paper

27:14 whose title was early experiments in science acceleration with GPT-5.

27:19 Like we, what you have experienced is literal acceleration.

27:23 Like this is something that would have taken you before,

27:25 I don't know, maybe a few days of work or something.

27:27 I would have given up.

27:28 Yeah.

27:29 Yeah.

27:29 So that's actually a great point.

27:32 You know, I would have given up.

27:33 This really enables scientists everywhere.

27:37 Like for example, mathematicians to be able to use code.

27:40 Most of our friends, they don't code, you know?

27:42 And now suddenly they have codex.

27:44 They can do all the experiments that, you know,

27:46 before they were trying to find the poor

27:48 grad student do the experiment for them.

27:50 Now they can do all of these experiments very easily.

27:53 The flip side is of course, like that scientists in all the disciplines,

27:57 they can also use more advanced mathematics now, thanks to ChatGPT.

28:01 I sat down with Bob Metcalf and showed him how to use Codex to do R because

28:04 he was working on a project and R was new to him and he wanted to learn that.

28:08 And that was kind of a fun experience

28:10 to take somebody who's got a great mind and say,

28:12 oh, here's, instead of spending a lot of time having to figure this out,

28:15 there's the tool for you.

28:17 But of course, now, as you alluded to before,

28:19 we should talk about the role of the human in all of this.

28:23 What is the place for the human?

28:25 Especially if we start to think about, you know,

28:27 let's think a little bit about the future.

28:29 I'm not a big fan of trying to predict the future.

28:31 I like to explain what other...

28:32 But what do you think will happen?

28:36 I think, you know, there is what my heart

28:40 tells me and there is the rational aspect.

28:42 So what my head tells me is, look,

28:46 the progress has been happening very consistently for the last four years.

28:52 From being able to solve math problems that would take you seconds,

28:55 to minutes, to hours, to days.

28:57 There is no reason.

28:59 Anybody would look at the situation would say, okay,

29:01 a year from now, you will have systems that can think for weeks.

29:05 Two years from now, systems that can think for years.

29:09 Not only that, but already today, we're finding that our models,

29:13 they are able to really surpass humans

29:16 in the sense that they can find mistakes in papers.

29:19 You know, we had system, we had agents internally that have been able

29:23 to come up with, to find papers and say, hey, actually this is wrong.

29:27 Here is the correct answer.

29:29 Not only that, but people tend to think

29:31 that AI is only good at answering questions.

29:35 Actually, no, it's also pretty good at asking questions.

29:37 Of course, you need to be, you know, again,

29:39 you need some research innovation there, which we had.

29:43 And now our models are very good at asking questions.

29:46 So good, in fact, that humans are looking at those questions and saying,

29:49 hey, maybe I should write a paper based on this question.

29:52 So this is, you know, really, really already happening now.

29:55 So I think what I'm trying to say is that in a year,

30:00 in two years, yes, models could do basic,

30:02 more or less everything that human researchers do.

30:06 So now what?

30:07 What is the role of humans?

30:09 Well, why is it that we're doing science?

30:11 What's the point?

30:12 You know, the point is not to, I mean,

30:15 it shouldn't be to just solve problems for the fun of solving problems.

30:20 We're solving problems because we're trying to understand something.

30:23 The understanding piece is key.

30:26 We're not solving problems to write papers, to show,

30:28 to say that we can write, you know, 10 times more papers than our neighbor.

30:33 That's not the point.

30:34 You can do competitive chess if that's your kind of deal.

30:38 We're trying to really understand deeper things.

30:42 And why are we trying to understand deeper things?

30:44 Because we want to have better control over our environment.

30:49 We want to be able to cure diseases.

30:51 We want to be able to build things better,

30:54 faster, more robust, more solid, all of those things.

30:57 So I think there is a chance that we're looking at a very,

31:02 very bright future using those tools as long as the human

31:06 stays in control and guides what are the problems that matters.

31:10 Problems that, you know, the AI doesn't care about curing disease.

31:13 I mean, you know, they will not suffer

31:15 from the same disease as we do, but we do care.

31:18 So we have to control them and to guide them towards those problems.

31:21 At the time of the advent of the first computers,

31:23 when the computer went from being a person that did

31:26 the math to an actual machine that did it,

31:28 you saw some people looking at maybe we all have to move

31:31 from math to physics because that's where the hard problems are going to be.

31:34 And there's not going to be any more

31:36 hard problems in mathematics because computers resolve that.

31:39 Now it was the 1940s and 1950s.

31:41 And it turned out that that's not the case,

31:43 that computation opened up a whole new branch of that.

31:46 And that's what's going to continue, that we're just going to see

31:49 that the mathematician that's in high school today is

31:51 going to have a very exciting future 30

31:53 years from now because of what's happening here.

31:56 I think math is going to be so much fun.

32:00 So, okay, so math is, so mathematicians enjoy solving problems.

32:06 But, you know, pre-AI, you know, we would think for months to solve a problem.

32:11 And that's, there's enjoyment in that, but there's, it's quite grueling.

32:15 That is pain.

32:15 There is pain too.

32:16 There is a lot of pain.

32:18 And there is a huge,

32:19 like there is a surge of dopamine when you actually find the solution.

32:24 That's going to be accelerated.

32:25 So, you know, more solutions, more fun.

32:27 But also, I think math is going to become much

32:30 more richer because it's going to be much more interconnected.

32:35 Because there is a lot of, at research level, a lot of math is hyper niche.

32:39 And when you write the paper, you know that there are only five living

32:43 humans right now that will care about this paper.

32:46 But you like the results, so you put it out,

32:49 and then the five other people appreciate it, so they read it.

32:52 But then 20 years later,

32:53 it's going to be in the archive somewhere, and nobody will read it.

32:58 But now that we have AI, the AI will have read it.

33:01 And if there is a useful connection, as Sebastian mentioned, it will surface it.

33:08 And then people 100 years down the line will discover

33:11 it and use it for whatever they want to use.

33:14 So I would now have much more confidence that my results that are just

33:19 put out there will be used if there is a use in the future.

33:24 And also, I'm now able to access the mathematics in a much broader way.

33:28 There are fields that I've not studied, but if a result comes up,

33:31 then I would still have to study that field

33:33 to be able to use that particular result in my research.

33:36 But there is no way I could have found that result without the assistance of AI.

33:41 But now it's accessible.

33:43 The model tells me, hey, you can use this to solve your problem.

33:45 And then, well, OK, I'll go and try to use that.

33:49 So math is going to be a much more interconnected enterprise.

33:53 And also, verifying correctness of mathematics is actually quite non-trivial.

33:58 Because imagine there's a proof written by somebody that's 300 pages long.

34:06 And it claims to solve a really important problem.

34:10 And this person is a very reputable person.

34:12 So like there's there and the paper at surface looks plausible.

34:18 How do you know?

34:18 Well, I mean, this is a process that takes years to verify.

34:23 And it's also not enough that one person reads it.

34:26 Many people need to read it and then try

34:28 to extend it and then look into the details.

34:31 This is a process that takes years.

34:33 And sometimes, like, fatally incorrect proofs are published.

34:38 So that's also a very slow process where the field initially accepts a result,

34:42 but later on discovers that it's unsalvageable.

34:45 So then it needs to get filtered out.

34:48 This is going to be so much more accelerated with AI.

34:51 So right now, our chat GPT and our AI models

34:55 are not perfect at verifying mathematics, but it's very good.

34:59 And also, it has much more patience than humans.

35:02 Yes.

35:02 So the truth is so much of the published mathematics have

35:05 minor mistakes and a lot of them do have major mistakes.

35:09 And we know because we have tested these things with our models.

35:13 But now I think the more richer future

35:16 of mathematics is that this will be through AI verification.

35:20 We will have much more certainty as to which results are correct,

35:25 which results are incorrect.

35:26 And we'll have a much faster feedback on this.

35:29 paper published, put out a week ago, we could get a verification on that.

35:33 And then we could trust and build on that as opposed

35:36 to waiting for five years to really ascertain the correctness.

35:39 So overall, math is going to be much more fun.

35:42 It's going to be much more interconnected.

35:48 We'll be able to trust the results more.

35:49 We'll be able to move faster.

35:50 And the mathematicians will solve harder and more interesting problems.

35:52 So maybe one thing that I want to add,

35:55 so I totally agree with everything that you just said.

35:59 It's going to be a lot of fun,

36:01 but I want also to look about one potential danger of the current progress,

36:06 which would be that we kind of hand the keys to the castle,

36:10 to the AIs, and that humans just start to trust the system a lot more,

36:16 and that they don't do the hard work that we kind of did to own our skills

36:20 and to own our skills to to be able to verify and to sit patiently for hours,

36:25 many days in a row or many weeks in a row to try to understand deeply a result.

36:31 And instead just kind of ask ChatGPT to explain it to us in simpler terms.

36:36 So basically I'm worried about potentially having a shallower understanding

36:40 of things because we rely too much on the tool.

36:43 So I think it's really important for the audience,

36:46 for everyone listening to us to understand that expertise

36:50 is even more valuable than it ever was.

36:54 The reason why we are able to squeeze out those results from ChatGPT is

36:58 because of all of those years of training

37:00 and our deep understanding of the subject.

37:02 If it wasn't for that, we would not be able to push the state of the art.

37:06 And we're seeing it.

37:07 It's not like we're seeing thousands of people,

37:09 like non-mathematicians suddenly being able to prove new results.

37:12 In fact, if anything,

37:14 We have seen recent examples in social media where non-mathematicians have tried

37:18 to use those tools to prove theorem and come up with, you know,

37:22 many tens of pages of proof.

37:24 And then it turns out to be just wrong.

37:26 So this is a danger that we have to grapple with.

37:29 It seems like that's going to be a problem in a lot of things.

37:33 You see people spend, you know,

37:35 using current models that often just reinforce things you want to hear.

37:38 And that can be kind of, you know,

37:40 I'm going to come up with some sort of unified theory or whatever.

37:42 Like, well, guess what?

37:43 That's going to be a lot harder.

37:44 Yeah, I mean, this sort of issue of mental sort of atrophy,

37:47 if you will, is also, I think, very prominent in coding as well.

37:51 So, I mean, I'm not a, you know, I wasn't a computer science major,

37:55 but I took some computer science courses and I did, I coded myself.

37:58 I wrestled with the debugger and most people of my age did.

38:02 But nowadays, you don't have to do that in your university curriculum.

38:05 And I think that's very dangerous.

38:07 I've heard some people in the sciences

38:09 who look at the progress are very optimistic,

38:10 like, well, we're not going to need scientists.

38:12 We're not going to need this anymore.

38:13 No.

38:14 Yeah, no.

38:14 Wow.

38:15 This is terrible.

38:16 So really, I want to make sure anybody listening, please do not say that.

38:20 This is the opposite of what we need.

38:22 We need more scientists than ever.

38:24 Those scientists are going to be more productive, more powerful.

38:27 They will do better things.

38:29 But we need them to be really, really good at their craft.

38:32 And I think this is where, you know, obviously,

38:35 OpenAI cannot do everything, you know, just to say it out loud.

38:39 And this is where the existing institutions have a very big role to play.

38:43 So academia needs to both understand the rate of progress and, you know,

38:47 how fast this is going, but also to kind of reclaim their role in that process.

38:53 Yeah, my hope and expectations,

38:55 we're going to see more people go into the sciences

38:57 because if you decide later on in life that you want

39:00 to get into this, it's easier to catch up if

39:02 you're dedicated because you have the greatest tutor in the world.

39:05 OpenAI just added to the ChatGPT has a visual

39:08 explanation tool now that helps you explain things.

39:11 And I think that people, you know,

39:13 just because all of a sudden an AI model is able to, you know,

39:17 completely top out, you know, a benchmark doesn't mean you go, OK, we're done.

39:21 We solved grade school math.

39:23 Congratulations, everybody.

39:25 AI is done.

39:26 It's like, no, there's a next level

39:27 and the next level and you're going to need people.

39:30 No, I think it will help.

39:31 I mean, the young generation to get up

39:33 to speed in science like so much more quickly.

39:36 That's for sure.

39:36 Like I cannot imagine if I had ChatGPT, you know, as a teenager.

39:40 I mean, I remember looking at Maxwell equation and being like,

39:43 what does it really mean?

39:45 How did they come up with this stuff?

39:47 Now you can just ask it and it will explain it to you so beautifully.

39:51 It's a big deal, but you still need to do the hard work on top of it.

39:56 Though with a lot more people trying

39:57 to create mathematical proofs who don't know what

39:59 they're doing and aren't really maybe putting

40:00 the right scholarship to make sure of that.

40:03 We've seen areas of code repos and whatnot and people

40:06 contributing fixes that aren't real fixes and things like this.

40:09 How do you solve for that?

40:10 If I'm somebody who's involved in mathematics or a journal right now,

40:13 I'm a little bit terrified.

40:15 Yeah.

40:15 So I think what Ernest said is that, you know, AI can help also for that.

40:20 So we can have on the other side of it, of those systems,

40:23 to have AI agents that are also going over everything,

40:26 trying to verify as much as possible.

40:28 And then, again, we do not want to trust fully the AI

40:32 to verify and to accept a paper or to accept a comment.

40:36 But we can have the AI agent flagging specific potential issues.

40:41 So kind of bringing to the front, okay, hey,

40:43 maybe this part, I'm not totally sure about it.

40:46 So that will accelerate,

40:48 that will help the human to have less to verify, basically.

40:51 And I think the sort of social structure of mathematics or code,

40:55 it has to change a little bit in a way that the human doing the commit

41:00 or human controlling the agent takes responsibility.

41:03 So in mathematics, there already is a culture of, well,

41:07 if you put out an incorrect proof, then, well, that hurts your reputation.

41:11 And you're putting your reputation on the line

41:13 when you put out a paper with your name.

41:16 And that has to, I think we need more of that.

41:19 If you're mathematically curious and somebody is watching this or listening,

41:22 and they maybe have an interest in math,

41:24 but maybe they didn't feel they were a math person,

41:26 but they're kind of curious to get started, what would you tell them?

41:29 Go chat with ChatGPT.

41:32 If you are interested in learning, then it's so helpful.

41:36 Even at the research level, when I need to learn a new concept,

41:39 I would habitually go to Wikipedia, and then it's just very dense.

41:43 I'm like, okay, well, after like 30 seconds, I go, okay, let me ask ChatGPT.

41:47 And then I ask it.

41:48 And then I also ask follow-up questions.

41:50 And when I do so, it gives me so much more helpful information that is tailored

41:56 to the parts of my knowledge that is

41:58 missing because I'm asking the questions tailored towards that.

42:01 And you could imagine explaining to ChatGPT your mathematical background,

42:07 the books that you've read, the material that you've learned,

42:11 and then ask you to come up with a question that would

42:16 be open and also would be understandable with your level of expertise.

42:21 Sebastian mentioned this.

42:22 I don't think people yet appreciate that these LLMs

42:26 are able to come up with good questions, but I think they can.

42:32 So having this companion that you can

42:35 talk about math with and talk about questions.

42:39 You could ask the model to help you solve it.

42:41 And once you have a solution, then you could keep talking and come

42:45 up with the next question, variations of this.

42:48 It becomes a much more, even though you're still in your room alone,

42:53 it feels much less of a solitary process.

42:56 And that's what really makes mathematics fun.

42:59 Because math, I think it really is a social endeavor.

43:04 I think toy problems would be fun.

43:05 And I tell people, you can start with how many M&Ms can you fit in your bathtub?

43:10 It sounds silly.

43:11 And you start to ask, then you go, how many words did you read last year?

43:14 How would you figure this out?

43:15 And then you can start to have

43:16 this real wonderful conversation and start asking these questions.

43:19 Next thing you know, you're starting to do more and more complex

43:21 mathematics and realize how it should affect you.

43:24 Gentlemen, this is great.

43:25 Sebastian, Ernest, thank you very much.

43:27 Thank you.

43:27 Thank you for having us.