Bret Taylor: A Vision for AI’s Next Frontier

0:00 Technology companies aren't entitled to their future success. AI, I think will change the landscape of software and I think it will help some companies and it will really hurt others. And so When I think about the what it means to build a company that's enduring, that is a really, really tall task in my mind right now because it means not only making something that's financially enduring over the next 10 years, but setting up a culture where a company can actually evolve to meet the changing demands of society and technology. when it's changing at a pace that is like unprecedented in history. So I think it's one of the most fun business challenges of all time. I just get so much energy because it's incredibly hard and it's harder now than it's ever been.

0:45 to do something that lasts beyond you. But that I think is the ultimate measure of a company. Welcome to the Knowledge Project podcast. I'm your host, Shane Parrish. In a world where knowledge is power, this podcast is your toolkit for mastering the best what other people have already figured out. If you want to take your learning to the next level, consider joining our membership program at fs.blog slash membership. As a member, you'll get my personal reflections at the end of every episode, early access to episodes, no ads including this, exclusive content, hand edited transcripts, and so much more. Check out the link in the show notes for more.

1:32 Six months after Brett Taylor realized AI was about to change everything, he walked away from his co CEO job at Salesforce. To start from scratch. That's how massive this shift really is. The mastermind behind Google Maps and the former chief technology officer at Facebook, Brat reveals the brutal truths about leadership, AI, and what it really takes to build something that endures long after you've reached the top. Brett's led some of the most influential companies in tech and seen exactly what makes businesses scale, what kills them from within. And why most founders don't survive their own success. In this conversation, you'll discover why so many companies are already on life support without realizing it, how first principles thinking separates the next wave of winners from everyone else.

2:19 And the hidden reason most acquisitions fail. We'll explore why AI is bigger than anyone suspects, plus the mindset shift that turns great engineers into exceptional CEOs. Whether you're a founder, an operator, or simply someone who wants to think sharper, this episode will change how you see your business, technology, and the future. It's time. To listen. And learn.

2:46 What was your first real aha moment with AI where you realized Holy shit, this is gonna be huge. I had two separate aha moments. Uh one that I don't think I really appreciated how huge it would be, but it kind of reset my expectation, which was the launch of Dali. Uh in the summer of

3:06 Twenty two, is that right? I might be off by year, but I think summer twenty two. And Uh the avocado chair. that they generated and I had been

3:17 Well, I my background is in computer science and and pretty technically deep. I hadn't been paying attention to Large language models, um uh I just d didn't follow the progress uh after the Transformers paper and I saw that and my reaction was I had no idea computers could do that. And

3:37 That particular launch, um You know, seeing a generated image of an avocado chair. I don't think I extrapolated to a you know, uh where we are now. But it uh for me shook me and realized I'd need to pay more attention to this space and open AI specifically

3:54 than I had been. Um I think I had that was the moment where I realized like I clearly have been not paying attention to something significant. And then it was you know, six months later, coincidentally, like the month after I I Left Salesforce, uh chat GBD came out and Well before it became a phenomenon that it did so quickly, but I was already, you know, plugged into it and and uh

4:16 And I was uh From then on, you know, I could not stop thinking about it. Um but that avocado chair. I don't know why. I think it was the There was a bit of an emotional moment where you saw a computer doing something That wasn't just rule based, but uh

4:32 creative and um the idea of a computer doing something creating something from scratch. Uh was uh well, it doesn't seem so novel, uh, you know, a few years later just blew my mind at the time. One of the unique things about you is that you've started companies, you've been acquired by Facebook and Salesforce. Inside those companies you rose up to be the CTO at Facebook, the co CEO at Salesforce. Talk to me about founders working for founders and founders working within a company.

5:04 Yeah, it's uh It's a very um Challenging transition for a lot of founders to make. I I think There's lots of examples of acquisitions that have been really transformative from a business standpoint. Uh I think YouTube, Instagram

5:20 Being two of the more prominent that have clearly changed the shape of of the acquiring company. But even in those cases, this you know, the founders didn't stay around that long. Uh, and those guys that's maybe a little unfair. They stick around for a little bit. I think the Interesting thing about about uh being a founder is it's not just

5:39 building a business, but it's very much your identity. Um, and I think it's very hard for people aren't founders to experience it. If you take everything very personally, you know From the product to the customers to the press to uh your competitors, uh the the both in inner and outer measures of of success. And I think when you go to be an acquired, there's a business aspect to it and you know can you operate within a larger company, but that's intertwined with a sense of identity. You go from being a

6:07 the founder of a company and the CEO of a company or CTO of a company, whatever your your title happens to be as one of the co-founders. to be in a part of a larger organization and to fully embrace that you actually need to change your identity. Um you need to go from being you know, the head of Instagram or my case the head of Quip to being a an employee of Salesforce or going from being the CO of Friend Feed to be an employee of Facebook.

6:35 And What I've observed is it's that identity shift is a prerequisite for most of the other things. It's not simply your ability to handle the politics and bureaucracy of a bigger company or to navigate a new structure. I actually think most founders don't make that leap where they actually identify uh with that new thing. It's even harder for some of the employees too, because most of the time in an acquisition an employee of an acquired company didn't choose that path. And in fact they chose to work for a different company and they You know, the the acquis determined a different outcome.

7:08 And that's why integrating acquisitions is so nuanced. And I would say that uh having the experience of having been acquired uh, you know, before And having acquired some companies before when I got to Salesforce, I really tried to be self aware about that and really tried to, you know, be a part of Salesforce, you know, and tried to shift my identity and and not be a single issue voter around Quip, you know, I'd really try to embrace it.

7:33 Um And uh and I think it's really hard for some founders to do. And some founders don't want to, honestly. You know, they uh maybe cash the check and and you know, that's the it's more of a transactional relationship. I um I I really actually I'm so grateful for the experience hav having been at Facebook and Salesforce, I learned so much. So but it really took a lot of effort on my part to just um Transform my perception of myself and who I am to get that value out of the company that acquired us.

8:00 How did you how did it change how you did acquisitions at Salesforce? You guys did a lot of acquisitions while you were there. And you're acquiring founders and and sort of startups and I think Slock was while you were there too. How did that change how you went about integrating that company into the Salesforce culture? I'll talk abstract about talk about some specific acquisitions too, but First I I think I

8:23 tried to approach it with more empathy, um and more realism, you know. Uh One of the nuanced parts about acquisition is There's the period of um doing the acquisition. There's sort of the period uh after you've decided to do it of doing due diligence, and then there's a period when it's done and you're integrating the company and sort of the period after. One of the things that I have observed is that uh companies doing acquisitions, often the part of deciding to do it. Is a bit of a mutual sales process. Um

8:55 uh you're trying to find a f fair value for the company and and there's some Back and forth there. But in the end of the day, there's usually some objective measure of that. Um influenced by a lot of factors, but but there's some fair value of that. But what you're trying to do is what are

9:12 Uh and uh corporate speak would be synergies, but like why do this? Why is one plus one greater than two? You know, that's that's why you do an acquisition just from first principles. It's often a exercise in storytelling, you know, uh You, you know, bring this product together with our product and customers will, you know, find the whole greater than the sum of its parts. This team applied to our sales channel or if you're a Google acquisition, you know, imagine the traffic we can drive to to this product experience.

9:42 Uh you know, in uh the case of something like an Instagram, imagine our add sales team attached to your amazing product and how quickly we can help you realize that value, whatever it might be. I find that People because there's sort of a craft of storytelling to uh for both sides to come to the same conclusion that they should

10:03 Uh do this acquisition. sometimes uh either simplifies or sugarcoats like some of the realities of it. Um You know, uh little things like, you know, how much control will the founding team of the acquired company have over those decisions? Um Uh will it be operated as a standalone business unit, or will your team be sort of broken up into functional groups within the larger company?

10:29 And it's sort of those little they're not little, but those I'll say boring but important things that often people don't talk enough about and You don't need to figure out every part of an acquisition to make it successful, but often you can end up running into like true third rails that you didn't find because you were having the storytelling discussions. Rather than

10:48 getting down to brass tacks about how things work and what's important. The other thing that I think is really important is being really clear what success looks like. Um and You know, I think uh Sometimes it's a business outcome, sometimes it's a product goal. But I found that um

11:05 If you s went to most of the like larger acquisitions uh in the valley And you Two weeks after it was closed, s interviewed the management team of the acquiring company and the acquired company, and you ask them like what does success look like two years from now? My guess is like eighty percent of the time you get different answers. Um

11:26 And I think uh it goes back to this sort of storytelling thing where you're talking about the benefits of the acquisition about talking about like what does success look like. So I really try to approach it, I try to um pull forward some harder conversations when when we're you know, when I'm doing acquisitions or even when I'm being acquired since it's happened to me now twice. So that you know, when you're approaching it, you not only get the Hey, why is one plus one equal greater than two? Everything's gonna be awesome. You know, but no, for real, like what you know, what does success look like here

11:56 And then, you know, as a founder, your job of an acquiring acquired company is to Tell your team that and align your team to that. And I think founders don't take on enough accountability towards making these acquisitions successful as I think they should. And um and it goes back to again a certain Uh Naivet, you know, it's like you're you're you're not your company anymore. You're a part of something larger. And I think, you know, successful ones work when everyone embraces um embraces that.

12:23 What point in the acquisition process is that conversation? Is that after we've signed our our binding, you know, sort of commitment, or is it we should have that conversation before so I know what I'm walking into. My personal take is it's not something you have you have to get to the point where The two parties wanna merge, you know, and that's uh Obviously a financial decision, particularly if it's like a public company, there's a board and shareholders.

12:50 Most acquisitions in the valley are a larger firm acquiring a private firm. That's not all of them, but I would say that's the vast majority. And in those cases, there's often a qualitative threshold where someone's like, Yeah, let's do this. We've kind of have the high level terms, sometimes a term sheet, you know, formally. I think it's right after that. Um, so where people have really committed to the the key things, how much value, why are we doing this, the big stuff. And there's usually uh, you know, many lots of lawyers being paid lots of money to turn those term sheets into

13:25 you know, uh uh more complete set of documents, usually more complete due diligence, stuff like that. That's a there's an awkward waiting period there. And it that's a time, I think, where like the strategic decision makers in those moments can get together and say Let's talk through what this really means. And um The nice part about having them for all parties is you've kind of made the commitment to each other. So it's you've

13:48 I think you have more social permission to have real conversations at that point. Um but you also haven't consummated the relationship, you know, and so uh There's a the power imbalance isn't Totally there and and you can really talk through it.

14:03 And it also, I think, engenders trust just because by having harder conversations in those moments, you're learning how to have real conversations and learning how each other work. So that's my my personal opinion would have it. So you mentioned the board. You've been on the board of Shopify. You're on the board of OpenAI. You're a founder. What's the role of a board and how is it different when you're on the board of a founder led company? Günstig ohne App

14:28 Bei Aldi gibt es einen Preis für alle. Diese Woche Bio-Joghurt mit Crispy Müsli, 150 Gramm für nur 89 Cent. Oder Igloram Spinat, 750 Gramm für nur 1,69 Euro. Entdecke weitere Angebote in deinem Aldi Nord. Aldi Gutes für alle. I um

14:48 really like being involved um in a board. Um and I've been involved in multiple boards. Because I think I am an operator through and through. I I probably self-identify as an engine first more than anything else, and I love to build Learning how to Be an advisor. Um, is a very different vantage point that I think

15:11 uh you see how other companies operate and you also learn how to have an impact and add value without doing it yourself. Um and it's a very and I've really I think become a better leader, you know, having learned to do that. I have really only joined uh boards that were led by founders because typically I think they you can speak to them, but I think that they sought me out because I'm a founder and I like working with founder led companies. Um

15:38 I I think the uh Founders I I'm sure there's lots of studies on this, but I think founders drive better outcomes for uh companies. Um there's a I think founders tend to have

15:52 permission to make bolder, more disruptive decisions about their business than a professional manager. There's exceptions, like Satcha, I think is you know, uh one of the greatest, if not the greatest CO of you know, our generation and uh as a professional manager, but you know, you look at Uh everyone from uh Toby Lukey to Mark Benioff to Mark Zuckerberg to Uh, Sam at OpenAI. And I think when you're have founded

16:18 A company all your stakeholders, employees in particular, uh give you the benefit of the doubt. You know, you created this thing, and if you say, hey We need to um do a major shift in our strategy, even hard things like uh layoffs. Found to get a lot of latitude and are judged, I think, differently. And and I think rightfully so in some ways because of the interconnection of their identity to the thing that they've created.

16:44 And so I actually really believe in founder led companies. Um one of the real interesting challenges is going from a founder led company to not, and you know, Amazon has gone through that transition, Microsoft has gone through that transition. Um For that reason. Uh, but I love working with founders. Um, and I I love working with people like Toby and Sam because they're so different than me, yet um uh and I can see how they operate their businesses and I am inspired by it. I learn from it and obviously working for for Market Salesforce, you you have like wow, that's really interesting. Like almost like an anthropologist. Like why did you do that? You know, I want to learn more. And so I love working with founders that inspire me because I just learned so much from them.

17:26 Such an interesting front row seat into what's happening. Do you think founders go astray when they start listening to too many outside voices? And this goes back to the, I'm sure you're aware the Brian Chesky. You know he founder mode. The founder mode. Do you think talk to me about that. I have such a new point of view on this because It is decidedly not simple. Uh so

17:48 Broadly speaking, I really like the spirit of founder mode, which is just having deep founder led accountability for Every decision at your company. Um I think that that's how great companies operate. Uh and When you

18:06 you know, proverbially make decisions by committee, or you're f more focused on process than outcomes. Um, that produces all the experiences we hate as employees, as customers. You know, that's the proverbial DMV, right? You know, it's like process over outcomes. Um and then similarly Uh you look at the disruption in all industries right now because of AI, you know, the companies that will

18:30 recognize where things are clearly going to change. Like everyone can see it. It's like a s you know slow motion car wreck. Everyone knows how it ends. you need that kind of decisive breakthrough boundaries layers of management um to actually make change as fast as required in business right now. The Issue I have not with Brian's statements. Brian's amazing, um, is how people can sort of interpret that and sort of execute it as a caricature. Yeah.

18:58 Uh You know, there was a I remember after Steve Jobs passed away and you know, um I I don't know Ste I've met Steve a couple of times. I I having never worked with him in any meaningful way. You know, but he was sort of uh if you believe the story is like kind of uh pretty hard on his employees and and very exacting. And I think a lot of founders were like mimicking that, you know, Dun to wearing a black turtleneck and yelling at

19:22 Yeah, their employees. I'm like Not sure that was the cause. You know, uh I think Steve Jobs taste and judgment through you know, executed through that, you know, packaging was the cause of their success. And Somehow. And then similarly I think founder mode can be weaponized as an excuse for just like overt micromanagement. And that probably won't lead to great outcomes either. And most great companies are filled with extremely great

19:46 individual contributors who make good decisions and work really hard. And uh Yeah, companies that are like solely executing through the judgment of individual probably aren't going to be able to scale to be truly great companies. So I have a very nuanced point because I actually believe in founders. I believe in actually that Accountability that comes from the top. I believe in cultures where

20:11 you know, founders have license to go in and all the way to a m small decision and fix it, the infamous question mark emails from Jeff Bezos, you know, that type of thing. That's that's the right way to run a company. But that doesn't mean that you don't have a culture where individuals are accountable and empowered and uh you don't want uh, you know, people trying to decide make business decisions because of what will please our dual leader, you know, which is like the caricature of this. And so, you know, if after that came out, I could sort of see it all happening, which is like some people take that and be like, you know what? You're right. I need to go down and be in the details, and some people will do it and probably make everyone who works for them miserable, and probably both will happen as a consequence. So

20:49 Totally. Thank you for the detail and nuance there. I love that too. Do you think engineers made good leaders? I do think engineers make good leaders. But one thing I've seen is that I think that I really believe that

21:04 Great. CEOs and great founders um start usually with one specialty, but become Uh more broadly specialists in our parts of their business. Um you know, I think the uh businesses are multifaceted and rarely is a business's success due to one thing. Uh like

21:25 engineering or product, which is where a lot of founders come from. Often your go to market model is important uh for a consumer companies, how you engage with the world in public policy becomes extremely important. And I think as you see Um

21:41 uh founders, you know, grow from doing one thing to growing to be in a real meaningful company like Airbnb or Meta or something, you can see those founders really transform from being one thing to to many things. Um so I do think engineers make great leaders. I think the first principle's thinking the system design thinking um really benefits things like organization design, strategy Um and But I also think that, you know

22:09 uh when we were speaking earlier about identity I think one of the main transitions founders need to make, especially engineers, uh is you're not like the product manager for the company, you're the CEO and On any given day. W do you spend time recruiting an executive because you have a need?

22:27 Do you spend time uh on sales because that will have the biggest impact Um, do you spend time on public policy or regulation because if you don't uh it will happen to you and and could really impact your business in a negative way. I think Engineers who

22:45 are unwilling to elevate their identity from what they were to what it needs to be in the moment often leads to sort of plateaus uh in companies growth. So A hundred percent. I think engineers make great um leaders and It it's not a coincidence, I think, that most of the Silicon Valley Great Silicon Valley CEOs came from engineering backgrounds.

23:06 Um But I also don't think that's sufficient either as your company scales and I think that Making that transition as all the great ones have is incredibly important. To what extent are all business problems engineering problems? That's a deeper philosophical question than I think I have the capacity to answer. Um, what is engineering? What I like about

23:27 Approaching problems. Uh as an engineer is uh first principles thinking and understanding uh the root causes of issues rather than simply addressing the symptoms of the problem. And I do think that Coming from a background in engineering that is

23:45 Um everything from process, like how engineers do a root cause analysis of a outage on a server is a really great way to analyze why you lost a sales deal. You know, like I love the systematic uh approach of engineering. One thing that I think going back to good ideas that can become caricatures of themselves. Like one thing I've seen, though, with engineers who go into other disciplines is Um

24:09 Sometimes you can overanalyze decisions in some domains. Let's just take modern communications, which is driven in social media and and very fast paced. Um Having a systematic first principles discussion about every, you know, tweet you do is probably not a great con strategy. Um and so Uh and then similarly, um

24:32 You know, there are some aspects of, say, enterprise software sales that you know, are rational, but they're human, you know, like forming personal relationships, you know, and and the the importance of those to building trust with a a partner. It's not all just, you know, product and technology. And so I would say I think a lot of things uh coming with an engineer mindset can really benefit, but I do think that uh Taking that to its like logical extreme can lead to analysis paralysis can lead to

25:05 uh overintellectualizing some things that are fundamentally human problems. And so Yeah, I think a lot. Can benefit from engineering, but I wouldn't say everything's an engineering problem in my experience. You brought up first principles a couple of times. You're Running your third startup now.

25:19 Sarah. It's going really well. How do you use first principles In terms of How to use that at work?

25:26 Yeah, it's it's particularly important right now because the market of AI is changing so rapidly. So If you rewind two years, you know, most people hadn't used chat GPT yet. Uh most companies hadn't heard the phrase large language models or generative AI yet.

25:49 And in two years you have chat GBT becoming One of the most popular consumer services in history, faster than his than any service in history. And you have to have Across so many domains in the enterprise. Uh

26:04 really rapid transformation. The law is being transfer transformed. Marketing is being transformed, customer service, which is where my company, Ciro Works, is being transformed, software engineering is being transformed. And The amount of change in such a short period of time is uh I think unprecedented. Uh and uh perhaps I lack the historical context, but it feels faster. than anything I've experienced in my career.

26:31 And so as a consequence, I think uh if you're responding to the facts in front of you and not thinking from first principles about Why we're at this point. And where it will probably be twelve months from now, the likelihood that you'll make the right strategic decision is almost zero. Uh so А за екзам Uh it's really interesting to me that

26:54 with modern large language models, one of the careers that is being most transformed is software engineering. Uh And You know One of the things I think a lot about is How many software engineers will we have our company three years from now?

27:09 What will the role of a software engineer be as we go from being authors of code to operators of code generating machines? Um What does that mean for the type of people we should recruit? And if I look at the actual craft of software engineering that we're doing right now Um I think it's

27:28 literally a fact that'll be completely different two years from now. Yet I think a lot of people building companies hire for the problem in front of them rather than doing that. But two years is not that long. Those people that you hire now. We'll just be getting really productive a couple years from now. So we try to Think about most of our long term business from first principles, everything from I'll say a couple of examples in our business.

27:53 Our pricing model is really unique and comes from first principles thinking rather than having our customers pay a license for the privilege of using our platform. We only charge our customers for the outcomes. Uh meaning if the AI agent they've built for their customers solves the problem. There's like a usually a pre-negotiated rate for that. And that comes from the principle that in the age of AI, software isn't just helping you be more productive, but actually completing a task. Uh what is the

28:20 right and logical business model for something that completes a task, while charging for a job well done rather than charging for the privileges of doing the software. Um Similarly, um, you know, we with a lot of our customers, you know We help deliver them a fully working AI agent. We don't Hand them a bunch of uh software and say, Good luck, you know, configure it yourself.

28:42 And the logic there is, you know, uh In a world where Uh making software is easier than it ever is before and you're delivering outcomes for your customer, um, the delivery model of software probably should change as well. And we've really tried to reimagine what like the software company of the future should look like and trying to, you know, model that in everything that we do.

29:04 Brilliant. How do you think software engineering will change? Is it you're gonna have fewer people or The people are gonna be Organized differently or H how do you see that? How geeky can I get? As geeky as you want, right? I I actually wrote a blog post uh right before Christmas about this

29:23 I think this is an area that deserves a lot more research. Uh I'll describe where I think we are today and smart people may disagree, but Uh a lot of the modern large language models, both the traditional large language models and sort of the new reasoning models are trained on a lot of source code. And it's a an important input to all of the knowledge that they're trained on.

29:47 Um as a consequence, even the early models were very good at generating code. Um so you know, uh every single engineer at at Sierra uses uh Cursor, which is a great product that basically integrates with the IDE Visual Studio Code to help you Generate code more quickly. Um it feels like a local maximum.

30:08 uh in a really obvious way to me, which is you have a bunch of code Writ by people. um written in programming languages that were designed To make it easy for people Probably this

30:23 funniest example of this is Python. Um it almost looks like natural language. Yeah But it's notoriously not robust. Um you know, most Python bugs are found by running the program. Because There's not static type check in. Um similarly there's most bugs uh while you could run a

30:42 fancy static analysis. Like most bugs show up. simply at runtime because Uh It's just not designed, um it's designed to be ergonomic to write. Um yet we're using AI to generate that. Are we and so we sort of design most of our computer programming systems to make it

31:02 easy for the author of code to type it. quickly. Um And we're in a world where actually generating code is going to like the marginal cost of doing that is going to zero. But we're still generating code in programming languages that were designed for human authors. And similarly

31:21 Um If you've ever Like looked at someone else's code, um, which a lot of people do professionally, it's called the code review. It's actually

31:31 quite hard to do a code review. Um, you know, you end up interpreting, you're trying to basically put the system in your head and simulate it as you're reading the code to find errors in it. So The irony now that of taking things that are code programming languages that were designed for authors And now having humans do the job of essentially code reviewing code written by an AI

31:54 Um And yet all of the AI is being in the code generation part of it, I'm like, I'm not sure It's it's great, but we're generating a lot of code with similar flaws to that we've been generating before from security holes to just functional bugs. And in greater volumes. And I think we're uh what I would like to see is if you start with the premise that generating code is free or or or going towards free.

32:20 what would be the programming systems that we would design. So for example Uh yeah, Rust is an example of a programming language that was design For safety, not for programming convenience. Uh You know, my understanding is that the, you know, Mozilla project, you know, there were so many security holes in Firefox, they said, let's make a programming language that's very fast. Uh, you know, uh but everything can be checked statically, including memory safety.

32:48 Well it's a really interesting direction where you weren't operating like optimizing for authorship convenience or optimizing for correctness. Are there programming language designs that are designed so a human looking at it can very quickly evaluate, does this do what I intended it to do? There's an area of computer science I studied in college called formal verification, which at the time was turning a lot of computer programs into math proofs and finding inconsistencies and it sort of worked well as not as well as you'd hope. But you know In a world where AI is generating a lot of code.

33:21 You know, should we be investing more informal verifications so that the operator of that code generated machine can more easily verify that it does, in fact, do what they intended us to do. And could a combination of a programming language that is more structurally correct and structurally safe and exposes more primitives for verification plus a tool to verify. Could you make an operator of a code generated machine twenty times more productive, but more importantly, make the robustness of their output twenty times greater. And then similarly, you know.

33:52 There's a Themes uh things go in and out of fashion, but like test driven development, you know where you write your unit test first or your integration test first, and then write code until it fulfills the test. most programmers I know who are really good not despise it, but it's just like a it sounds better than it than it is in practice. But again, writing code is free, you know, so writing tests is free, you know. How can you create a programming system where the combination of great programming language design Formal verification.

34:21 robust test because you didn't have to do the tedious part of writing them all. Could you make something that made it possible to write increasingly complex systems that were increasingly robust. And then similarly like the elephant in the room for me is the anchor tenant of most of these code generated systems are uh an IDE right now, you know. And Um Uh, that obviously doesn't seem as important in this world.

34:44 And even with code in agents, which is sort of where the world is going. It doesn't change the fact that like, you know, who's accountable for the quality of it, who's fixing it. And I think There is a world where we can Make reasonable software by just automating what we as software engineers do every day, but I have a strong suspicion that if we designed these systems with the role of a

35:07 uh software engineer in mind being an operator of a machine rather than the author of the code. We could make the process much more robust and much more productive. And and it feels like a research problem to me. It doesn't feel and I think a lot of people, and for good reason, including me, are just excited about the efficiency of software development going up. And I I wanna see the new thing, though. I'm I'm Constructively dissatisfied with where we are. It's so interesting that if software AI is good enough to write the code, should be good enough to check the code.

35:38 Th that's a great um great question. But actually all you know, it's still funny to me that we'd be generating Python, you know, just because for anyone who listening right now has ever operated a web service running Python. It's CPU and intensive, really inefficient, you know. Should we be taking most of the unsafe C code that we've written and converting it to a safer system like Rust? Uh you know, if

36:01 authoring these things and checking it are relatively free. Shouldn't all of our programs be incredibly efficient? Should they all be formally verified? Should they all be analyzed by a great agent. I I do think it can be turtles all the way down. You can use AI to solve most problems in AI. The thing that I'm trying to figure out is like what is the

36:21 system that a human operator is using to orchestrate all those tasks. And You know, I go back to the history of software development, and most of the really interesting metaphors in software development came from breakthroughs in computing. So Yeah, the C programming language came from Unix and uh when these time sharing systems were really it went from sort of punch cards to something that were were a lot more agile Um, small talk came out of the development of the graphical user interface at Xerox Park and

36:50 You know, there's a uh sort of a confluence of message passing as a metaphor and and the graphical user interface. Um, and then there was a lot of really interesting um principles that came out of networking, you know, and um And sort of distributed systems, distributed locking, sequence in. I um I think we should recognize that we're in this brand new era as significant as the GUI, you know, it's like a completely new era of software development.

37:16 And if you were just to say, I'm gonna design a programming system for this new world from first principles, what would it be? And I think when we develop it, I think it will be really exciting because rather than Um automating and turning up the speed of just generating code and with the same processes we have today, I think will feel native to this system and give a lot more Uh control um to the people who are orchestrating the system in a way that I think will really benefit software overall.

37:45 Let's dive into AI a little bit. How would you define AGI to the layman? I think uh a reasonable definition of AGI uh might be that Any task that a person can do at a computer, um, that system can Do on par better. Um

38:05 I'm not sure it's a precise definition, but uh I'll tell you where that comes from and its flaws, but uh uh there's not a perfect definition of AGI, um, in my opinion. Uh or There's not a precise definition of AGI. I'm sure there's good good answers. One of the things about the G and AGI is about generalization. Um, so can you have a system that is intelligent in domains that it wasn't explicitly trained um to be intelligent on

38:33 And so I think that's one of the most important things is like given a net new domain can this system uh become uh more competent and more intelligent. Than a person sort of trained in that domain. Um and

38:49 Uh And I think that's sort of the, you know, at or better than a person is certainly a a good standard there. And and that's sort of the definition of superintelligence. The reason I mention at a computer is I do think that It is a bar that means like if there's a digital interface um to that system, um it affords the ability for AI to interact with it, which is why that's a bar. That's uh

39:15 reasonable to hit. Um I say that because one of the interesting questions around AGI is how quickly it does generalize. Um and Uh there are domains in uh the world that Um

39:32 No. The progress in that domain isn't necessarily limited by intelligence, uh, but by other social artifacts. So As an example, and I'm not an expert in this area, but if you think about Um

39:45 uh the pharmaceutical industry My understanding is, you know, the one of the main um bottlenecks uh is clinical trials. Um so no matter how intelligent a system would be in discovering new Uh therapy is

40:02 Uh it may not materially change that. And so you may have something that's discovering new insights in math, and that would be delightful and amazing. Um But

40:14 the existence of that uh system that's super intelligent in one domain may not translate to all domains equally. Uh, I just heard uh at least a snippet of a talk by Toller Cohen, the the economist and Um It was really interesting to hear his framing on this about which parts of the economy could sort of absorb intelligence uh more quickly than others. And so

40:37 I choose that definition of AGI recognizing that uh there's not a perfect definition. Because it captures the ability of this intelligence to generalize. while also recognizing that the um Demains of society, like it might not apply with uh equal velocity, even once we reach that point of a system being able to have that level of intelligence. When I think about

40:59 What Artificial intelligence is limited by, or the bottlenecks, if you will. I keep coming back to a couple of things. There's regulation, there's compute. There's energy, there's data. And there's LMs.

41:13 Am I missing anything? Uh So you're saying the ingredients to AGI? Yeah, like w W there's limitations on each aspect of those things and there seem to be the main contributors to the

41:24 What's limiting us from even accelerating. At this point, is that How do you think about that? Yeah, what you said is roughly how I think about it. I'll put it into my own words, though. I think the three primary inputs are Data.

41:38 Compute and algorithms. And Uh data is probably obvious, but you know Uh One of the things after the transformer model was introduced is it afforded

41:51 And architecture with just much greater parallelism, which meant models could be much bigger and train more quickly on much more data. Um, which just led to a lot of the breakthroughs with that's the L and L just they're large. Yeah and uh The scaling laws, you know, a couple of years ago, you know, indicated like the larger you make the model, um, the more intelligent it would be and out of degree of efficiency that was uh tolerable.

42:19 Uh And There we are, you know, there's lots of stuff written about this, but You know, there's in terms of just like textual content to train on, you know, the availability of new content is certainly waning, and and some people would say I think there's like a data wall Uh I'm not an expert in that domain, but it's been talked about a lot and and you can read a lot about it.

42:40 There's a lot of interesting opportunities though to generate data too. Um so Uh there's a lot of people working on simulation. If you think about a domain like self driving cars, simulation is a really interesting way to generate it. Is that synthetic data? Yeah, I would say that's synthetic data. The synthetic data has a Uh simulation and synthetic data are a little different. So

43:03 You can generate synthetic data like generate a novel. Okay. Simulation, I would put, at least in my head, and I'm sure that like academics might critique what I'm saying, but uh I view s s simulation as based on a set of principles like the laws of physics. So if you're building a real world simulation for training a self driving car. Um, you're not just generating arbitrary data like the roads don't.

43:27 turn into loop de loops, you know, because that's not possible with physics. So by constraining a simulation with a set of uh real world constraints, the data has more efficacy, you know, and so uh and there's uh sort of a it constrains the different permutations of data you can generate from it. So it's I think a little bit higher quality. But then along those lines, you know. Uh a lot of people wonder if you generate

43:54 Uh synthetic data. Um how much value can that add to a training process? Um You know is it sort of uh regurgitating information already had. What's really interesting about

44:05 you know, reasoning and reasoning models is I think uh I feel really optimistic these models are generating net new ideas. And so it really affords the opportunity to break through uh some of these the data wall as well. So data's one thing and I think Both synthetic data and simulation are really interesting. opportunities to to grow there. Then you have compute. And

44:28 Uh This is Um something that you know that's why uh there's so many data center investments. That's why NVIDIA as a company has has grown so much. Um, the probably the more interesting kind of breakthroughs there are these reasoning models where

44:45 Uh, there's not quite such a formal separation between the training process and the inference process where you can spin more compute at the time of inference to generate more intelligence, um, which has really been a breakthrough in a variety of ways. I think is really interesting, but it shows you how you can run up against walls and and find new opportunities to use it. And then finally algorithms and The biggest breakthrough was obviously the transformers model attention's all you need. That paper from Google that sort of led to where we are now, but there's been a number of really important papers since then from The idea of chain of thought reasoning to um what uh at OpenAI what we did with the O one model, which is to um do some reinforcement learning those chains of thought to

45:26 uh really reach new levels of intelligence. Um And so I do think that I mentioned some anecdotes about some breakthroughs there, because my view is that each one of them has their own Problems. Compute. It's very capital intensive. Um and

45:43 A lot of these models the half life of their value is pretty short because new ones come out so frequently and so You know, you're you wonder like, you know, can we afford uh what's the business case for investing those CapEx? And then you have a breakthrough like, uh you know, oh one and you're like, gosh. You know, with a distilled model and moving more to inference time, it changes the economics of it.

46:07 You have data and you say, Gosh, we're running out of textual data to train on. Well, now we can generate reasoning, we can do simulations. Oh, that's an interesting break there. And then on the algorithm side, as I mentioned, just the idea of these reasoning models is really novel itself. And Each of these at any given point. If you ha talk to an expert in in one of them, and I'm an expert in none of them. They will tell you the sort of current plateau that they can see on the horizon, and there usually is one. I mean

46:33 You'll talk to different people about how long the scaling laws for something will continue and you'll get slightly different opinions, but no one thinks it's gonna last forever. Um And At each one of those, because you have so many smart people working on them, you often have people discovering a breakthrough um in each of them. And so as a consequence I I really do feel optimistic about the progress towards HEI because

46:58 one of those plateaus might extend a while if we just don't have the the key idea that we need to break through. The idea that we will be stuck on all three of those demands. feels very unlikely to me. And in fact what we've seen because of the potential economic benefits of AGIs were in fact seeing breakthroughs in all three of them. And Um as a consequence, um, you know, you're just seeing just the blistering pace of progress that we've seen over the past couple of years.

47:24 At what point does AI start making AI better than We can make it or Making it better while we're sleeping or We we can't be too far from that.

47:35 Well th it might reflect back to our software engineering discussion, but you know, the broadly this is the area of AGI around self improvement, which is meaningful from a improvement standpoint, but also obviously from a safety standpoint as well. So I um I don't know when that will happen, but I do think, you know, by some definition you could argue that it's happening already in the sense that every engineer in Silicon Valley is already using coding agents and Um

48:06 platforms like Cursor to help them code, so it's contributing already. Um, and I imagine as uh coding assistants go to coding agents in the future. most engineers in Silicon Valley will show up in the morning and But this is sort of the difference between uh You know, the assisted driving and Tesla versus like self driving. Yeah. Right. Like at what point do we leap from

48:28 I'm a co pilot in this to I'm I don't have to do anything. I mean it's a question that's there's so much nuance to answer. I'm not sure to answer because I'm not sure you'd want to necessarily. Like I think for some software applications that's important. But when we brought up you know, we were talking about the active software development. People have to be accountable um for the software that they produce. Um and That means if you're doing something simple like a

48:55 software as a service application, that it it's secure. that it's reliable, that it the functionality works as intended for something as meaningful as uh, you know, uh an agent that is um somewhat autonomous. Does it have the appropriate guardrails? Um does it actually do what the operators intended? Is there appropriate safety measures? So I'm not sure there's really any system where you'd wanna turn a switch and and go get your coffee. But I do think to the point on, you know, uh these broader safety things is I think that when you think about

49:30 uh more advanced models we need to be uh developing not only more and more advanced um safety measures and safety harnesses, but also um using AI to supervise AI and things like that. So it's a part uh Probably my colleague on the board, Zico Kulter, is probably a better person to talk through some of the technical things, but there's a lot of prerequisites to get to that point. And I'm not sure it's simply like the availability of the technology, um, just because it is

49:56 uh that at the end of the day we are accountable for the safety of the systems we produce not just open like every Every engineer, um, and and and that's a principle that should not change. What does that mean, like when we say safety and AI? that seems so vague and general that everybody interprets it quite differently. Like how do you think about that? And how do you think about that in the world where Uh let's say we regulate safety in the United States and another country doesn't regulate safety. How does that affect the dynamic of it?

50:30 I'll answer broadly and then go into the regulatory question. So I really like OpenAI's mission, uh which is to ensure that AGI benefits all of humanity. That isn't only about safety. Um, and and I and I believe intentionally so, though obviously the mission was created prior to my arrival. Because it's both about safety, kind of hypocritical First Do no Harm and I don't think one could credibly achieve that mission if we created something unsafe. So I would say that's the most important part of the mission.

51:00 But there's also a lot of other aspects of benefiting humanity. Um, is it universally accessible? Is there a digital divide where some people have access to AGI and some don't? Um Uh Similarly, you could argue that does it are we maximizing the benefits and minimizing the downsides. Uh clearly uh AI will disrupt some job, but it also could democratize access to healthcare, education, expertise.

51:26 Um so As I think about the mission, it starts with safety, but I actually like thinking about it more broadly because I think At the end of the day, benefiting humanity is the mission and um Uh safety is a prerequisite, but it's almost like going to my analogy of the Hippocratic Earth. A doctor's job is

51:44 Yeah, f uh, to cure you. First do no harm, but then to cure you. And a doctor that did no harm but didn't cure you wouldn't be great either. So I really like to think about the uh holistically. And Um again, uh Zeke or Sam might have a uh more complete answer here, but broadly I think about Does the system that represents AGI um align with the intentions of the people created it and the intentions of the people operating it, um so that it

52:11 It does what we want and w it's a tool. Um uh that benefits humanity that um a tool that we're actively using um to affect the outcomes that we're looking for. And that's kind of the way I think about safety. Um and Uh it can be meaningful things like misalignment or or more subtle things like unintended consequences. Um and I think that latter part is probably the area that's uh really interesting uh from a

52:36 um intellectual and ethical standpoint as well. If I look at um Uh what was the bridge in Canada that fell down where it motivated the ring that a lot of engineers. Yeah, I forget the name of it, but just look at the whether it's the Tacoma Narrows Bridge in Washington or uh Three Mile Island or These intersections where um Uh

52:59 We've engineered these um what what at the time people hope would be positively impact humanity, but Something went wh horribly wrong. Um Sometimes it's engineering, sometimes it's bureaucracy, sometimes it's a lot of things. And so I don't think when I think about safety I don't just look at the technical measures of it, but how does this technology manifest in society? How do we make decisions around it? And you could take Put another way, technology is rarely innately good or bad. It's sort of what we do with it. Um and I think those social constructs and uh matter a lot as well. Um so I think it's a little early to tell because we don't have this kind of superintelligence right now.

53:38 Um and I think It won't just be a technology company defining how it manifests in society and you could imagine uh taking a very well aligned AI system and a human operator directing it towards something um that would uh objectively hurt society. And and there's a question of like who gets to decide, who's accountable, and and it's a perennial question. I mean it's whether you're deciding Uh, you know, uh

54:05 Uh should you use your smartphone in school? You know, who who should decide that? And I there's parents who will tell you, hey, it's my decision, it's my kid, and then there's principals will tell you it's not benefiting the school and I'm not sure that's gonna be my place or our place, but there there'll be a number of those conversations that are much deeper than that question that I think we'll need to answer. Um as it relates to regulation, uh

54:29 There's two Uh not conflicting forces, but two forces that exist somewhat independently but relate to each other. One is The pace of progress in AI and ensuring that you know, uh the f the folks working on frontier models are ensuring those models do benefit humanity. And

54:47 Uh and then there's the uh sort of geopolitical landscape, which is, you know, do you want uh AGI to be created by the freedom uh sort of uh the West, um where by democracies. Um, or do you want it to be created by more totalitarian governments And so I think the Inherent tension for regulators will be um a sense of obligation to ensure that, you know

55:14 uh the technology organizations creating AGI are in fact focusing enough on um Better infinity and humanity, all the other the uh stakeholders they're the whose uh interests that they're accountable for And ensuring that the West remains competitive. Um and and uh I think that's a really nuanced thing and I think uh you know my

55:37 My view is it's very important that the West is uh leads in AI, and I'm very proud of the fact that Um, you know, OpenAI is based here in the United States and and we're investing a lot in the United States and I think that's very important. And I also, you know having sort of seen the inside of I think we're really focused on benefiting humanity. So I I tend to think that, you know, it needs to be a multi stakeholder dialogue, but I think there's a really big risk that some regulations could have

56:02 the unintended consequence of of uh slowing down this larger conversation. But I don't say that to be dismissive of it either. It's actually just a impossibly hard uh problem and I think you're seeing it play out as you said in in really different ways in Canada, United States, Europe. China elsewhere. I wanna come back to compute and the dollars involved. So I mean on one hand you have

56:24 Um if I just I could start an AI company today by you know, going putting my credit card down using AWS and leveraging their infrastructure, which they've They've built they've spent the hundreds of billions of dollars and I get to use it on a time based model. On the other hand, you have people like OpenAI. uh Microsoft investing tons of money into it that may be more proprietary or Um how do you think about

56:51 The different models competing and then The one that really throws me for a bit of a loop is Facebook. So Facebook has spent meta. Oh God. So meta, I'm I'm like aging myself here. So Meta comes along and You know. possibly for the good of humanity, but like I tend to think Zuck is like incredibly smart. So I don't think I don't think he's spending, you know, a hundred billion dollars to develop

57:17 A free model and give it away to society. How do you think about that in terms of Return on capital and return on investment. It's a really complicated business to be in just given the capex required to build a frontier model. But let me just start with the couple definitions of terms I think are useful.

57:37 Um I think most large language models I would call foundation models and I like the word foundation because I think it will be foundational to most intelligent systems going forward and Most people build in modern models uh

57:52 particularly if they involve language, image or or Audio. shouldn't start from building a model from scratch. They should pick a foundation model, either use it off the shelf, Or a fine tune it. Um and so it's truly foundational in many ways. Uh in the same way

58:08 most people don't build their own servers anymore. They lease them from one of the cloud infrastructure providers, I think. foundation models will be something trained by companies that have a lot of capex and leased by a broad range of customers who have a broad range of use cases. Um And I think that leads in in the same way that data center builders having a lot of data centers enabled you to have the capital scale to build more data centers. I think the same will largely be true of

58:37 Uh, you know, building the huge clusters to to do training and things like that. Foundation models, I think, are somewhat distinct from frontier models. And frontier models, I think it's a term credit to to Reed Hoffman, but Uh I may be mistaken on that. That's where I heard it from. And these are the models that are usually like the one or two that are clearly the leading edge, O three as an example from OpenAI. And these frontier models are being built by labs who are trying to

59:05 Build A GI that benefits humanity. And I think if you're Deciding whether you're Building a foundation model and uh what your business model is around it. It's very different business than

59:18 I'm gonna go pursue AGI. Uh 'Cause if you're pursuing A GI really there's only one answer, which is to build and train and move to the next front, you know, horizon because if you can truly build something that is AGI, the economic value is so great. Uh I think there's a really clear business case there.

59:38 If you're Yeah. Pre training a foundation model that's The Fourth best.

59:44 Uh That's gonna cost you a lot of money. And The return on that vestment is is probably fairly questionable because Why use your fourth best large language model versus a frontier model or an open source one?

59:58 Uh from meta. And as a consequence of that, I think we have probably have too many people building models right now. There's already been some consolidation, actually, of companies uh being folded into Amazon and and Microsoft and others. But I do think it will play out a bit like the cloud infrastructure business, where a very small number of companies with very large CapEx budgets uh are responsible for both building and operating these data centers and then developers and and consumers will use things like chat GPT as a consumer or as a developer, you'll license

1:00:32 uh and rent, you know, one of these models uh in the cloud. Um How it will play out is a really great question. You know, I think the uh I heard one investor talk about these as like the fastest depreciating assets of all time. Um Uh

1:00:47 On the other hand, you know, I Uh, if you look at the revenue scale of something like an open AI and and what I've read about places like Anthropic, let alone Microsoft and Amazon. It's pretty incredible as well. And so you can't really if you're one of those firms, you can't afford to sit on the sidelines as as the World transforms. But I I would have a hard time personally like funding a a startup that says I'm gonna do pre training. You know, it's it's uh it's I don't really know like what's your what's your um

1:01:17 differentiation in this marketplace. And I think a lot of those companies you're already seeing them consolidate because they have the cost structure of a pharmaceutical company, but not the business model. But this is just it though, right? Like OpenAI has a revenue model around A revenue model. Microsoft has a revenue model around their AI investments. They just updated the price of Teams. with Copilot, you know, uh Amazon has a revenue model around

1:01:44 AI, in a sense, they're getting other people to pay for it through AWS and then they're getting the advantages of it. uh at Amazon too, from a consumer point of view, and all the millions of projects Bezos was doing an interview last week. He said there's every project at Amazon basically has an AI component to it. Facebook, on the other hand, has spent all of this money already and with You know, an endless amount Presumably.

1:02:09 In sight or like not in sight, an endless amount to go. they don't have a revenue model specifically around AI, where it would have been cheap, obviously, for them to use a different model, but that would have required presumably giving data away or Like I'm just trying to work through it from Zuck's point of view. If Yeah, I actually will take Mark at his word. And you know, that post he wrote about open source, I think was very well written and encouraged people to read it.

1:02:38 I think that's a strategy. And you know, if you look at Facebook Um So that was what that was that was what was called. You know, the company's always really embraced open source. And if I look at really popular things from React to uh you know, now the Llama models, it's always been a big part of their strategy to court developers around sort of their ecosystem and Mark articulated some of the strategy there and there I'm sure there's elements of commoditizing your compliment.

1:03:07 But I also think that You know, if you can attract developers towards models, there's a strength. Um I uh you know, I'm not really on the inside there, so I don't really have a perspective on it other than I actually think it's really great that Totally. There's different players with different incentives all investing so much and I think it is really

1:03:28 furthering the cause of like bringing these amazing tools to society and Um but a lot changes. I mean If you look at the Price of GPT four O Mini. You know, it is uh

1:03:41 So much higher quality than like the highest quality model two years ago and much cheaper. Um I I haven't done the math on it, but it's probably cheaper to use that than to host self host any of the open source models. So even Even the existence of the open source models, it's not free. I mean inference cost money and so There's a lot of

1:04:02 complexity here and and actually I have the email even being relatively close to stuff, like I have no idea where things are going. But Yeah, it's um You could talk to a smart engineer and they'll tell you Oh yeah, if you built your own servers, you'll spend less than renting them from, say, Amazon Web Services or Azure. That's sort of true in absolute terms, but d misses the fact, like, do you want someone on your team building servers? Oh, and in fact

1:04:27 If you change the way your service works and you need a different SKU, like you all of a sudden are doing training and you need NVIDIA uh you know H one hundreds. Now all of a sudden your built servers like this. You know, asset that's worthless. So Yeah, I think with a lot of these models, you know, the presence of open source is incredibly important and and uh and I uh really appreciate it. I also think like the economics of AR

1:04:51 pretty complex because The hardware is very unique. The cost to serve is much higher. Um techniques like distillation have really changed the economics of models, whether or not it's Open source or or

1:05:06 uh s hosted and and and leased Um So it's I think broadly speaking for developers it's kind of an amazing time right now because you have a Like a menu of options that's incredibly wide. And I actually think of it as, you know, just like in cloud computing, you'll end up with a price performance quality trade off. And for any given engineering talents, they'll have a different answer and that's appropriate. And some people use uh open source Kafka, some people work with Conflient. Um

1:05:35 Great. You know, like that's just the way these things work, you know, and uh So you don't think AJ is gonna be like a winner take all. You think there's gonna be multiple options that have By definition, whatever the definition is of AGI. Well, First I think

1:05:49 OpenAI, I believe, will play a huge part in it because uh there's both the technology, which I think OpenAI continues to lead on. Um, but also chat GPT, which has become synonymous with AI for most consumers. But more than that, Um it is the way most people access AI, um today. And so One of the interesting things, like what is AGI? We talked about you know, opinions on what the definition might be. But the other question is like

1:06:18 How do you use that? Like what do you What is uh what is the packaging? Um and some of uh intelligence will be Simply the outcomes of it, like a discovery of a new drug, which would be, you know, remarkable and hopefully we can cure some illnesses. Uh But others will be just how you as an individual access it. And you know, I

1:06:39 Most of the people I know, like if they're signing an apartment lease, we'll put it into chat GPT, get a legal opinion. Uh, if you get, you know, lab results from your doctor, you can get a second opinion on on Chat GPT. Um Clay and I use uh the O one Pro mode for like give criticizing our strategy at Sierra all the time. And so for me, what's so remarkable about Chat GPT, which was this, you know, quirkly named

1:07:06 research preview that has come to be synonymous with AI is I do think that It will be the delivery mechanism for AGI when it's produced and uh Not just because of the main researchers at OpenAI, but because of the amazing like utility it's become from individuals. I think that's really neat because I don't know if it would have been obvious if we were having this conversation three years ago, um

1:07:30 You know And you were talking about artificial general intelligence, I'm not sure w either of us would have envisioned something so simple as a form factor. Yeah. Uh to absorb it that you just talk to it. Yeah. Um So I think it's great and especially as I think about the mission of OpenAI, which is to ensure That A GI benefits humanity. What a simple accessible form factor. There's free tiers of it. Like what a kick ass way to benefit humanity. So I really think that will be central to what we come as society to define uh as a GI.

1:08:02 You mentioned using it at Sierra to critique your your business strategy. What do you know about prompting? Yeah. Other people miss. I mean, you must have the best prompts. People think that, you know, because I'm affiliated with it. You're not going like, here's my strategy, what do you think? What are you putting in there? Um

1:08:21 I often with the uh reasoning models which are slower will use a faster model first, GBT four O to refine My prompts. Um so Uh Over the holidays, um, partly because I was thinking about the future of software engineering. I've I've written a lot of compilers in my time. I'm like Written enough that I

1:08:42 Yeah, it's like uh Uh It's it's easy for me. So I decided to see if I could have O one Pro Mode, um Generate.

1:08:52 In to end a compiler front end parsing the grammar um check in for semantic correctness, generating an intermediate representation, and then using Uh Uh L VM, which is sort of a

1:09:06 compiler collection, um, that's very popular to actually do Yeah, run, run it all. And I would spend a lot of time iterating on four O to sort of like refine and and make more complete and specific what I was looking for And then I would put it into a one promo and go get my coffee and you know, come back and get it.

1:09:26 I'm not sure if that's a viable technique, but it's really interesting because I do think in the spirit of AI being the solution to mo more problems than AI. I'm having a uh lower latency, simpler model, help refine Essentially I like to think of it as like you're like a product manager and you're asking, you know, an engine what to do. Is your is your product requirements document complete and specific enough? And uh Waiting for it is sometimes slower than and so I I like doing it in stages like that. So that's my my trip. Uh at some point there's probably someone from OpenAI listening who's gonna like roll their eyes, but that's just uh that's I've Who can I talk to at OpenAI that's like the prompt. Yeah, I think I'm like so curious about this because I've I've actually taken recently to

1:10:09 Uh getting Open AI. Or chat GBT, I guess, if you want to call that. I've been getting Chat GBT to write the prompt for me. So all Prompt it with I'm prompting an AI. Yeah. Here are the key things. Similar to my technique. I want to accomplish what would an excell look like. And then I'll copy paste that prompt that it gives me back into the system. Uh but I'm like, I wonder what I'm missing here, right? Like it's a good technique. I mean there's lots of engine techniques like that, like self reflection.

1:10:40 is a technique where you have a model observe and critique, you know, a decision like a chain of thought. Uh so uh in general, you know, that mechanism of self reflection is I think a really effective technique. You know, at Sierra we Help companies build customer facing AI agents. So um If you're setting up a Sonos speaker, you'll now chat with an AI. If you're a serious XM subscriber, you can chat with Harmony, who's their AI to

1:11:07 Manager account. Um We use all these tricks, you know, self reflection to detect things like hallucination or decision making. generating chains of thought for more complex tasks to ensure that it's you know, you're putting as much uh compute and cognitive load into important tricks. So Uh you know, we're

1:11:29 the there's a whole industry around sort of figuring out how to exact the like robustness and and um precision out of these models. So it's really fun, but changing rapidly. Hypothetical question, you you've been hired to lead or advise a country. uh that wants to become an AI superpower. What sort of thing is?

1:11:49 um steps would you take? What sort of policies would you think would help create that? How would you bring investment from all over the world into that country? And researchers, right? Like so now all of a sudden you're competing. It's not the United States. Like how do you how do you sort of set up a country, like from first principles all the way back to like what does that look like? What are the key variables? Well, I mean especially Uh this is definitely outside of my domain of expertise, but I would say One of the key ingredients to

1:12:19 modern AI is compute, um, which is a a noun that wasn't a noun until recently, but now compute. is a noun and Uh, you know, I do think that's one area where policymakers can um uh because it involves a lot of things that Uh Touch uh

1:12:37 federal and local governments like power land, um, and then similarly attracting the capital, which is immense to finance uh to the real estate to purchase the uh you know uh compute itself Um and then to sort of operate the data center. And again, there's really immense power requirements uh for these data centers as well.

1:12:59 Um and then you know, it's attracting sort of the right researchers and research labs to You know leverage that, but in general, where there is compute, the research labs will find you, you know, and so I think that's it. And then there's a lot of national security implications too, just because, you know, these models are very sensitive, at least the frontier models are. And so Um, you know how you your place in the geopolit was quite important. Like will research labs and

1:13:28 Uh will the US government be comfortable with training happening there and and export restrictions and things like that. But I think a lot of it comes down to infrastructure, uh, as it relates to policy is my intuition. Uh, you know I think right now so much of AI is constrained on on infrastructure that that is the input to a lot of uh of this stuff. Um Uh and and then there's a lot around, you know, attracting talent and all that. But as I said, you know, you look at the

1:13:58 research labs, it's not that many people actually. It's a lot, but the Compute is a limited resource right now. That's a really good way to think about it. I I think about this from the lens of Canada, right? Which is like we we don't have enough going on in in AI. We we tend to lose most of our great people to the States. uh who then go to set up infrastructure here for whatever reason and don't bring it back to Canada.

1:14:24 And I I wonder how Canada can compete better. So this is like sort of the lens I I like look at these questions through. How do you see the the next generation of education? Like if you were setting up a school today from scratch. And again, hypothetical, not your domain of expertise, but like using your lens on AI. How do you think about this? So like What skills will kids need in the future? And what skills do we probably don't need to teach them anymore that we have been teaching them?

1:14:54 We'll start with the the benefits that I think are probably obvious, but I'm incredibly excited about I think education Can become much more personalized. Oh, totally. Have you seen Synthesis Tutor, by the way? No, I have not. Oh, so they developed this uh Synthesis, this AI company developed this tutor, which actually teaches kids. And it's so good that El Salvador, the country, just recently adopted and replaced their teachers. And uh like it'll teach you, but it teach you specific to what you're missing.

1:15:27 So it's not like every lesson's the same. It's like well, you're not understanding this foundational concept. So it's like K through five or six right now. That's amazing. And you know, I actually And the results are like off the charts. Well It doesn't surprise me and I I don't actually view it as like necessarily replacing a teacher, but my view is if you have a teacher with twenty eight kids in his or her class the likelihood that they all learn the same way or learn the same pace is very unlikely. And you know, I can really think of a say an English teacher, a history teacher orchestrating their learning journe, uh topic, say AP European history in the United States.

1:16:05 There's a curriculum, they need to learn it. Um how someone will remember something or understand the significance of Martin Luther, you know Is very different. And Um you can, you know, generate a audio podcast for someone who might be an audio auditory learner.

1:16:23 Um you can create cue cards for someone who needs that kind of repetition repetition Um you can visualize uh key moments in history um for people who just Maybe wanna more viscerally appreciate why this was a meaningful event rather than this dry piece of history. And all of that, as you said, can be personalized to the way you learn and how you learn and I think it's just incredibly powerful. And so One of the things I think is neat about AI is it's uh

1:16:49 democratizing access to a lot of things that used to be fairly exclusive. A lot of wealthy people, if their child was having trouble in school, would pay for a tutor, a math tutor, science tutor. Um and you know if you look at uh kids who are trying to get into, you know, uh big name colleges, you know, if you have the means, you'll have someone prep you for the SATs or Help you with your college essays. All of that should be democratized if we're doing our jobs well. And it means that we're not

1:17:18 Lebanon people's opportunity from by their means. And I think that's a just the the most uh American thing ever, uh Canadian as well. Um It's the most incredible thing. It's the most incredible thing humanity and And so I I just think education will change for the positive in so many ways, um, because Uh I I actually with my kids walking around when they ask uh you know, if you have little kids ask why, why, why and you know.

1:17:44 There's some point a parent just starts making up the answer or being dismissive and like we have chat GPT out and it's like the best when you're traveling and put on advanced voice mode and be like ask away. A hundred percent. And I'm listening too, you know? It's like you're uh you Live through your children's curiosity. And um my daughter went to high school and came home with Shakespeare for the first time. And I was she asked me a question, I was like, I I felt this is like total inadequacy. I was like, I was very bad at this the first time, and then we put it into chat GPT and it was the most thoughtful answer and she could ask follow up questions. And I actually was You know, with her because I was like, Oh, I forgot about that. Didn't even think about that. So I d I just think it's incredible. And I would like to uh in public school systems, uh

1:18:28 I think it's really I think it'll be a really great uh when public school systems formally adopt these things so that they lean into Uh tools like ChatGPT uh as mechanisms to like raise uh the performance level of their classroom and and hopefully you'll see it in things like test scores and other things because uh kids can get the extra time even if the school system can't afford it for everyone. Uh and then most importantly, kids car getting explanations according to their style of learning, which I think will be um quite uh important as well.

1:19:03 As it relates to skills, it's really hard to predict right now. And I I would say that I do think learning how to learn and learning how to think will continue to be important. So I think Most of you know primary and secondary education shouldn't And is not vocational necessarily. Um, some of it is. Uh, you know, I took auto shop and all of that, and I'm glad I did, but I couldn't fix my

1:19:25 Electric car today with that knowledge, yeah. Things change. And I don't think it needs to be purely, you know, um, nonvocational, but you know, the basics of learning how to think, uh, learning um Uh Write in, read in math, physics, uh, chemistry, biology.

1:19:44 Not because you need to memorize it, but understand the mechanisms that uh Uh create the world that we live in is is quite important. Um I do think that the

1:19:57 There's a risk of people sort of uh becoming ossified in the tools that they use. Um so you know, uh Th uh let's go back to our discussion with software engineering for a second, but I'll give other examples. You know, if you define your role as a software engineer is how quickly you type And see your IDE.

1:20:17 the next few years might leave you behind, you know, because that um that is no longer a differentiated, you know, part of the software engineering experience or will not be. But your judgment as a software engineer will s continue to be uh incredibly important. Your agency and making a decision about what to build, how to build it, um, how to architect it. Uh maybe using AI models as a creative foil.

1:20:44 And so I think that uh just in the same way, if you're an accountant, you know, using Excel doesn't make you less of an accountant. Uh and Um and just because you didn't, you know handcraft that math equation. It doesn't make the results any less valuable um to your clients. Um and so I think we're gonna go through this transformation where I think the um

1:21:05 The tools that we use to create value in the world will change dramatically. And I think some people who define their jobs by their ability to use the last generation's tools really, really effectively. um will be disrupted. But I I think we if we can empower people and and to reskill

1:21:26 Um and also broaden the aperture by which they define the value they're providing to the world. I think a lot of people can make the transition. The thing that is sort of Uncomfortable, uh not really an education where it's just earlier. In in most people's lives. It's just I think the pace of change uh exceeds that of most technology transitions, and I think it's unreasonable um to expect most people to

1:21:49 change they were work that quickly. And so I think the the next Five years I think will be, you know, for some jobs really disruptive and tumultuous. But if you take the longer view and you fast forward twenty five or or fifty years, I'm incredibly optimistic. I I think it's the the change. Will require um uh from society, from companies and from individuals, like an open mindedness about reskilling and and

1:22:14 reimagining their job to the lens of this like dramatically different new technology. At what point do we get to I mean, we're probably on the cusp of it now and it's happening in pockets, but what point Do we start solving problems that humans haven't been able to solve or eliminating paths that we're on, maybe with medical research, that it's like, no, that that the this whole thing you've spent thirty billion dollars on, you know, based on this 1972 study that was fabricated, but that one study had all these derivative studies and like I'm telling you it's false, you know, because I can look at it through an objective lens. And get rid of these thirty billion. Why you're smiling. Oh no, I just I hope soon. I mean I hope I mean I

1:22:57 Uh There was a A lot of there's a one of the models, I can't remember which one, introduced a very long context window and there's a lot of people on X over the weekend putting in their uh thesis, you know, like grads, grad school thesis in there. And and it was actually critiquing them with like surprising levels of fidelity. Uh And uh I think we're sort of there, perhaps, with the right um right tools, but certainly over the next few years. I you know, we talked about how gener what does it mean to generalize AI.

1:23:27 Certainly in the areas of um science that are, you know largely represented through text and digital technologic math being probably the most uh most applicable There's not really anything keeping AI from getting really good at math. There's not really an interface to the real world. You don't need to do a clinical trial to verify something's correct. So I feel a ton of optimism there. Um

1:23:52 It'll be really interesting in like, you know, areas of like theoretical physics. Um Uh, you'll just intent you'll continue to have the divide between the applied and the theoretical people, but I think there could be like really interesting New ideas there and perhaps some uh finding logical inconsistencies with some of the, you know. uh fashionable theories which has happened many times over the past few decades.

1:24:13 Um I I think I think we'll get there soon. And I actually I um What's really neat about it is most of the scientists. I know people who are actually like doing science. Like they're the most excited about these technologies. And I they're using them already. And I think that's really neat. And I think we're hopefully going to be I really hope we see

1:24:32 More breakthroughs in science. One of the things I am not an expert in, but I've read a lot like a a lot as a amateur about it's just the slowdown in scientific breakthroughs over the past, you know, a few decades and and some theories that it's because of the degree of specialization that we demand of grad students and things like that. And I hope with you know, in general with AI, um Democratizing access to expertise. Um

1:25:00 I I have a completely personal theory that It will benefit deep generalists in a lot of ways, too, because your ability to understand a fair amount in a lot of domains. And leveraging AI, um, knowing where to prompt the AI to to go explore and and um

1:25:20 bringing together those domains, it will start to shift sort of the intellectual power from people who are extremely deep to people who actually Can uh orchestrate Uh intelligence between lots of different domains for breakthroughs. And I think that'll be really good for society, because most scientific breakthroughs aren't

1:25:38 They tend to be, you know, cross pollinating Very important ideas from a lot of different domains, which I think will be really exciting. How important is the context window? I think it could be quite important, um Especially It certainly simplifies working with an AI. If you can just give it everything and

1:25:55 instruct it to do something. Um and so And and assuming it works, you know, you can extend a context window and it can um Mm. the the tension can be spread fairly thin and and and The robustness of the answer can be questionable. So but assuming let's just for argument's sake, you know, perfect robustness, um

1:26:16 I I think it can really simplify the interface uh to AI. The Not all uses I also think that we're talking about open source models and and APIs. Um I also think that you know most

1:26:31 What I'm excited about in the software industry is Not necessarily a large language model with a prompt and a response being the product of AI, but actually end to end closed loose systems that use large language models as pieces of infrastructure. And I actually think that a lot of the value in software will be that. And for many of those applications, the context window size can matter, but often because you have contextual awareness of the process that you're executing. Um

1:26:58 Yeah, the contact window is a little bit less important. So I think it matters a lot to intelligence. Um Yeah, there's a I can't remember someone one of the Some researcher said, you know, you put all of human knowledge in the context window and you ask it to invent the next thing. And that's a obviously a reductive uh thought, but it but interesting. Um Uh, but I actually I'm equally excited about sort of the industrial applications of large language models, sort of like my company, Sierra. And if you're um returning a pair of shoes at a retailer and it's a process that's fairly complicated and

1:27:29 Uh, you know, is it within the return window? Uh you know, uh do you want to return it in store or do you wanna send it, do you wanna print a QR code, blah, blah, blah, blah. Um The orchestration of that is as significant as the models themselves, and I actually think as we I'm just like uh uh computers, you know, there's gonna be a lot of things where computers are a part of the experience, but it's not like manifesting itself as a computer. So I I'm actually equally excited about those. And I think context window is slightly less important than those applications. Do you think that the

1:27:59 output from AI should be copyrightable or patentable or Well, let me just take an example if I go to the US patent office, I download a patent for Let's say the arrow press. And I upload it to uh O One Pro. And I say, I can't upload it yet,'cause you don't let me do the PDFs, but I upload it to four and

1:28:18 Uh so I say, Hey, what's the next logical leap that I could patent off this? It would give me back diagrams and an output. And presumably if I I look at that and I'm like, Yeah, that's legit. I wanna file that patent. Can I? I don't know to answer that question. I'm not an expert in sort of intellectual property, but I Uh Uh I think there will be a an interesting question of was that your idea because you used a tool to do it.

1:28:44 I think the answer's probably yes, that you you used the tool to do it. But I also think that the um Uh Uh In general, like the sort of marginal cost of intelligence will go down a lot. So a lot of the

1:29:00 You know, I think in general, like we'll uh we'll be in this renaissance of of new ideas and intelligence being produced. And so Uh, I think that's broadly a good thing. And I think, you know, the marginal value of that insight that you had might be lower than it was, you know, n years ago. What I was hoping you would say is that, you know, that's gonna become less and less important because I feel like all the patent trolls and all of this stuff that slows down innovation in some ways. uh obviously like there's legitimate patents that people infringe on and there should be legal recourse. But if I could just go and patent like a hundred things a day.

1:29:36 It seems like that should not be allowed. Well, in general, I think that you know companies you know I think Pan's make sense if it's projecting something that's an active use that you, you know, invented and you're you're trying to uh you know st like the standard, you know, uh legal rationale for patents just generating a bunch of ideas and patenting it seems uh destructive to the value of a card. So here's the idea I had last night to counter this because I was like, I don't want somebody doing this. Yeah. Uh and I was thinking, like, what if prior art eliminates patents? Yeah. So it's like what if I just set up like an instance and just publish it on a website. Nobody has to read that website. Here's a billion ideas. Exactly, but it's like basically patenting like anything. Prior art patent. But it's creating prior art for everything. So like you can't compete on that anymore. I don't know. I was like thinking about that. I thought it was fun.

1:30:29 Um Tell me about the Google Maps story. This is like now legend and I wanna hear it from you. Uh this is my weekend coding. Is that what you want to hear about? Yeah. Um yeah, so Uh. I'll start with just like the story of Google Maps, the abbreviated version. Uh

1:30:46 We had launched a product at Google called Google Local, which was sort of a yellow pages uh search engine. Uh, you probably probably most listeners don't even know what yellow pages are, but it was a thing back then. And Um We had licensed maps from MapQuest, which was the dominant sort of mapping provider at the time. And It was sort of an eyesore on the experience and also

1:31:08 always felt like it could be a more meaningful part of the kind of local search and navigation experience on Google. So Larry Page in particular was really pushing us to really invest more into maps. Um, we found this uh small Company with a Like four people in it, if I'm remembering correctly, started by Lars and Jens Rasmussen Coldware Two Technologies where

1:31:29 Um they had made a Windows application called Expedition, um that was just a beautiful uh mapping product. Um It was running on Windows long after it was sort of out of fashion to make Windows apps, but they they're sort of where the technology they're comfortable with. But they're really um their their maps modeled the A to Z maps in in the UK and were just beautiful and They just had a lot of passion for mapping. So

1:31:55 We did a little aquair of them and took together the Google local team and Lars and Jens's team and and said, Okay, like let's take the good ideas from This Windows app and the good ideas from Google Local and like Let's bring them together to make something completely new. And that and that's what became Google Maps. But there's a couple of idiosyncrasies in the integration because it was a um

1:32:17 Windows app It really helped and hurt us in a number of ways. Like one of the ways it helped us is the reason why Google Maps We were able to drag the map and it like uh was so much more interactive than any web application that preceded it was The standard that we needed to hit from interactivity was set by a native Windows app, not set by

1:32:37 the legacy uh, you know, websites that we had used at the time. And I think that by having the goalposts so far down the field because they had just started with this Windows app, which was sort of a quirk of Lars and Young's just like technical choices. We made much bolder technical bets than we would have otherwise. I think we would have ended up much less interactive had we

1:33:00 uh not started with that quirky technical sort of uh decision. But the other thing was this Windows app, there's a lot of like it's hard to describe the like early two thousands people didn't live it but like X M L was like really in fashion. So like most things and Windows and other places like XML and XSLT, which was a way of transforming XML into different XML, was the basis of everything. It was like all of enterprise software was like XML this, XML that so Similarly when we were taking some of these ideas and putting them in a web browser

1:33:31 We kind of like went into autopilot and used like a ton of XML And it made Everything just like really, really tedious. And so Google Maps launched with some really great ideas like the dragable maps and and we did a bunch of stuff with the local search technology so you could you know overlay restaurant listings was really great. It was a really successful launch.

1:33:53 Uh we're like the hot shots within Google afterwards. And uh but it really started to show its craft. And we got to this point where we decided we wanted to support the Safari web browser, which was relatively new at the time. This is before, you know, mobile phones. And uh there was much less XML support in Safari than there was in Internet Explorer and Firefox. And so One of the engineers implemented like a full XSLT transform engine in JavaScript to get it to work, and it was just like Shit on top of shit on top of shit. And so

1:34:25 What was a really elegant fat like fast web application had sort of quickly become something, you know, there's a lot of dial-up modems at the time. I think so, like you'd show up to maps. And it just was slow and like it just bothered me as like someone who Takes a lot of pride in their craft. And so I got really uh energized and like over a weekend and a lot of coffee, like rewrote it. Um but it's rewrote the whole thing, though.

1:34:52 Rerrote, yeah, more or less the whole thing. And it it took probably another week of like, you know, working through the bugs. But yeah, I sent it out to the you know, the team after that weekend. And it was It was night the reason I was able to do it, yeah, I'm like a decent programmer, but You know, you'd also like lived with every bad decision up to that point, too. So I knew Exactly

1:35:14 the output I was gonna like I had simulated in my head, like if I could do it over again, this is the way I'd do it. So by the time I like put my hands on the keyboard on like, you know, Friday night, I it wasn't like I was designing a product. Like I knew I had been in the every detail of that product since the beginning, and including maybe the bad decisions too. They're not all the bad decisions. And so it was just very clear I knew what I wanted to co accomplish. And for anywa you know any engineers worked on a big system, you have the whole system mapped out in your head. So I knew Knew everything and I and I Um

1:35:45 And and I also knew that You know There's a lot of pride of authorship with engineering and code. So I sort of knew I really wanted to finish it over the weekend so that people could use it and see how fast it was. And kind of overcome anyone who is like, you know.

1:36:01 you know, protective of the code they had written a few months ago. And so I really wanted the prototype to go out. And so I did it. And then I didn't It's funny I never talked about it again, but I think Paul Buhay, who's uh was the co-creator of uh uh the uh Gmail and and I worked and started Friend Feed with me. Um, he was on an interview and mentioned this story, so now all of a sudden it's like Uh everyone's talking about it. I was like, Well thank you, Paul. It's a little embarrassed uh that people know about it, but uh it was it was it's a true story and um And it and uh XML is just the worst. Did you get a lot of flack from the people who had built the system you effectively replaced? Like you were part of that team, but everybody else had so much invested in it, even though it was like shit on top of shit on top of shit.

1:36:44 You know, um I wrote a lot of it too. So yeah, I I'm sure there was some around it, but actually I think good teams want to do great work. And so Uh I think there was a lot of people constructively dissatisfied with the state of things too, and um Uh You know, I think uh you know, the engineer had written that X SSL T transform I think was like, you know, a little bit That's a lot of work. So you have to throw out a lot of work, which feels bad, but

1:37:08 Particularly, you know, um Lars and Jens and I like we want to make great products. And so I don't think there was a Yeah, at the end of the day, everyone's like, Wow, that's great. You know, we went from a bundle size of two hundred K to a bundle size of twenty K and It was a lot faster and better. So You know, broadly speaking I think good engineering cultures you don't want a culture of um

1:37:30 You know ready, fire, aim, but I also think you just need to be really outcomes oriented. And I think people If they become a They'd start to treat their code as too precious. It can really uh impede forward progress. Um and Yeah, I'll just take like I

1:37:47 My understanding is like a lot of the early self driving car s software was a lot of Hand coded heuristics and rules and You know, a lot of smart people think that eventually it'll probably be a more monolithic model that uh encodes many of the same rules. You have to throw out a lot of code in that transition, but it doesn't mean it's not the right thing to do. And so I think in general Um yeah, there might have been some feathers ruffled, but at the end of the day, everyone's like that's faster and better. Like let's let's do it, you know, which is I think the right decision. That's awesome.

1:38:17 I'm gonna give you another hypothetical. I want you to share your inner monologue with me as you think through it. So if I uh told you you have to put a hundred percent of your net worth into a public company today and you couldn't you couldn't touch it for at least twenty years. What company would you invest in? And like walk me through your thinking. I literally don't know how to answer that question. Um

1:38:39 How would you think about it without giving me an answer? Like what Yeah, it's a good question. I first give you how I'd think about it, but I'm so uh having not been a public company CEO for a couple of years, I'm blissfully don't pay attention as much. Um to the public markets and in particular right now it's Obviously valuations have gone up a lot, so There's a but because it's a long term question, maybe that doesn't a matter of the I think what I'd be thinking about right now is um

1:39:07 over the next twenty years, like what are the parts of the economy that will most benefit from this current wave of AI. That's not the only way to invest over a twenty-year period, but certainly it's a domain that I understand. And In particular, you know, I mentioned that uh talk I heard a snippet of from Tyler Cohen, which is like it will probably AI will probably benefit different parts of the economy um disproportionately. There will be some parts of the economy that can essentially Um

1:39:35 Where intelligence is a limiting factor to its growth and where you can absorb almost arbitrary levels of intelligence and generate almost arbitrary levels of growth. Obviously there's limits to all of this just because You change one part of the economy, it impacts other parts of the economy and that was what Uh

1:39:54 Uh. Tyler's point was uh in his talk. But I would probably think about that because I think that Over a twenty year period, there are certain parts of society that won't be able to change extremely rapidly. Um

1:40:07 But there will be some parts that probably will and it'll probably be domains where intelligence is a s is the scarce resource uh right now. And then I would probably try to find companies that will disproportionately benefit from it. And I assume this is why like NVIDIA stock is so high right now, because If you wanna sort of get downstream Yeah, NVIDIA will probably benefit from all of the investments in AI.

1:40:30 Um I'm not sure I would do that over a twenty year period, just assuming that the the infrastructure will shift. So I don't have an intelligent answer, but that's the way I would think about it if I were if we're doing that exercise. I love that the the Where do you think, like what's your intuition say about what areas of the economy are limited. By intelligence and not just economy, I mean perhaps politicians.

1:40:51 Uh might be limited by by this and and aid and benefit from, in which case countries could benefit enormously from AI and unlock growth and potential in their economy. But I I think maybe just to scope the question, like what areas of the economy do you think are limited by intelligence? Or Workers like smart workers, in which case like That's another limit of intelligence. Uh two that are I think

1:41:17 Probably going to benefit a lot our technology and finance, um, you know, where Your Yeah, if you can make better financial decisions than competitors you'll generate outsize returns. And that's why over the past, you know, thirty years,

1:41:33 you know, of machine learning, um, you know, uh hedge funds and financial service institutions, everything from fraud prevention to true investment strategies has already been an area of domain. uh domain of investment. Um software similar as we talked about, I think that Uh at some point we will be um we will no longer be supply constrained in software, but we're not anywhere close to it right now. And You're taking something that has always been the scarce resource, which is

1:42:03 software engineers and you're making it not scarce. And I think as a consequence If you just think of like how much can that industry grow, we we don't know. Um, but we've been so constrained on software engineering as a resource. Uh who knows over the next twenty years, but we'll find out uh where where the limits are. But to me, intellectually, there's just a ton of growth there. Um And then broadly, I think areas of like processing information are areas that will

1:42:30 really benefit um quite a bit here. And so That and I think the the thing that I would think about over a twenty year period is like second and third order effects, which is why I don't have an intelligent answer. And if you're asking me to put all my money in something, I would think about it for a while. Um probably use O One Pro a little bit to help me. Um but uh You know, because you can end up uh generated a bunch of growth in the short term, but then You know, if everyone does it or commoditize the whole industry, you know, type of thing. So you know, there it used to be

1:42:59 You know, before the in introduction of the freezer ice was like a really expensive thing and now it's free, you know. And so I I think it is really important to actually think through those if you're talking a time frame of like twenty years. And that's why I having not thought about this question ahead of time, I Um you could be quite simplistic elsewhere, but I would say software and finance are areas that I I think stand to reason should benefit. Quite a bit. I I love that response. How do you balance

1:43:24 Uh having a young family with also running a startup again. I work a lot. Um, I don't The

1:43:34 I really care and love care about and love working. Um, so One thing is that I Um well there's always trade offs in life. Um If I didn't love

1:43:45 work in. I I wouldn't do it as much as I do, but I I just love uh Love to create things and love to have an impact. And so I like jump out of bed in the morning and um Work out, go to work, and then spend time with my family. Probably probably you know being

1:44:02 Honest. First I'm not perfect at it, but second I don't have a ton of hobbies. You know, I basically work and spend time with my family. Um the first time we talked, you saw the couple of guitars in my background. I haven't picked one of those up in a while. Um the I I mean, I literally pick it up occasionally, but I, you know, do not devote any time into it. And I don't regret that either. Like I am so passionate about what we're building at Sierra. I'm so passionate about Open AI, I'm so

1:44:29 love my family so much. I don't really have any regrets about it. But I basically just like life is all about Where do you spend your time? And mine is at work and with family. And so that's how I do it. I don't know if I'm particularly balanced, but I don't strive to be either. I really Take a lot of pride and and I love I loved work. Having sold the companies you started twice. How does that influence what you think of, Sierra? Like are you thinking like, Oh, I'm building this in order to sell it? Or do you think differently, like this is my life's work?

1:44:59 I'm building this with That's not going to happen. I absolutely uh intends here to be a enduring company and an independent company. But to be honest Every entrepreneur with every company starts that way. And so

1:45:13 Um, you know, uh I'm really grateful for both Facebook and Salesforce for having acquired my previous companies and hopefully I had an impact about those companies. But You don't start off well, at least I never start off saying, Hey, I wanna Make a company to sell it. Um Uh

1:45:30 And uh but I actually think with Sarah we have just a ton of traction in the marketplace. Uh I really do think Sarah's a leader in helping consumer brands build customer facing AI agents, and I'm really proud of that. So I really see a path to that. And I joke with Clay. I wanna be, you know, an old man sitting on his porch, you know, complaining how the next generation of leaders at C don't listen to us anymore. You know, I want this to be something that

1:45:55 not only is Enduring, but outlives me. Um and I think Just actually uh I don't think we've ever talked about this, but it was really interesting um moment for me when Google went from its one building in Mountain View to its first corporate campus. It uh we moved into the Silicon Graphics campus, which was right over near Shoreline Boulevard and in Mountain View.

1:46:15 And uh SGI had been a really successful company enough to build a campus. And we when we it was actually quite awkward. We moved into like half the campus. They were still in half and they're like With this up and coming company, they're declining. And then when Facebook, when we moved out of the second building we were in Palo Alto, was slightly larger building, I think we released it from HP, but when we finally got a campus, it was from Sun Microsystems, who had gone through an Oracle acquisition and had been sort of on the decline.

1:46:45 And it was interesting to me because both SGI and Sun um had been started and grown to prominence in my lifetime. Uh obviously I was maybe like a little younger, obviously, but in my lifetime enough to build a whole corporate campus and then decline fast enough to sell that cor corporate campus to a new software company. And for me I I it was just so interesting to have done that twice to move into like a you know A used campus. Yeah, from the previous uh uh owners.

1:47:16 It was a very stark reminder that's technology companies aren't entitled to their future success. And I think we'll see this actually now with AI. AI, I think, will change the landscape of software to be um tools of productivity that to agents that actually accomplish tasks. And I think it will

1:47:35 Help some companies who for whom that's a uh amplifies their existing value proposition and it will really hurt others where it will essentially the seat based kind of model of legacy software will wane um very quickly and then really harm them. And so When I think about the what it means to build a company that's enduring, um That is a

1:47:57 really, really tall um task in my mind right now because it means not only making something that's financially enduring over the next 10 years, but setting up a culture where a company can actually evolve to meet the changing demands of uh society and technology at a when it's changing at a pace that is like unprecedented in history. So I think it's one of the most fun business challenges of all time. And I think it has as much to do with culture as it has to do with technology because Every line of code and Cay will be completely different, you know, probably five years from now, let alone thirty years from now. Um And uh I think that's really exciting. So when I think about it, I just get so much energy because

1:48:41 Um, it's incredibly hard and it's harder now than it's ever been. Um to do something that lasts beyond you. Um, but that I think is the ultimate measure of a company. You mentioned AI agents. How would you define that? What's an agent? I'll define it more broadly and then I'll tell you how we think about it at Sira, uh, which is a more narrow view of it. The word agent comes from agency and I think it means affording

1:49:05 a software the opportunity to reason and make decisions autonomously. Um And I think that's really all it means to me. And I think there's lots of different uh applications of it. The three categories that I think are meaningful. And I'll end with the the C one just so I can talk about it a little more. But one is personal agents. So I do think that most people will have

1:49:31 Probably one, but maybe a couple AI agents that they use on a daily basis that are uh essentially amplifying themselves as an individual. You can do the rote things like Help triage your email to Helping you schedule a vacation, you know, you're flying back to um Edmonton and help you arrange your travel.

1:49:54 Um, two more complex things, like, you know, I'm gonna go ask my boss for promotion, like help me role play and Um, you know, uh I'm setting up my resume for this job, help me do that to I'm applying for a new job, help me find companies I haven't thought of that I should be applying to. Uh And I think these agents will be Really powerful. I think it might be a really

1:50:15 hard product to build because When you think about all the different services and people you interact with every day, it's kind of everything. So it's not It has to generalize a lot to be useful to you and because of the personal privacy and things like that. It has to work really well for you to trust it. So I think it's gonna take a while to go I think it'll be a lot of demos. I think it'll take a while to be robust. The second category of agent is I would say um really filling a persona uh within a company. So

1:50:44 A coding agent, a paralegal agent, um A analyst agent. Um I think these already exist. I mentioned Cursor, there's a company called Harvey that makes a legal agent. I'm sure there's a bunch in the analyst space. Um these do a job and they're more narrow, um, but they

1:51:04 uh they're really commercially valuable because most companies hire people or consultants that do those things already. Like Analyze the contracts of your supply chain, right? That's a Kind of a rote. kind of law, but it's really important and yeah, I can do it really well. So I think that's why Uh this is the area of the economy that I think is really exciting and and as uh I'm really excited about all the startups in this space because you're essentially

1:51:30 Um, take in what used to be a combination of people and software and really making something that solves a problem. Uh And by narrowing the domain of of autonomy You can have more robust guardrails and even with current models actually achieve something that's effective enough to be commercially viable today. Um And uh

1:51:52 And by the way, it changes the total addressable market of these models too. Like I don't know what the total addressable market of legal software was three years ago, but it Couldn't have been that big. I I couldn't tell you like a legal software company. I probably should I just can't think of one. But if you think about the money we spend on lawyers That's a lot. And so you end up where

1:52:10 You're broadening the the addressable market quite a lot. The domain we're in, um I think is somewhat special, which is Um A company's branded customer facing agent. And the reason why I think it's

1:52:25 Uh one could argue we're sort of uh helping with customer service, which is a a a persona, a role, but I do think it's broader than that because if you think about um a website, you know, like your insurance company's website. Try to list all the things you can do on it. You can look up the stock quo. You can look up the management team. You can compare their insurance company to all their competitors. You can file a claim.

1:52:50 You can you know, uh buy you can bundle your home and auto. You can um Uh um add a member of your family to your premium. There's a million things you can do on it. Essentially over the past Thirty years.

1:53:05 websites, a company's website singular has come to be the universe of everything that you can do with that company. I like to think it was like the digital instantiation of the company. And that's what we're helping our customers do at Cyrus help them build a conversational AI that does all of that. So you know most of our customers start with customer service and it's a great application because no one likes to wait on hold and and having something that has perfect access to information is multilingual and empathetic is just amazing. But you know, when you put a conversational AI as your digital um front door, people will say anything they want to it. And um we're now doing Product discovery, consider purchases, going back to the insurance example. Hey, you know, I've got a fifteen year old daughter. I really am concerned about the cost of her premium until she grows up. Tell me

1:53:57 Um, which plan I should be on. Tell me why you're be better than your competitors. That's a really complex interaction, right? That's not something that Can you make a web page that does that? No, that's but that's a great conversation. And so We really w aspire that when you encounter a branded agent in the wild, we want Sierra to be the platform that powers it. And it's super important because

1:54:18 There was a case, at least in Canada, where an AI agent for Air Canada. Hallucinated a bereavement policy. Right. They were found liable. to hold themselves to what the agent said. Yeah, I mean it it turns out And it was an AI agent. There was no human involved in the whole thing. Well look, it's one thing of Chat GPT hallucinates something about your brand. It's another if

1:54:40 You're AI agent. Who said something about your brand. So The bar just gets higher. So the robustness of these agents, the guardrails, everything is more important when it's yours and it has your brand on it. And Um so It's harder, but I also I'm just so excited for it because

1:54:58 This is a little overly intellectual, but I really like the framing. If you think about a modern website or mobile app Um it's essentially You've created a um directory of functionality from which you can choose. But the The main uh person with agency in that is the the creator of the website. Like what are the universe of options that you can do? Yeah, when you have an AI agent representing your brand, the agency goes to the

1:55:23 the cons the customer. They can express their problem any way they want in a multifaceted way. And so it means that like your customer experience goes from the enumerated set of functionality you've decided to put on your website to whatever your customers ask. And then You know, y you can decide how to fulfill those requests or whether you want to. Yeah. Um but I think it would really change the dynamic to be really empowering to consumers. As you said, I mean the reason that that Air Canada case, um is the reason we exist. You know, companies if they try to build this themselves

1:55:57 Um, there is a lot of Ways you can shoot yourself in the foot. But in particular, um, too, your customer experience should be wedded should not be wedded to one model, let alone even this current generation of models. So with Sierra, you can define your customer experience once in a way that's abstracted from all of the technology. And it w can be a chat, it can be you can call you on the phone, it can be all of those things and As new models and new technology comes out.

1:56:26 our platform just gets better, but you're not like re implementing Your customer experience. And I think that's really important because Yeah, we were talking about what's happened over the past two years. Can you imagine if you're a consumer brand like ADT home security and thinking about like how can you maintain your AI agent in the face of all of that, right? It's just not even It's not tenable. I mean it's not what you do as ADT. So they worked with us to to build their AI agent. How do you

1:56:50 fend off complacency. Like a lot of these companies and And maybe not in tech specifically. But They get big. They get dominant.

1:57:01 And then they take their foot off the gas. And that opens the door to competitors. And there's like a natural entropy almost to bureaucracy in some of these companies that and the bureaucracy sows the seeds of of failure and competition. How do you how do you fend that off constantly? It is a really challenging thing to do at a company. Um One of the

1:57:24 There's two things that I've observed that I think manifest as corporate complacency. One is bureaucracy. Um and I think The root of bureaucracy is often uh when something goes wrong, companies introduce a process to fix it.

1:57:43 And over the like uh sequence of thirty years the layered some of all of those processes that were all creative for good reason, with good intentions. end up being a um bureaucratic sort of machine. Where um

1:58:02 the reasons for many of the rules and processes are are rarely even remembered by the organization. But it creates this sort of uh natural inertia. Um, sometimes that inertia can be good, you know, it's like, you know, if you end up with uh you you've there's definitely been stories of executives coming in and ready, fire aim new strategies that backfire massively. Um But often it can mean in the face of a technology shift or a new competitor, you just can't move fast enough to to address it.

1:58:31 The second thing that I think is more subtle is as a company grows in size, uh often its internal narrative can be stronger than the truth from customers. Um I remember one time uh when the sort of peak of the smartphone wars and end up visiting a friend on Microsoft's campus and

1:58:52 Uh I got off the plane in, you know, Seattle Tacoma Airport, drove into Radmond, went onto the campus. And all of a sudden everyone I saw was using Windows phones. Um I assume it must have been a requirement or formal or social, like you you were definitely Uncool if you're using anything else. And from my perspective at the time, like the war had already been lost. Yeah. Um

1:59:18 Like it was definitely a two horse race between uh Apple and and Google on on iOS and Android. And I remember sitting in the lobby waiting for my friend to get me from the security check in And uh I made a comment, like it wasn't a confrontation comment, but I made a comment to someone who's at Microsoft. I was like

1:59:37 Um You know S something along the lines, are you are you required to use Windows phones? How these other and I just sort of was like curio curious. And then I got a really bold answer, which is like, Yeah, we're gonna win. Like we're taking over the smartphone market. And I was like I didn't say anything'cause it was like a little socially awkward of like, no, you're not. Like you lost like four years ago. There's something there's a process there's something that's happening that's preventing you from getting reality. Well and that's the thing is if you think about it, if anyone if you've ever worked for like a large company

2:00:10 You know Your When you work at a small company, you care about your customers and your competitors, and you feel every bump in the road. When you're a you know junior vice president of whatever and you're Yeah, eight levels.

2:00:25 below your, you know, CEO And you have a set of objectives and results, uh Your you might be focused as I wanna go from junior vice president to senior vice president That's what success looks like for me. And you end up with this sort of myopic focus on This internal world in the same way.

2:00:45 your kids will focus on, you know, the social dynamics of their high school, not the world outside of it and it's it's probably rational, by the way,'cause like You know probably their social life is more determined by those, you know, one thousand kids in their high school than it is like all the things outside. But this is that's the life of a person inside of these big places. And so you end up where

2:01:05 Uh you know, if you have a very senior head of product is like our This competitor says they're faster, but this next version we're so much better. And then everyone says all of a sudden that's like W the Windows phone is gonna win. That's what everyone says. And And you truly believe it because everyone you meet

2:01:23 says the same thing and you end up reflecting you know, uh customer anecdotes through that lens, and you end up with the sort of reality distortion field manifested from the sum of of this sort of myopic storytelling that um exists within within companies. The What's interesting about that is like, you know, the ability for a culture to believe in something is actually a great strength of a culture, but it can lead to this as well. And so the combination of bureaucracy and inaccurate storytelling. Um, I think is the reason why companies sort of die. Uh and

2:01:59 And it's really remarkable to look at, you know, the blackberries of the world or the Tivos or the you know There you can really um you know, as the plane is crashing, like tell the story that you're not. And um and And and then similarly as I said, like culturally, you can still have c like the person in the back of that crashing plane being like, When am I gonna get promoted to SVP? Yeah, I and then you're like what the you know. Yeah. And and that's I mean, this is like I mean, I've seen it a hundred times. And so I think it really comes down to leadership, you know, and I think that

2:02:33 One of the things that most great companies have is they're obsessed with their customers. Um and I think the The free market doesn't lie. And so I think the one of the most important things I think for any like enduring culture, particularly in an industry that changes as rapidly as software, is How close are your employees to customers and How much can customer like the direct voice of your customers be a part of uh your decision making. Um and

2:02:59 That is something that I think you need to constantly work at because That you know, person employee number thirty thousand four hundred sixty two You know, how does he or she actually actually directly hear from customers? It's it's not actually a simple Is it direct? Is it filtered? How many filters are you?

2:03:20 Um I think the other part on leadership is you know, we talked about bureaucracy is process is there to serve the needs of the business and uh Often. uh mid level managers uh

2:03:35 don't get credit for removing process. They often are held accountable for things going wrong. Um, and I think it really takes top down leadership to uh you know, remove bureaucracy. Um and Uh it is not always comfortable, you know, when companies remove spans of control or uh

2:03:56 All the people impacted will It's like antibodies. Uh and for good reason. I mean, it makes sense. Their lives are negatively impacted or whatever it is. But it almost has to come from the top because you need to give uh air cover Uh almost certainly something will go wrong, by the way. I mean like processes usually exist for a reason. Um, but when they accumulate um without end, you end up with bureaucracy. So

2:04:20 Those are the two things that I always uh and you could smell it when you go into a really bureaucratic company. The The Inaccurate storytelling, the process over outcomes and it's just uh It sort of Sucks the energy out of you when you feel it.

2:04:36 That's a great answer. We always end these interviews with the exact same question, which is what is success for you? Success for me. We talked about how I spend my time with my family at work is, you know, having a Happy, healthy family and Being able to work with my co founder Clay for the rest of my life, making Saranta an enduring company. That would be a success for me.

2:05:03 Thanks for listening and learning with us. The Farnum Street blog is where you can learn more about my new book, Clear Thinking. turning ordinary moments into extraordinary results. It's a transformative guide that hands you the tools to master your fate. Sharpen your decision making. and set yourself up for unparalleled success.

2:05:22 Learn more at fs.blog slash clear. Until next time.

What you see above is a preview of the first minutes. One unlock costs 10 credits and covers this episode forever: full segment and word-level timestamps on this page, plus .txt, .srt, .vtt and word-level JSON downloads, as many times as you like.