Bret Taylor - The Agent Era - [Invest Like the Best, EP.386]

0:00 I know firsthand how complex the tech stack is for asset management firms. And seemingly every new tool and data source makes the problem even worse, adding more complexity, more headcount, and more risk. Ridge line offers a better way forward, one unified platform that automates away the complexity across portfolio accounting. Reconciliation, reporting, trading, compliance, and more, all at scale. Ridge line is revolutionizing investment management, helping ambitious firms scale faster.

0:25 Operate smarter and stay ahead of the curve. See what Ridgeline can unlock for your firm. Schedule a demo at ridgeline.ai. Hello and welcome everyone. I'm Patrick O'Shaughnessy and this is Invest Like the Best. This show is an open ended exploration of markets, ideas, stories, and strategies that will help you better invest both your time and your money. Invest Like the Best is part of the Colossus family of podcasts, and you can access all our podcasts, including edited transcripts, show notes, and other resources to keep learning at joincolosis.com.

1:00 Patrick O'Shaughnessy is the CEO of Positive Sum. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of Positive Some. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of Positive Sum may maintain positions in the securities discussed in this podcast. To learn more, visit psum.vc.

1:29 My guest today is Brett Taylor. Brett's resume is Absurd. He built Google Maps, famously rewriting the whole thing in a weekend. He was the CTO of Facebook in critical years. He founded Quip.

1:42 He was the chair of the board at Twitter. He was the co CEO of Salesforce. The incredible list goes on. Now Brett is the co-founder of Sierra, a conversational AI platform for business. And he is the chairman of the board at OpenAI. Together we discuss the past, present, and future of AI agents.

1:59 New programs that will begin doing incredible amounts of work for us humans in astonishing ways that are a thrill to consider. Brad believes agents will become a meaningful part of the future and transform the ways in which we interact with technology. We discuss a strategic approach to AI integration, the different categories of agents and their scopes. And the essentials of craftsmanship. This is a must listen with one of the great builders of our time.

2:22 Please enjoy this discussion with Brett Taylor. Okay, Brett, so I thought of an unusual place to start with a very strange topic that probably most people won't know about, but I promise it will set the stage for many of my questions. Can you explain the concept of the mythical man moth? So it's called Mythical Man Month. And It's essentially the myth that I think stereotypically a lot of

2:48 folks who come from well say a business school background as opposed to computer science background have about software, which is to make a software development project go faster, you should add more people to it. And often the inverse is true. In fact, there's some proverbial statement it's if you want to make a software project go twice as Slowly add another person to it. I think it's that truism I think is based on I think when you're designing a very complex system It's often individual engineers who have a system

3:20 Mapped out in their head. But It's very intricate. And similarly when you have really high functioning teams, whether it's early teams I worked on like Google Maps or Some of the folks designing some of these very powerful AI systems right now, small teams who are completing each other's sentences, so to speak, and own parts of the system and really understand it and work. In unison is it.

3:43 Very magical and Fragile Team dynamic and adding more people. often requires you add a lot more process, you add a lot more bureaucracy, you disempower Some of the most advanced technologists on the team.

3:56 And can broadly speaking just slow things down. And so whether you look at projects like healthcare dot gov that had a billion people working on it and went very slowly, or Just the classic software projects at most companies that take twice as long and cost twice as much. That's the idea of the mythical man month, and I would say it's broadly true though, like all Rules of thumb. It's just a rule of thumb. And every situation's different.

4:20 And do you think that the primary reason that it's an interesting heuristic is this idea that you need to have a systems understanding or an architects almost understanding of a system to be able to move really fast to change it or build it. Is that the thing behind the thing? that fewer people are able to keep that model in their head. And if you start to distribute that, you lose that benefit of speed. I think it's a few things. So

4:44 If you have a small, accountable, empowered team Often it means they understand the customer problem they're solving precisely. I. You're not just implement in a piece of code. You're actually

4:57 Delivering a solution to a problem. And I think when you have hundreds of people Working on a project often the Part of the problem that you're staring at is so narrow. Often you lose sight of the forest for the trees. And that's part of it.

5:12 I think similarly When you have lots of people working on a project, it requires a lot of upfront planning about how you might architect the solution just to accommodate the hundred plus people. Imagine. Let's say you're cooking dinner. And you have a hundred people in the kitchen and you need to put them all to work. You're slicing the carrots. You're the carrot slicer. That's your job.

5:32 But probably you have no idea what the meal is that you're producing. But in particular if you get in the middle of that dinner production and you Realize you're doing it wrong. Making changes is quite difficult as well. And so I think that

5:46 You have this combination of individuals on the team having a very narrow view of what they're doing and why, which is Problemat for a variety of obvious reasons. You end up with less agility because of just the natural overhead of coordinating between all those people and the act of needing to coordinate requires so much upfront planning that you end up with a lack of agility as well when you encounter new information, either Maybe it's not working as well as you thought technically, maybe a new customer requirement comes in. Probably a combination of all those things and just the natural bureaucracy that comes with

6:18 larger teams, you need product managers and engineering managers and this and that and the other thing. Whereas if you have what I think Amazon called the two pizza box team You reduce the need for a lot of that. Overhead. And you get a lot of those other benefits. We're hopefully in a smaller team

6:34 Most people understand why they're doing what they're doing, not just what they're doing, which is quite empowering and important because it enables A sophisticated engineer to find the shortcuts that He or she might not know are okay to take without knowing the actual solution that they're delivering.

6:50 And more importantly, just enables agility. And I think that Most big systems. Engineering is the act of taking science and applying it. To reality.

7:00 And So much of engineering is that moment when your system meets reality, whether it's a rocket ship or a piece of software. And you really need Accountable, high frequency decision making to make that successful. Just to nail home this point, because I think agents, one of the things they'll do probably is enhance the power of the most powerful engineers.

7:21 Maybe by scary amounts. I don't ask you about that. But just to give the people in the audience a sense of one of these stories in situ. Everyone I ask about you brings up the Google Maps legendary rewrite story early in your career. And It just seems like such a fun opportunity to ask you to tell us the long version of that story. Bring us into the room the context around what happened, what you did.

7:44 But then Explain literally what was going on. What made it possible to do something at that scale so quickly with such a small group. Yeah, I'll start with where Google Maps came from and then Talk about that that rewrite story, which is I don't know how that made the rounds on social media, but that's it was pretty funny. Google Maps was a small team.

8:04 The engineering leaders were Lars and Jens Rasmussen two Danish brothers who had started a company called Where Two Technologies that Google had acquired That had built a Windows app called Expedition.

8:17 Which was a really cool installable Windows app a little bit. Like cool horizontal. But rather than being three D mapping, it was Three D

8:25 Imagery. It was just mapping. In the process of onboarding them to Google. We had made prototypes of taking that expedition product experience and making it work in a web browser. And at the time it was mind blowingly hard.

8:39 Just for context, I'm not sure how many listeners really alive or using the internet in two thousand four, two thousand five, but most web pages were static web pages. Google search results for ten blue links. probably the most advanced web applications at the time, what was probably Gmail, which had launched in the middle of two thousand four. And a couple of products like Google Suggest, which would auto complete as you typed. And that was the most advanced thing you had. In trying to take this Windows application and making it work in a web browser, you could actually drag the map. Click your mouse and drag left and see the map move. Which seems so simple now, but it's the first time you really had anything that interactive.

9:18 Inside of a web browser. The lot of the engineering of Google Maps was encountering the contours of what was possible in a web browser. We would crash Firefox and Internet Explorer like it was going out of style. But luckily a number of members of the Firefox team had actually started working at Google at the time, so we could walk upstairs in building forty three, I think it was, and be like Can you load Firefox in the debugger and tell us what the hell is going on here? And we'd figure out what's going on and couldn't go back in time and pass the browser, but they'd figure out like what was going on so we could work around it.

9:51 So by the time We had launched Google Maps. And then subsequently added satellite imagery. to it. We launched in February of two thousand five and added satellite imagery in August, if I'm remembering correctly. It was just the code base was this accumulation of hard one

10:07 Lessons almost like you had started with a house that was just like a living room and added bedrooms and kitchens and different remodel stages. And each one of them was just banging their head against the wall and trying to figure out this technical problem and like this Code base had turned into this Total mass.

10:23 And it was in part because we had no idea what we were doing at any stage of the project because we're literally doing research about how to actually make all of this work properly. It sort of reached a breaking point where we were trying to get at least this is my memory, it might be embellished through The lens of Bad memory.

10:41 And history. But we were trying to get it work well in Safari, which was at that point a nascent web browser Safari has become more important because of mobile phones, but it was just Uh. New browser on the Mac.

10:53 Half the hacks that we had just weren't working properly there and Yeah. messiness of that code base and all of the lessons that it was over using XML, which was super trendy at the time. Good riddance. I don't think anyone would use it for most web things today.

11:09 It was very clear that we were Build in on a really good thing. Cappy Foundation. I ended up with the benefit of having experienced all the hard learned lessons from the whatever six months or so we had been working on it.

11:21 I just rewrote it. And I rewrote it With all the knowledge that we had accumulated. I think the story's a little bit overblown. I did rewrite it. But it was based on all the lessons I learned. So I was just like I took all the lessons that the team had collectively learned in that time

11:37 And I basically said if I knew all this at the beginning Here's how I would architect the system. And I think I would have the benefit of having been a part of all the mistakes that we had made up to that point and the benefit of those lessons and I think

11:52 Had the luxury of Starting from a blank sheet of paper with all those lessons and The main goal at the time was to get the bundle size, which is the size of the JavaScript your browser would have to download before Showing a map down, we got it down to like twenty K uncompressed, which is pretty impressive in this day and age. And in particular got it working really reliably on all these different browsers and

12:13 So it's mainly just trying to get the product to grow more rapidly. But it was Largely because we you just don't know what you don't know at the beginning of a project. And going back to your original question on the mythical man month. If you have to go through a committee to talk about rethinking something, you'll never get it done. When you had this small team who

12:31 With a really clear objective and some empowered people in this case I was Pretty empowered. You can do that. It's such an interesting story because I think it becomes ultra relevant in today's world. And you hear a lot about this, maybe the mythical ten X engineer, hundred X engineer. thousand X engineer the leverage available to one person.

12:51 With a growing toolkit. And maybe that's a great excuse to bridge the conversation into agents. I think everyone listening will have heard that term. maybe have thought about it a little bit, have gotten excited about the prospect of some sort of autonomous agent doing work on their behalf or their company's behalf. But it would be great for you to ground us in

13:11 your definition of what one of these things is. If this becomes a really critical part of the world of technology in the next year or two. I think it'd be great for everyone just to have a level set, simple definition from your perspective on what an agent is and does. I'll start with Maybe the academic

13:27 flavor of this, but then I'll move into what I think is maybe the more what I believe is the more relevant. definition, but agent is like the word app. There's not one definition and I think it will be a noun that is quite meaningful in the age of AI. The word agent in the context of AI comes from the word agency and essentially

13:45 Is A system that can reason and take action autonomously is the way I think about it. And A system that is agentic. There's one where Software and AI

13:56 Can reason and make decisions and take action without Human intervention. Which is really exciting. It's something that is relatively new. Though the idea is certainly not new. I think the effectiveness of reasoning with AI systems has become so meaningfully better over the past couple years that I think the concept is

14:14 Like many parts of AI, the ideas are not new, but the effectiveness is. And so we're living in an era of agents now. In practice, I think The word agent, just like the word app or site in the age of the web will become Important to all of us. So

14:29 one agent that I think is important is What my company Sierra does, which is Your company's conversational AI. And so Just imagine you're a retailer.

14:40 I think you'll put as much care and attention into your AI agent as you do your website or your mobile app, or if you're a bank. And you'll put as much care and attention to your AI agent, which can help a customer look up the balance of their checking account or perhaps be an interface to your investment banking arm or

14:59 Wealth management arm. Or if you're a Streaming service your agent might Hel people sign up for a plan or upgrade or downgrade their subscription as an example. In that case, an agent is something like website or mobile app that's branded and it's yours. And there are parts of it that are about

15:17 agency and sort of the AI definition of the word, but more importantly, it's your thing. It's your digital asset. It becomes a digital Manifestation of your brand. But that's what my company Sierra does, and we think that's one really important part of Asia, just like in nineteen ninety five. The way you existed online was to have a website. We think in twenty twenty five.

15:35 The way you will engage with your customers will be your AI agent and we think it's a really important new category. But then taking okay, what are the other types of agents out there? One will be I like to think of them as well. persona based agents that are internally facing they do a job. You've talked about software engineering. I think there'll be a software engineering agent that will

15:55 work to produce software. I was looking at A startup called Harvey, I think, that's making a legal which is super interesting. And I think across many job functions there will be AI agents that produce the output of a Whether it's a paralegal or a software engineer.

16:14 Or an operations analyst, things like that. So that's one. So there's your company's agent. There's A persona based agent that does a job. And then the third one category I think is probably personal agents. So this is the agent that will work on your behalf.

16:28 Whether it's Helping you Plan a vacation or Organize your calendar or perhaps Triage or in box.

16:35 And things like that. I think technically they're all similar, but my guess is They're different enough in what job they accomplish for you that there's probably different companies will build those different categories of agent. If you're building a software to be a personal assistant agent. The breadth of systems you have to integrate with is

16:54 Infinite'cause different people use different calendars and different this and different that. And there's lots of interesting investment into that. If you're building a coding agent, it's a much more narrow use case, but very deep. And you're probably evaluating it based on benchmarks of the effectiveness of The software produced and the robustness of the software it produces and then for Sierra, where we're helping companies build agents. It's about customer experience more than anything else.

17:18 It's everything from how does it become a brand ambassador, how does it follow the guidelines of your different procedures, how do you actually take all these different facets of your company, whether it's sales or service or whatever's on your website will probably be in your AI agent. How do you actually operationally Get that. Manifest that as a conversational experience.

17:37 Interesting thought exercise would be across those three categories. If we think about One dot oh which I'll define as what's possible today with current technologies. Two dot oh, which is maybe the next generation of LLMs or something from the major providers that come out. and others. And three dot oh is more distant future

17:54 Can imagine it, but not yet technically feasible or something. How would you discuss agents at those three levels of development from your perspective? My sense is that progress and AI Will be fairly iterative. With occasional jumps and capabilities. Certainly with the original advent of Chat G P T and

18:15 Instruction tune in. A large language model, that was a big jump. I think GPT four. represented a similarly big jump. But I think in between we'll see lots of iterative improvements to these models. So

18:28 Burst, I think that I think of it more as a gradient. Then I do steps. I might be wrong. It's very hard to predict the future right now, but I I do think of it that way. And I think if it if I think about the how to responsibly

18:42 deploy AI. It's responsible iterative deployment. But every step of the way. How do the creators and users of AI recognize its limitations. How do you build in the safeguards recognizing that those limitations are real?

18:57 And How does that change over time? And with that in mind, how do agents change? More agency. More autonomy. Over time so that

19:06 Right now, I think. Whether it's hallucinations or jail breaks or all the different ways AI models Can go Hurry.

19:15 They're more frequent. with current models. And as a consequence, do you Probably if you're doing this responsibly, don't want as much agency, right? You want more Smaller, more bite sized tasks. You look at the way.

19:27 Software developers are using coding agents. Today it is that. It's maybe write a function, not write a program. I mean a little bit reductive. That's the way I think about it. And if you progress forward and you have Better reasoning capabilities, better tool use. You can turn up the knob on agency responsibly. And then similarly, I think as you think about Safety in AI.

19:46 As these models become more advanced. You want more advanced guard rails. As well. And especially once AI can access tools or access the internet, the surface area of opportunities to things go wrong increases well. Particularly as the models and agree if there's

20:02 uh tension there, right? As the models get better, you can give the AI more agency to complete tasks. But I think the level of sophistication of the safety issues that you have to Candle also. Get more broad and complex as well. So my sense is over time they'll be more powerful and have more agency effectively.

20:21 And be more complete in the jobs that they accomplish on our behalf. And that would have to be command so it was improved guardrails and safety recognizing Uh. As you provide an AI more agency, the surface area of of things that you'll need to protect against get broader as well.

20:36 What do you think are the next most important unlocks for the power of these agents? You mentioned their access tools, access the internet. I've heard people talk about the ability to have some sort of stored memory about you, the customer, the specific customer, or just memory in general. That doesn't just live inside of a context window that's always refed in or something. Are those the three things that we need to unlock the next tier of productivity out of agents? Are there other things that you and Sierra are focused on? I'd love to get down to the nitty gritty capabilities and roadblocks that you're thinking about and working on that might make these things as ubiquitous as you think they will be. Yeah, I'll start with the vantage point of Sierra. We help companies build customer facing AI agents. Today if you're setting up A new Sonos speaker, you can chat with an AI agent they've built on our platform to help you set it up.

21:22 Yeah. You're a serious XM subscriber, you can chat with Harmony. Which is their AI agent they built on our platform. And if you're A Weight Watchers member, if you click on the twenty four seven live coaching tab in their app, that's an AI agent. They built on our platform.

21:37 One of the things that I think is a nuanced problem that is not strictly technical in nature is Just the act of actually designing conversational customer experiences is a relatively new discipline. I remember in the early days of the internet, most websites Looked.

21:54 Like D V D. Intro screens like they're very graphical. There's four big buttons. It's really interesting to go on the Wayback Machine and look at them. And I would say it took a number of years to evolve into sort of the design idioms that we recognize with websites today. And now if you go to a retailer, they'll have a hamburger menu in the top left and

22:15 The way you filter through items and There's They're sort of emergent from people's lived experiences both designing and using websites. And now you can talk to Almost any web developer and they're not only two similar technologies to make a website, but even the design process and

22:32 Photoshop or Figma to design a website. They're sort of established practices, some of which are obvious and some of which are actually subtle. Like why did this become the way These things are done and It's the cumulative experience we have. Building with them. Mm.

22:49 website or and a mobile app and an AI agent. is both the breath and non determinism. Of AI agents. So If you have a menu on a website

22:59 You can control how what links are there and it's essentially multiple choice. Here's the options available to you. If you have an AI agent With a freeform text box, people can type whatever they want into that. And so Your concept of what your customer experience

23:14 Is defined by you. But it's also defined by your customers by what they write in there. It reminds me of going back to my web uh analogies here reminds me of going from like Yahoo directory to Google search rather than having a taxonomy Of everything available. It's just freeform and

23:31 There's a much longer tale of queries in Google than there was. In Yahoo because of the expressiveness of a search box versus a Directory. And I think that that's one of the really interesting and I think exciting opportunities with conversational AI for customer experiences. Is it's a really

23:48 authentic way to actually hear from your customers what they want from you. And I think we've and is uh sort of stands to reason. Your website was the rails on which your customers communicate with you and this is so free for them that I think it's much more expressive and we've had Multiple customers learn

24:04 things about their customers that they didn't expect by providing this really freeform experience. And then similarly, I think the other really interesting thing when I mentioned non determinism is the word agent comes from agency. And it's really How much creativity do you want to give your AI in interacting with your customers? I think

24:22 If you start from a position of control, you can say I want to put guardrails around everything. But then you're conversational customer experience is somewhat robotic. You've essentially defined the multiple choice options of Your customers experience.

24:37 If you give your agent too much agency. In the extreme case it will hallucinate. But in the more practical case, it just might not protect your brand in the way that you want it to. And I would say that design question is both a technology question, the which obviously we're quite invested in solving and I'm really excited about some of the work we've done there. But there's a deeper question here, too. It's actually a philosophical Branding and design question as well.

25:01 And what we're trying to do at Sierra is Not necessarily predefine the answers to those questions. I think every company and every brand will have a different perspective on what's correct for their brand experience. But provide a platform that's powerful and expressive enough, whatever your answers are personally to that question, you can build your agent on Sierra.

25:19 It's so interesting to think about The customer experience going to a website where I buy shoes or something. I think one of your first customers was flip flops and there's a funny story around that. But I'm going to buy a pair of sandals, let's say, on a website. And rather than click around, I just describe what I want and I can imagine like another pane on the right just starts like showing me stuff. And then maybe I check out through this same thing as well. And that's a simple version of tooling or ability to take action. I'm curious what the hardest parts for you have been to build.

25:45 It's quite technically daunting to even think about. how to build something like this, let alone one that's adjustable and tunable to my specific brand. So talk a little bit about how hard Of a technical challenge. this is for Sierra. Like the degree of difficulty you've encountered relative to say your expectation.

26:02 Yeah, it's a really wonderful question. I think that's the thing. Generative AI broadly. Is it technology? with which it's very easy to make a demo and very hard to make an industrial grade system. And I think that's the area of technical challenge that we're really trying to dive into.

26:20 And I think it's one thing to say this system does the correct thing ninety percent of the time. And it's really an ink blot test whether ninety percent is a really good number or a horrible number. And it also depends on the process. And so If it's a consumer application that was Helping you with your homework maybe ninety percent is decent.

26:38 Yeah, it's something. operating revenue impacting part of your business, or there's a compliance concern, it's absolutely unacceptable to be wrong. Ten percent of the time. And so a lot of the challenges that we're facing are We like to say that

26:51 Software systems are moving from rule based to goals and guardrails based. And it's a very different mental model for building software systems. Rule based systems, if you think about just the software development lifecycle that's evolved over the past. For twenty years.

27:07 It's really about how do you make more and more robust rule based systems. How do you ensure that the same input produces the same output, that it's reliable, that it's stable, and that a lot of like true innovation in the way we make software. To make them more secure and robust. Now if you have parts of your system that are built on large language models. Those parts are

27:28 really different than most of the software that we've built on in the past. Number one is They're relatively slow. Compared to generate a page view on a website takes Nanoseconds at this point.

27:39 Might be slightly exaggerating down the milliseconds. Even with the fastest models. It's quite slow and The way tokens are emitted. Number two is it can be relatively expensive. And again, it really varies based on the number of parameters.

27:53 And the model, but again, the marginal cost of that. Page view is almost zero at this point. You don't think about it. Your cost as a software platform is almost exclusively in your headcount. With AI, you can see the the margin pressure that, you know, a lot of companies face, particularly of their training models or even doing inference with high parameter count models. Number three is they're non deterministic, fundamentally, and you can tune certain models to more reliably

28:19 Have the same output for the same input, but by and large It's hard to reproduce behaviors on these systems, what gives them Creativity also leads to non determinism. And

28:31 So this combination of it's you've gone from S Deterministic. reliable systems to relatively slow, relatively expensive, but very creative systems. And I think it violates

28:43 a lot of the conventions that software engineers think about Have grown to think about when producing software. And it becomes almost a statistical problem. Rather than just a methodological problem.

28:54 And so that's really what we're doing. We've tried to solve we shared on our website, but we've have a process we call the agent development lifecycle, which is the name comes from saying the software development lifecycle. Here's what you should do with these agentic platforms. It's also we've developed a lot of unique technology to Make these systems more robust. Wait. Having one AI model supervise another AI model to

29:17 layer different models on top of each other to produce statistically more robust results. And then as importantly, we've developed ways that Folks who aren't experts in AI can express the behavior that they want in their agent. You shouldn't have to be an AI expert. to make an agent just like you shouldn't have to have a PhD in computer science to make a website. And I don't think we're there yet, but that's really what we're trying to solve and broadly speaking, I would say On the spectrum of

29:43 fundamental research institutions like OpenAI were not that were applied. We're really thinking about How do we engineer on top of these foundation and frontier models to produce robust and reliable agents for our customers. I love the title of this one Kevin Kelly book, What Technology Wants, and I'm curious what agents want.

30:03 If I'm a customer, I'm a prospective customer and I want to go work with Sierra To make the best possible version of a conversational agent for my customers to use. What can the companies provide that make the agent do the best job? Yeah, it's a great question. I would say that there's two types of knowledge that I think really produce a really robust agent.

30:24 One is the factual knowledge of your company. This just browns the agent. So that it won't just make something up. There's a Pretty widely used technique.

30:34 called retrieval augmented generation in AI right now, that effectively means rather than relying on the knowledge encoded in the model to answer questions. You present the model with knowledge maybe stored in a knowledge base or a database and Say Hey, summarize the content from here.

30:51 Don't rely on the three. On the information you've been trained on. That has been an effective technique for two reasons. One is that

30:59 It means that You don't necessarily need to train or fine tune a model to use it with proprietary Data, which is a much cheaper deployment methodology. And it also can be effective at preventing hallucinations as well, because you're effectively rather than relying on the AI to Determine what it knows or doesn't know, you present the AI with the knowledge that it's allowed to know. Simple way of putting it.

31:22 And that's factual knowledge. I would say that's Necessary, but woefully incomplete. Because that would enable your AI agent to answer questions, but it wouldn't necessarily enable it to Orchestrate a complex process or take action.

31:36 On your customers' behalf. The other type of knowledge is procedural knowledge. Sonos speaker stops working. What would the best Sonos engineer ask you and do to figure out whether it's a problem with your hardware. problem with your Sonos app, a problem with your Wi Fi.

31:52 Like what is the process by which You do that. If you're a subscription streaming service. What is the process of processing a upgrade or downgrade to your membership? Are there different offers available based on

32:05 Your membership level. Do you have a promotion run in? What's been the most effective technique to keep people a subscriber for a long period of time. This is all the stuff that if you were a person, an expert in it. And so coming with that knowledge of not only here's the factual knowledge for our company, but here's the processes that represent our greatest customer experiences. What does the best salesperson do? What does the best customer service person do? What is the most effective marketer at your company, how do they describe your products? And that's often there we work. with our customers to improve when they deploy AI.

32:36 And then the third thing is just access to the underlying systems themselves. I think the AI agents shouldn't just be about answering questions or having a conversation. They should actually be able to take action on your behalf. So that's a retailer processing a return or A subscription service changing your level of membership. Or connecting to the telemetry system of a consumer electronics company so we can say, Hey, we know your device phone home, you're connected, we now figure out this other problem.

33:02 Or even with something like Series X M sending a Signal down from a satellite to refresh your radio if your radio stopped working. So those three ingredients factual knowledge, procedural knowledge, and systems integrations, I think are the three T ingredients and then With

33:17 the right methodology. Your agent can do anything that a person can do on a computer, which is just an incredible opportunity for customer experiences. How long does one of these things take to build? I imagine there could be quite a bit of upfront work. Last time we talked, you described this interesting process. I think you called it spring cleaning that tends to happen probably around defining process and best practices within a company. And obviously it's super early days of this, so I'm sure your answer will get shorter and shorter as we get better and better at this. But just today. If you meet a new customer that wants to install one of these agents on their website or app or whatever, how long does it take Start to finish. for a new customer to get live.

33:54 Between one and three months. We have a model where we Have a really hold the hands of all of our customers so that they don't need to be AI experts or experts and agents at all. To get started, which is

34:06 Pretty unique. I think it's a new enough technology that The last thing I always joke. But you can build a couch by going to Home Depot, but you need to be a carpenter. Or you can go to IKEA and all you need is an Allen wrench. When these technologies are new, we really felt like it was important that any company, no matter how many resources they have available, can deploy them. So we have a really high touch model

34:28 So that we can get customers live as quickly as possible and I'm really happy with it. How rapidly we can do it. Can you tell that flip flop story from I think one of your first customers? Yeah, one of our first customers.

34:39 Was a Shoe company and We had built the agent to handle common retail use cases to my memory I might be misremembering here is things like Where's my order?

34:49 Things like that that we expected. Uh customer actually sent us one of the first conversations and it was Again, my memory might be a little off here, but I'm going to a wedding in Hawaii.

35:00 What sandals will go with my bridesmaid dress? And It was one of those things at the time they was like, I can't really tell you that tell me to transfer you to someone who can and It was really interesting to me because it illustrated the point on expressiveness that we spoke about earlier. Which is

35:16 Whatever your conception is Uh. What your agent will be used for. Because it's free form, your customers will express themselves completely freely. And as a consequence, you might think, I'm gonna launch this. agent to handle customer service, which is a great initial application, but you might find your customers are pulling you in a different direction.

35:36 But I think that's remarkable. And so many of our retail customers are doing things like product recommendations and just think about the best experience you've had at a brick and mortar retail store. If you have that associate who's Understanding why you're in the store, what you're looking for. They're not too pushy, but they are really trying to listen and understand what you're trying to do. Maybe they're

35:57 Offering you uh products you didn't think you needed. Maybe they're steering you to the right. section in the store, that sort of consultative experience. What's so neat about conversational AI is you can scale that. One of the things that I think is really interesting just from an economics perspective is Technology fundamentally drives productivity in the economy, if you're an economist. And what I really like to think about for new technologies is

36:21 What? thing used to be extremely tedious and expensive that is now really Sh. And effective. And in this case.

36:29 If you have two million customers. too if you actually think of the cost of having direct conversations with all of them, it's tremendously expensive. Usually that involves building a contact center either in house or with a BPO. an outsourced firm. It probably means that you're

36:47 Overstaffing it to handle peak. Seasons. Yeah, it involves huge amounts of training for the workforce in that call center. So every time you have a new product or a new process you need to train everybody. And it becomes a huge cost for your company. And that's why so many as consumers

37:04 It's usually pretty hard to have a direct conversation with the brands that we work with every day. And it's not because they necessarily don't want to, it's just that economically it's really hard to make all of that work and justify that expense. Yeah, and it really depends on the margins of the business and the value you as a customer are providing, like whether you can get on the phone. With AI.

37:24 We have this opportunity to bring down the cost of a conversation down to something that's fairly marginal. Think it's really exciting. We'll be able to actually have a direct personal conversation with so many consumer brands in a way that just wasn't possible before. And I think that's Much more than automating what is currently maybe a customer service interaction. I think it's bigger than that because I think brands can have a much bolder vision for

37:48 One of the conversations we want to have with our customers and now that the cost is tolerable What are the areas now that we don't need to treat this as this cost center that we're trying to minimize and we can treat this as a incredibly impactful way to interact digitally with our customers. Where are we gonna have conversations in ways that just weren't possible before? I could imagine today, tomorrow, hitting a button that says Here's a ton of context about me so that you can do a better job serving me the right product or service or Come to a faster, more efficient outcome, or whatever. When do you think that happens? That there's more information about each individual person being served.

38:23 'Cause it seems like that's the other part of this equation that would unlock magical, incredible experiences. Like the best salesperson would know everything about me. They would be much better suited to serve me then. And so That seems like a piece that it's not on my radar anyway. I'm curious if it's on your radar. I think we're To some degree.

38:40 Already there. But I Just to reinforce your point. I always like to start with our experiences as consumers of these technologies and

38:50 First, we all speak different languages. Second, we're all as you said, our relationship with different consumer brands might be in different states. And today with AI You can look up information about the customer, you can tailor the experience to them, you can speak in the language that the customer speaks in a multilingual way. With a single AI agent.

39:11 Maybe someone wants to have a long conversation, maybe someone needs to get in and out very quickly, the agent will reflect to that urgency or reflect the tenor of that conversation. Again going back to my Point on the cost of having a conversation, there's no need to get off the phone. You can have as long of a conversation as the customer wants and There's not a

39:29 operational reason why You need to rush someone through that conversation. I just think of empathy. Through this. And

39:38 Similarly, I think One of the things that's really also exciting about AI Is you can take the best experiences you've provided to your customer, whether it's a great sales experience or Great marketing conversation.

39:51 And you can scale it. I think that's If you just imagine retraining your workforce on a methodology that it's a slow and expensive enough process, not only is it hard to do, you don't want to do it frequently. Now with AI if you discover a technique that works either intuitively or through some measured automation, you can actually deploy it instantly. It actually enables not only a degree of empathy and personalization that was not possible before, but as a leader of a company and a brand ambassador, you can try things more quickly.

40:24 When you find a magical moment you can operationalize it. There's a great Book. It's called unreasonable. Hospitality. It was a really interesting

40:33 Those anecdotes. Of people having these really delightful experiences in this Three star Michelin restaurant. They seemed like these one off moments.

40:44 And a lot of his vision of that book is how do you operationalize that and repeat it? And I really do believe that AI is an opportunity to do this for all of your customers and say you can scale things and have them be personalized at the same time. And I think that is only possible with AI, and I think it's a really meaningful opportunity for brands who lean into this technology. What do you think about the future interplay between the three kinds of agents you talked about before? I can imagine a world where I have a personal agent and I send them to do the shopping. And most of the interaction between a company's Sierra powered agent is with my personal agent or something crazy like that. Turtles all the way down.

41:21 Yeah, turtles all the way down. This all seems incredible. For like the first order reasons you've described, but it also seems like it's gonna change the way we do stuff in unpredictable ways and you have the front row seat here. And so I'm just curious, yeah, how you think like the interplay of this is gonna

41:36 Change companies, change consumer behavior. Just like the bigger macro questions are fascinating to me. It is a really great question, and I'll start With the humility that it's very hard to predict the future right now. In two thousand seven, if you watch Steve Jobs introduce the iPhone. How much would have you predicted correctly about its impact on society?

41:57 What's interesting about that moment too is the intersection of technologies I social networks existed in two thousand seven. And one could argue that we didn't really fully realize the societal implications of social media until you've had the smartphone and social media and push notifications. And they all add up to

42:15 A lot of the issues that I as a parent and we as society Grapple with now. So you have both the first order impact of these technologies and the second and third order Which are

42:26 Very hard to predict. Unlike the smartphone, I think the impact of AI agents will be fairly broad and there'll be some counterintuitive implications that I'm excited to experience, but I'm not sure I can fully predict. So that huge cat. I do think that personal agents will interact with company agents.

42:45 Clay, my co founder, Clay Before and I have a bet. I won't tell you the dates. But we have an internal bet about win the conversational traffic to our company agents. Will be more

42:57 Personal agents than people. Like where are we? Cross that line. But the fact that Clay and I have talked about it I think reflects the depth of our conviction that This will be

43:07 A meaningful part. Oh. The future of how we interact with computers. Similarly, one of the things that I am really interested in seeing is how we interact with the devices around us. I thought

43:19 Alexa we have Alexa's on our countertops at home as a lot of people do. Siri was a huge part of the Apple Experience. I would say neither platform really transforms The way I use my phone. Because I think the limitations

43:34 of those platforms which pre dated sort of modern large language models. It will be interesting to see How all the microphones around us turn into more full featured experiences with the prevalence of conversational AI. And then similarly if you look at Countertop speakers, car play.

43:51 Things like that. How full featured can those experiences get when most of the software we interact with has fulversational capabilities. Similarly if you've ever used Androids. or Apple's in car experience it tends to be very app centric.

44:07 And I think It'll be interesting to when You have a personal agent integrated with those experiences and you can say Send an email to so and so about this and all these other things, and all of a sudden rather than Chewing things up on your commute. You're actually like getting work done on your commute in a completely ends free way.

44:25 Those types of things as I said, like the first and second are effects. I just feel like the power The surge we saw in excitement around Smart speakers and all that. Might come back. When those things are actually more effective.

44:37 computers as opposed to mechanisms to get the weather and turn on music, as they are at least in my household. And then similarly, in the same way like first iPhone apps were like flashlight and really rote uses of the Yeah, small schemer things. It took a couple iterations at the app store to get the door dashes and the Ubers and the WhatsApps. My sense in this world of conversational AI and agents. There'll probably be some really interesting native

45:03 Agentic platforms and tools that come to exist as well. And I'm Excited to See those as well. It's so interesting to think about How this changes

45:13 Work, people's productivity, businesses. I'm curious about each. We opened by talking about the hundred X engineer or something. Do you think that all of this just blows out? That phenomenon of rising inequality around People's productivity that like the very best person in any field is just gonna get that much more effective and maybe capture that much more of the rewards available in that field. Does all this technology naturally promote more

45:37 I'll call it inequality in focus on productivity, like How do you think about that? It just seems Bizarre that So many of the things you've described are really exciting. And it just makes me wonder

45:47 the profound shifts that might happen in human labor productivity and the returns to being one of the best. 'Cause I've seen some of these engineers working with cursor and Claude and everything and they're able to do the work of like ten people. Or even like the people in the research labs at OpenAI or elsewhere, it seems like the very best people are having a disproportionate impact. On the world.

46:07 And This might accelerate that. So do you worry about that? What do you think about that? Let me start with a optimistic view of this. And then I'll talk about maybe some of the more feelings of caution that I have as well, because I think it's very multifaceted. And implications.

46:23 One of the Very exciting parts of AI and AI agents specifically. Is it removes a lot of the gatekeeping in a lot of professions. Think about the

46:34 Directors. That use the most special effect, like James Cameron or something like that. And the hoops that he had to jump through for the first Terminator movie. Or the director of Inception of Batman, Christopher Nolan. And

46:47 You look at the cost of making like inception and you wonder if he as a One year old. University Film student. had the ability through

46:59 describing in natural language different visual effects and all of that. Could someone with very few resources actually make a film as powerful as Inception? When I come from the era where I strung two V CRs together to edit a movie. The every iteration of the technology has enabled YouTube creators and now TikTok creators and I think with these tools.

47:21 It enables People with good taste. and good judgment to accomplish a lot more than they could individually. And I do think that Whether it's the cost of hiring a VFX screw or the cost of

47:36 Even learning how to use some of these really arcane tools. That I'm hopeful that it means people from any part of the economic ladder any country. may have the opportunity to produce really exceptional art.

47:51 Or produced exceptional products because There's fewer people you need to ask permission and get financing from to do so. And I think that I'm very hopeful that by technology breaking down many of those barriers and requiring less societal permission to do things and finance things that we'll see. Hopefully some really

48:13 meaningful new creations come from people who might not uh otherwise have the resources or, as I said, societal permission to do so. I do think that depending on how you reflect the lens of equality. I think it can be a broadly equalizing Technology As well. I do think that

48:31 The point on leverage that you made, which is software engineers become even higher leverage, especially the most exceptional ones, is true. I feel like the act of being, say, a software engineer is going from Someone whose job it is to emit code on a keyboard. To be in the operator of a code generating machine.

48:51 And the people who learn how to do that really effectively, we'll be able to operate it. At some incredible scale. I broadly think That is a good thing.

49:01 I think that Just Writing equations on a piece of paper wasn't the job of accountant in Excel displacing that didn't necessarily change the job of the accounting department at your company. I don't think the job of a software engineer is to

49:16 Type into a terminal. The job of a software engineer is to produce software. So I think these tools Are broadly helping the job of software engineering become more self actualized.

49:27 More closer to what The value you're producing. On the flip side of this. I do think that There will be like any new tool.

49:36 The folks who learn how to use them effectively. Will gain outsized value. From using them. I think the proverbial digital divide is a Increase in concern because if

49:48 the productivity gains from effectively using these tools is amplified just by the power of the technology. It's as important as it ever was and much more important now to make sure everyone Has access um to these technologies. And then broadly if you fast forward Thirty, forty, fifty years. I firmly believe that

50:07 society will conform the economy around the technologies that we produced. Just like When this country was founded, we were largely an agrarian economy. Then we became an industrial economy and now we're a services economy, thanks to the evolution of technology around us. The same will happen with AI and I'm not sure We could even recognize many of the jobs that exist on the other side, but we are

50:30 status seeking creatures, we will create an economy around The technology that defines our society. But that transition can be quite uncomfortable. And I think one of the Things that I've thought a lot about is Unlike the say adoption

50:45 Of the personal computer or the adoption of the smartphone. Or the adoption of broadband. Those involved. Infrastructural investments that took place over many years, let alone the adoption of electricity, which took even longer.

51:00 This is largely a software Defines. A revolution, notwithstanding the investment in data centers, but in terms of like our adoption as consumers. As a consequence, I think the change will happen more rapidly.

51:13 Because the adoption Can happen as instantly as the technology becomes effective. And I think that Short term I think it is. Something that we collectively

51:24 companies, research labs, governance really need to think about because It's one thing to shift from an industrial economy to a services economy over decades. It's another thing to change the nature of our work over just a handful of years. And that's something that I think is a really meaningful issue that I don't think has a simple solution.

51:43 In addition to what you're doing at Sierra, you're the chairperson on the board at OpenAI. I know Sierra uses several models, eight or nine models, to build its product and I'm sure that will proliferate and improve over time. I'd love to zoom down to the individual model level and hear your insight on where we are and where this might be going. I had this moment with my kids where when GPT four O came out and the

52:04 Audio conversational aspect of it. And the multimodality of it became, I guess, new and powerful. I just noticed like my kids use it way more as a result of being able to talk to it and ask it questions. It's like Unbelievably effective. What do you think of that shift and

52:19 What are the meaningful shifts happening at the model level, in your opinion? That people might be Underestimating or underappreciating. Given that these things are getting better, as you said, in a gradient way all the time. with these discrete jumps, but you have a cappered seat in many ways.

52:34 In AI. But at the individual model level, what is going on that's exciting to you that Maybe outsiders don't fully appreciate. Well start with what I view as the

52:43 Inputs to progress. Um current sort of like large language models and generative AI. There's really three axes that Or

52:53 continuously improving to produce these amazing experiences that Open I was producing. One is The research, the algorithms. Probably the most

53:02 meaningful breakthrough over the past decade was the Transformers model, which was a paper out of Google called Attention Is All You Need and created the current wave of excitement around AI. And you can think of the algorithmic breakthroughs as

53:19 Methodologies. To either Enhance the scale or the speed of AI or enhance the Reasoning capabilities of AI, things like that. Number two is the data.

53:29 These large language models, the first L is large. They're trained on a huge amount of data. Not just text anymore, but also images and video. That's called multi modal. And There's a lot of Questions about

53:43 what data is available to train on. I think a lot of textual content. large amount of the textual content humanity has produced has already been used in training. They're not All of it, but there's a lot of Other modalities. There's also a lot of interesting companies working on things like simulation and Data generation. So

54:00 How can you have more data to improve these models? And the third thing is infrastructure. So How do you apply more compute? Two

54:08 These different training processes, either pre training or post training. And I think part of the reason why we've just seen such continuous improvement From OpenAI. And at the research labs is all three of those were just seeing tons of investment. Everyone I

54:23 I think NVIDIA is reporting earnings this week and everyone's eager to see If the infrastructure investment has continued. All of the great minds in AI now are working on large language models, give or take. I'm obviously being a little reductive there and so you're seeing just More and more of the smartest minds working on the algorithmic and methodological breakthroughs.

54:43 And then similarly, as I said, on the data side, you not only are Going from text to multimodality, but also some really interesting concepts of synthetic data generation and simulation and all these other things that are really exciting as well. And the combination of all that is Even if one of them hits a plateau, you can see progress in the other and No one really knows when the scaling laws which just the idea that a linear input and compute and data will produce roughly a linear improvement in quality will stop.

55:09 But because there's the smartest minds right now are working on all three of those. You're seeing just remarkable progress. Again, not simply from open AI, though I obviously biased, but I think open AI is the leader You mentioned that I just wanna say that I I've mentioned a couple of times, but I think as exciting as

55:26 Multi modal models. That demo of GPT four oh was so remarkable. I had so many People text me about it, just talking about just how emotional the experience of watching that was. And now a lot of people are using it. In the OpenAI app as well.

55:42 What I love about it is it's The way A science fiction author would describe how you should interact with a computer. We start out with punch cards, then we had mice and keyboard. Now we have touch screens.

55:55 Which are fuel Slightly more haptic and slightly more natural. Every time you watch a science fiction film or a movie, you're just having a conversation, right? And it was probably the first time for a lot of people where it really Felt right. But didn't have that uncanny valley.

56:12 Feeling of I'm talking to a robot. And I think that's really exciting. And so Obviously multi modal models to generate Images and video are quite exciting, like Sora But I think that particular thing, which is

56:25 The interface to computers. Goes away. And we can simply have a conversation to interact with software and interact with this digital world. Is so powerful. I think it's going to

56:37 I I don't know if you had this experience, but my grandparents are now deceased, but All but one of my grandparents. Skipped ever having a PC, but they did have iPads. Which was interesting. So like They had retired by the time sort of PCs became prevalent in homes and

56:53 They're like that's not for us. We don't need a computer in our house. When the iPad came out though, it was accessible enough and they could read the newspaper on it and all these other things that they started using it and it was interesting, they sort of skipped a generation of technology and it was so interesting that Computers were the friction and intimidation of using a computer is too much in the era of mice and keyboard.

57:14 But the iPad thing. Than Steve Jobs and Johnny Ive was enough that this didn't feel so intimidating. Now imagine If they were alive to see just having a conversation would Everyone knows how to talk. We talked a little bit about the

57:27 Impact of AI on equality, which is a very important conversation. I think of just the accessibility of software in this world of conversational is just such a tremendous breakthrough. It's just the correct way we should interact with software and no one needs an instruction manual on how to have a conversation. What's the end state thinking? Is it brain computer interfaces? And is that the end state of zero friction? Man machine interaction.

57:52 It's a great question. I think we'll see lots of experimentation there. I say that with Pause because Since the introduction of the iPhone in two thousand seven. We've seen a lot of devices

58:04 From startups. Apple and Google and others that are supposed to augment that smartphone. And it turns out the smartphone is just such a powerful universal device. It does a lot of things. Perfectly and uh

58:18 Long tail of things. Well enough. That it's been very hard for any new consumer device to Take over that. I think that

58:27 Silicon Valley is made for this moment of experimenting with different ways of interacting with this technology, whether it's a Speaker on your kitchen counter. Or maybe airpods in your ears, or maybe It's brain computer interface in your brain and I'm not sure which of those will end up the

58:45 optimal combination of Accessibility and cost and convenience. to represent the way like the median person uses this technology. But we have a lot of the ingredients already and thanks to the proliferation of consumer Smartphones and others. I think we already have a supercomputer with a speaker in our pocket.

59:05 Connected to very sophisticated headphones. If I had to guess right now for the foreseeable future, that will be the dominant Interface. The best. But because it exists. And in the same way it's been hard for

59:18 consumer device companies to break into like take a slice of what your iPhone does and do it slightly better. That it's hard when you have this Really mature smartphone market. But I do think we'll see some really mind blowing experiences. It's just hard to predict right now. So I'm really excited to see

59:35 The proliferation of R and D. Sure. And I think that I use the phrase it's Everyone's gonna have their Iron Man suit. That's gonna be very personalized to them and

59:46 The thing that I'm really excited about is I hope technology Not to a From What we do. That's not a forgone conclusion. I think

59:54 But We spend so much of our time Staring at screens nowadays. I'm not sure it's great for us personally or for our interpersonal relationships. And I think I'm hopeful that

1:00:05 Some of these technologies will facilitate technology receding to the background, despite being a lot more powerful. I'd love to talk about the future of business and building companies. I've seen you write about like Coas's theorem before. The joint stock company's a couple hundred years old. Technology.

1:00:21 has often changed the nature of companies and businesses and products And I'm curious where you think it will change the most in the future as a result of agents, as a result of artificial intelligence, and so on, because If I was a big company right now, I would just feel scared. I wouldn't know specifically how to be scared. Because the vector of attack seems hard to predict. But I would sure feel like wow, the way that

1:00:45 a normal fortune whatever company his structure today. If I was starting a new one tomorrow, probably would think about some things differently. So how do you think about the future of the nature of a company and building a company? That's a wonderful question. I do think the thought exercise which you've

1:01:01 Into that is What would Given the Products and services your company makes. Imagine a world where

1:01:09 A huge percentage of Some of the operational tasks can be automated. What would the shape of your company be and what would the implications be on your business model? Probably the most Complicated.

1:01:21 Second order effect will be Can a new competitor emerge with drastically different profit margins than yours. I think the reason for that is often if a company has a different cost basis Then you

1:01:35 They can compete unequally on Pricing and packaging and things like that. And so I think it could be really interesting if you have Broadly speaking, I I'm

1:01:45 Having started a few companies and worked at some larger firms. I think a lot about the sort of creative destruction in Silicon Valley. And One thing that I think is Broadly true is when there's a new technology emerges, like the internet and the web browser or the smartphone, it tends to correlate with some startups that become very large companies. And I think the reason for that is

1:02:07 Disruption in the sort of traditional sense of the word, where effectively companies built natively for these platforms Can have different business models in the case of the internet. Like Google under Disney sort of out based consumer software was like a pretty both a meaningful technology and a mean meaningful business model. And move more with more agility than some of the incumbents in this space. I think the same is probably true of a lot of different markets.

1:02:33 Where do you look at? What Amazon did to many of the incumbent bookselling firms. With the birth of the internet. So broadly speaking, I think operationally, how do you think about Yeah.

1:02:44 long term implications of automation, what is the shape of your company and how do you set up yourself for that? And then secondly If you imagined you were starting from scratch, creating some Competitor. Providing similar products and services.

1:02:58 What would be the unequal advantages you'd have as a startup and how do you inoculate your company against that. And I think that's gonna be the most important thing for companies who are thinking about the disruption that may come from this. I do think incumbents have a lot of tools at their disposal, though. They have existing customer bases, they have existing expertise. And as a consequence, I've given this advice, which is I think the most important thing companies need to do. Is adopt AI internally and externally.

1:03:25 Anyone listening can give me a call for the external use cases will help you build a customer facing agent, but internally Your employees should be using an officially sanctioned version of chat GPT, right? That should be a part of how they do their jobs. You should be thinking about not only short term opportunities with co pilots and the like, but saying Let's set some ambitious milestones for automation and some of these back office processes.

1:03:47 So that you can set yourself up for the future and not be on your heels when up starts to take advantage of this and The history is littered with companies that didn't Adopt technology. Quickly enough. The sort of proverbial blockbuster Netflix story.

1:04:02 But there's also a lot of examples of companies who adopted these new technologies quite effectively and took advantage of their scale and distribution and market expertise. To adopt it. I think it's early enough that nothing is too late, but I think it's really important to both adopt it and start developing those lessons. And

1:04:20 Also strategically think about game theory out what a new competitor might do differently, particularly things that would impact your business model. And I think those are things that are just hardest to fix more than any technology adoption curve. Can you explain a bit about OpenAI's unique structure and the impact that you think it has on the work that it does? Most other foundation model company well, there's different structures. I think research labs is like a great way to Talk about them.

1:04:45 or label them or something. OpenAI has this fascinating history from nonprofit to very unique governance structure today. What could you teach us about it and what makes it interesting to you? So really simply open AI as a Delaware nonprofit. So what that means, so if most Delaware companies Or

1:05:05 Basically the Fiduciaries of the company. The board. Serve shareholders. And that's The conventional Delaware Sea corporation like Sierra.

1:05:14 With a nonprofit Rather than being a fiduciary to shareholders. Folks like me were fiduciary to the mission. So fundamentally it's not about Shareholder returns, it's about serving a mission.

1:05:25 And the mission of Open AI is to ensure that artificial general intelligence Benefits all of humanity. It's a really broad mission. I think that's part of the reason that OpenAI both because of the influence of chat GPT and the impact of the organization, but also That mission is

1:05:41 uh ink blot test for whomever reads it on They're a greatest. aspirations for what could go right or could go wrong with AI and I think the breadth of the mission is also part of the reason why open AI is at the center. Of so much of this AI conversation.

1:05:55 As you alluded to on the unusual structure. A number of years ago before my time, the company Created a for profit subsidiary. to raise the capital That it needs.

1:06:05 To build artificial general intelligence. I think the I don't want to speak for The founders, but Broadly speaking, I'm not sure anyone knew.

1:06:13 When open AI was founded, the amount of capital you would need to build AGI it's not really a Bring on a bunch of researchers and think a lot. It's Build. Computers and train a lot. As well.

1:06:25 And I joke you can't really have a nonprofit steak dinner fundraiser and raise Five billion dollars. Doesn't really work. Notably to as you mentioned research labs. My understanding is that Anthropic is a B Corp, which is a benefit corporation, which means it has dual accountability to a mission and shareholders. So Think a lot of the research labs.

1:06:44 Broadly speaking, or I think trying to recognize the importance of being mission driven in the age of AI and really Focusing on benefiting humanity, which can has never been more important. Well also creating a structure that can accommodate raising the kind of capital that you need to build AGI, which is a lot more than I think anyone knew a decade ago.

1:07:04 One of the things that comes up a lot when you ask around about you and your story and your background is this Paul Graham essay Keep your identity small. And if you look at the list of things you've done. Not just the incredibly impressive companies. Facebook and Twitter and OpenAI and Sierra and all these others. But also the different roles that you've played, CTO, head of product, CEO. Chair of the board.

1:07:26 It seems that you've done a good job of Evolving or adapting your way through A landscape which itself is changing quite rapidly. And Again, let others say it for you and not have to say by yourself.

1:07:38 It seems like you are exceptionally good at that adaptability. I'm curious. What you do. That makes that true. I've told Cheryl Sandberg this story and she has no recollection of it, but I'm confident

1:07:51 Yeah, I remember it correctly. I credit her For this. Adaptability in my personality. But basically I was struggling a bit as CTO of Facebook. And I was struggling with

1:08:05 Basically scaling. It was probably the largest organization I had ever managed. Yeah. Cheryl was everyone's mentor at that company. I was probably twenty nine at the time. She was the old guard and she had more experience than all of us and was not only generous with giving advice, but also

1:08:22 Would tell you the uncomfortable things you didn't necessarily Want to hear but you needed to hear on how you were doing. And she had this very hard conversation with me, basically that I wasn't holding the teams I was work that were working for me to high enough standard. And That the reason why

1:08:39 this project was going poorly was basically because I was trying to be heroic and do the job myself rather than Grow this team underneath me. It was a very impactful conversation for me because

1:08:50 Yeah, you start out dejected like I suck. What do I do wrong? But I w reflected on it and I was reflecting that I had gone from being Kind of a Technical architect.

1:09:03 a very senior influential individual contributor to managing this big team. And I was trying to conform the new job to me. Rather than Conform myself to the new job. And that was my reflection.

1:09:16 Yeah. I don't know if it was like the next Monday it was Soon after that I Woke every morning And thought rather than

1:09:24 I'm at this with a strong sense of my identity and who I am. I'm gonna think every morning. What is the most impactful thing I can do today? So accomplish the goals that the company wants to accomplish.

1:09:36 And It worked and the Counterintuitive. side effect of it was it was delightful for me. So I had this image of myself as a technologist and engineer, and the reason why I was subconsciously trying to conform the role to my identity is I thought

1:09:52 This is my identity and this is who I am and What makes me happy. I found out through the act of trying to have a looser Conception of my identity. Is not only was I better at my job

1:10:04 But the act of actually just having things go better. With Very fulfilling for me. And it became a This self

1:10:11 Fulfilling prophecy which is I was just focused on How do I achieve the end that I'm trying to achieve and By having a very loose concept of Who I am and I was just trying to do the most impactful thing to accomplish

1:10:24 The big picture, like what are we trying to solve? Things started working really well. And it was a Reinforced to me. That's

1:10:32 Actually what I care about is impact. More than anything else. And it's become something that I am very grateful for. I'm very grateful for that hard feedback I got in a sort of vulnerable moment. Because I think I probably did have a overly ossified conception of my own identity and now it's

1:10:49 I talked to some people who knew me on Google Maps and think of me as an engineer and I talked to some people at Facebook who think of me in that technology leadership role. I talked to some people from Salesforce who Think of me as a An executive on the leadership team as COO and co CO. And they're all right.

1:11:06 In those circumstances that's who I was. And I'm very comfortable putting on multiple hats and very comfortable taking them off as well when the timing is ortees. Talked a lot about technology and product a bit today. What about sales? What did you learn about world class sales while at Salesforce? Mark Benioff is just a remarkable The greatest of all time. An enterprise software. I'm so grateful to have worked with him.

1:11:32 We're also really different and I just learned so much from them as a consequence of that. The thing that I really appreciated at Salesforce and Mark individually. Is just that genuine Focus on the success of customers. It was something that there was a true accountability.

1:11:49 And I think Every tech company talks about Focusing on customers. Most of the time it's lip service and it's like How do people actually spend their time and what would your customers say about you? As the true measure of it. And I saw in Salesforce the most sort of customer centric. company I'd ever experienced. And it really changed

1:12:07 I thought which was the company that Salesforce Acquire was customer centric, and I was like, We weren't. I just hadn't seen Great, yeah. And so I really saw that from him. How did they do it better than Quip? Probably deep listening. I think that

1:12:21 There's a Though I don't think the term had been coined yet was A product led growth. Company. Like Slack and Atlassian and others.

1:12:30 And as a consequence, I would say The anchor tenant. of our strategy was really around the product and how we improved it. And The concept. I don't know if we like so formally articulated was the better we make the product, the happier customers will be, the bigger our business will be.

1:12:46 Which is True, but that can lead to either patronizing customers they don't really want what they say they want. They just don't understand the true power of these cool abstractions that we've made or All these other things.

1:12:59 And it can lead to arrogance. And I see this a lot with Really product centric. B to B SaaS companies. Bad. You're trying to convince the world of your worldview?

1:13:08 Which is Really powerful. When it's right. It could be quite arrogant when it's wrong. And for Salesforce it's just a more mature place with a

1:13:17 Really great CO and founder. And I just saw an organization that just constitutionally was really deeply listening to its customers. I'm really interested in three of the values around which Sierra is built that we haven't talked about yet. I have a way of asking about them. I'll probably do it three times, which is how you would assess this value in another company. and how you promote it in your own. So the first one is intensity. This value. was intentional.

1:13:40 Because of the market that we're in. There's a Mark Twain quote, History doesn't repeat itself, but it rhymes. And I think the AI boom. Well rhyme.

1:13:49 With the dot com bubble. And what I mean by that is that There will be a lot of hype. A lot of snake oil, but there will also be some generational firms created in this wave. In the same way you had WebVAN and Pets.com, but you also had

1:14:05 Amazon, Google. eBay, PayPal, Salesforce, and others come out of it. Similarly I think that a lot of the ideas The We at Cyr are pursuing and

1:14:17 Companies like Amazon were pursuing the dot com bubble aren't necessarily unconventional or surprising ideas like selling books online or Let's apply AI agents to help with customer experience. They're conceptually obvious, but it's all about execution and Amazon got to write the history books, not buy dot com and Google thought to write the

1:14:37 History books not Alta Vista. Or inktom. Because of the Product technology. Go to market execution.

1:14:45 And so The idea of intensity is We know we don't have the luxury of patience. Have you inventing a new concept like maybe Facebook and Slack did. You exist alone because you've built a new way of doing something.

1:14:58 In a model where a lot of the technology is the rising tide that's lifting all boats. We could sit around and pontificate about the future and the world could move on without us. And I think that was true. In nineteen ninety seven, and I think it's true now. And we really wanted to reflect that in our values.

1:15:16 What do you do to up intensity inside the business? How do you get more intensity in a business, besides talk about it? For me it really comes down to urgency. And caring about every detail. And I think

1:15:28 Like all values, if you think about company values. Is it a poster on the cafeteria wall or do you feel it? I always bring up Two of the values I really felt early on in my career at Google don't be evil. And it was really about how we thought about

1:15:42 advertising and transparency around it. I felt as a product manager at Google, we talked about it all the time. And People were proud to do things differently than the status quo at the time. And and Facebook, though. This expression has been Maybe

1:15:58 understandably maligned, but this move fast and break things, which was the pace I remember going into Facebook and just feeling The pace. Just feeling it. You go in there and you're like The first day I re engineer. Ship code to production.

1:16:11 And you're like Well This is awesome. This is so cool. And through the things that you do and your lived experiences at a company, you feel these values. And so for what I think about intensity, it's If there's a competitive deal

1:16:23 Do you feel the depth with which every colleague you're working with cares about it when you're shipping a feature? Do you just sense the urgency on getting it out there, having it be perfect. In all aspects of Your brand

1:16:37 Your product, your technology. Is there a sense of urgency and focus? And I think Like many things in a culture, you reflect what you feel all around you. And so I think it starts with Clay and Me. You feel it from your colleagues and you come in and you feel that intensity and then you can reflect it back out. What about craftsmanship, which seems Maybe at first blush a little bit in tension with intensity.

1:16:58 Craftsmanship is always something I've admired. When I think of the company that represents craftsmanship the most to me would be Apple. There's that Famous, I think was of Next, not Apple. Where

1:17:08 Steve. cared about the inside of the work station, not just the outside and like how it looked, and famously I think cared about less the Aesthetics and layout of the factory floor. Like any of these things, there's probably bad caricatures. Of these values, but

1:17:22 What's interesting about Apple Whether you go into an Apple store or visit their website or unbox one of their products or turn it on. You can Feel the craftsmanship, the details are right. I think that's a good thing.

1:17:34 This is a stereotype. But I think that if most people think about the products that are in their lives that are well crafted Almost universally you'd pick Consumer products.

1:17:46 And it's In part because there's Really direct. correlation between sales and craftsmanship and can in the consumer world, like a Sonos speaker. Being just incredibly well crafted.

1:17:57 In enterprise software often the products that you use are not well crafted. And it's in part because typically the buyer of enterprise software is distinct from the user of enterprise software. Not always. As I mentioned, some of the product led growth companies like Slack, that's different, but it's by and large true. And so it become backhanded joke in software circles about the enterprise software you love to hate.

1:18:20 And it does the job, but it does so with such clunkiness and inelegance that it's embarrassing. I think as we're building this new type of software, which is helping companies build AI agents, the craftsmanship in every interaction with Sierra. Whether it's a slide or a phone call. Or a document or a product. If you feel that the details are right, you have trust that we're gonna get all the details you don't see right as well.

1:18:44 And we really felt that was important, particularly in a Area of software that is so new. And were an upstart in this world and a startup that I think it would just help us in developing trust with what we're building and also be a place where product designers, engineers would be proud.

1:19:00 of the products that they work on, which I think is really important. How do you keep yourself from being overwhelmed? I know that you care tremendously about your family and spending time with them. You're building an ambitious business that requires intensity, which Often equals time. You're an important player in other huge businesses like Shopify and Open AI. That's a lot.

1:19:20 Going on. And I'm sure you love it all. Which is a big reason that you do it and you get to learn At a crazy pace. How do you keep the dam from Breaking with all that responsibility and all that action.

1:19:32 I'm not sure I'm perfect at it, first of all. I think I am Like many focused and intense people, not Perfectly balanced in all ways. Nor do I really try to be.

1:19:43 I always think that people's strengths and weaknesses are often strands of DNA. They're quite intertwined. One of the things I try to do is recognize my own strengths and weaknesses. And then try not to fight gravity. I am whom I am. But broadly To answer your question on how do I attempt to try to find balance?

1:20:01 Is intentionality. So I think that Especially in the age of push notifications and email. Often you can let

1:20:09 Your interactions with others define how you prioritize your time. That's a choice. If you're gonna Choose to be in box zero, that's a choice. It's a completely rational choice, by the way. But

1:20:21 Your time is going to be a function of how many people are sending you email. And so You're not in control of that time. Similarly, if you have a really intense job and you want to have dinner with your kids. Have dinner with your kids. It's just a choice. You can go home or not go home.

1:20:36 But you can say yes to Dinner meeting or no to a dinner meeting. One of my mentors, Susan Wojisky, just passed away and was famously would never take a dinner meeting. So she could be home with our kids and was most powerful executives in Silicon Valley. I would just say that I think

1:20:51 A lot of People present things as choice is not available to them and I'm not trying to be insensitive to the power hierarchy of Where you work or anything like that, but Everything is a choice. And so I really tried to be

1:21:06 Self critical and accountable. But the choices of how I spend my time. Try not to pretend that anything is not a choice. You can quit any job, you can Abandon your family. Probably neither was something that most people want to do, but everything is a choice.

1:21:21 And so I try to not let the world happen to me. It's very taken me a long time to get good at saying no. It still feels painful, but I've gotten better at it. So intentionality is the short answer. Coming back to agents where we started, what question Or questions. Would you encourage companies to ask of themselves on the dawn of the sort of agent era.

1:21:41 I'll start with uh maybe a broader question to ask and then talk about the first step because for whatever reason in my head they were distinctive and I want to answer both. So There's a author and business thinker named Clayton Christianson. He wrote a book called Competing Against Luck.

1:21:58 Which created this framework called Jobs to be done. The idea of jobs to be done is that As a company, you really need to say what job is my customer hiring me to do. The example that I'm probably screwing up

1:22:11 Because They were consulting for a fast food restaurant that was trying to increase milkshake sales and they were saying like how can we improve The recipe of our milkshakes to make them sell more They ended up doing a study where they discovered that a lot of people were buying milkshakes in the morning.

1:22:29 Because it was a very Portable. Item and a commute. Easier to eat than like a sandwich when you're on a c long commute. And filled you up.

1:22:38 And then there was parents in the afternoon who were buying milkshakes to basically appease their kids. I don't actually remember the anecdote precisely, but they started promoting milkshakes more in the morning, making smaller size for the kids in the afternoon or something, and basically recognizing that Changing the recipe was not going to actually increase sales because you conflated the taste of the milkshake with what is the job your customers hiding them to do. Which is to provide

1:23:02 Sustenance on a long commute and to Provide a way to make your child happy in the afternoon. And if you frame it with the value that you're providing You might solve a different problem than the recipe of the milk shape, which is the instinctive yet incorrect thing that you would change about a milkshake to have it sell differently.

1:23:21 Think in the age of Massive technology disruption. It's very important for companies to understand what is the value that we're providing to our customers. But uh A great example of this was it was an earnings call from Netflix years ago.

1:23:35 I didn't hear it, but I saw a recollection of it where I think Reed Hastings mentioned that the greatest competitor Netflix at the time was Fortnite or something. I'm probably again screwing up They said. And it was because the job of Netflix wasn't to stream a movie. The job was entertainment in the evening and you're gonna choose between do I wanna play Fortnite or do I watch a film and

1:23:56 It means that your main competitor might not be HBO. Your main competitor might be this gaming platform. So in the age of technology, I think so many companies And uh Organizationally and psychologically conflating. what they do and the value they provide with how they do it.

1:24:11 And it's often a huge pitfall because you end up constraining how you respond to technology disruption. Through the lens of your existing delivery model or organizational model. And the companies that have a very crystallized view of why they exist. Why do our customers Talk to us. What value are they getting?

1:24:30 It can become so much Simpler to learn how to apply technology because You don't view the way things are done as precious. You understand value provided. I think that really needs to start at the top. And really have about stand and

1:24:44 It's uncomfortable for individual employees sometimes because Maybe the thing you're doing is an artifact of the current technology ecosystem that could change. But if you buy into that vision and have a beginner's mind about The different roles of the company. You can get on the train, right? You can reskill, you can find new opportunities. So I think that's the most important thing is

1:25:03 What is the job our customers are hiring us to do? What will be different in the age of AI and what will be Stay the same and how do we map a path from point A to point B? In terms of getting started, I think that two things that I would do that are probably true across industries are number one Build a customer facing AI agent so that your customers are experiencing an AI version of your company. I think that's just like having a website in nineteen ninety six. You can start small.

1:25:28 I think I remember a lot of websites in ninety ninety six were like here's our phone number. Like you have a dot com on your business card. And I think having an AI agent Is really meaningful. And number two is Open the door to use any I at your company, whether it's Coding assistance, chat GPT for employees. Think about it just the emergence of a spreadsheet and finance. The more your company is

1:25:50 not only not discouraging the use of these models and day to day use, the more you'll employees will be coming to you with really meaningful ways to improve your company works. And I'm not sure Top down is important, but actually I'm a huge believer that especially at larger firms It's often the people on the ground, where the real problems are, how to apply these technologies in meaningful ways and then Empowering and providing the structure so you can empower your employees is just hugely impactful.

1:26:17 Your companies are the ones that I'm watching to see what is the next generation of this technology enabling. It's been so fun to hear. All the ins and outs of it here with you today and just learn from your experience. I always ask the same traditional closing question, what's the kindest thing that anyone's ever done for you? My wife saying yes that she'd marry me. Super simple. It is. Thank you so much for your time.

1:26:38 Thanks. Um If you enjoyed this episode, check out Join Colossus.com. There you'll find every episode of this podcast complete with transcripts, show notes, and resources to keep learning. You can also sign up for our newsletter, Colossus Weekly, where we condense episodes to the big ideas, quotations, and more, as well as share the best content we find on the internet every week.

What you see above is a preview of the first minutes. One unlock costs 10 credits and covers this episode forever: full segment and word-level timestamps on this page, plus .txt, .srt, .vtt and word-level JSON downloads, as many times as you like.