Why great AI products are all about the data | Shaun Clowes (CPO Confluent, ex-Salesforce, Atlassian)

Transcript from https://podmenti.com/t/a4332a57380138db

I love that you have very strong opinion about this, which is just the state of the product management career and how it most PMs are not
That great. Why is it that product management is still such a relatively undeveloped discipline? Like we're like fifteen to twenty years into this thing. And so there's something about the current state of product management that isn't getting at the truly important things, the truly value added things. If we were doctors, you'd be like that's totally unceptable. What's the answer, Sean? How do we solve this problem in everything? Always talk from the
Customers' perspective, from the market's perspective, from the compender's perspective. A very small number of PMs do that. They get dragged into internal politics, they get dragged into Scrum management or Scrum execution or product delivery. And you just can't win that way. You kind of have this hot take that the way AI will most impact product management is data management. Well, you've got this synthesis machine, which is
LLM thing that's gonna help you do synthesis. But if it hasn't got all that data to do synthesis on top of, it's got nothing. And so that means that LMs can only be as good as the data they are given and how recent that data is. In the future, if you can easily clone a B2B SaaS app like Salesforce or Atlassian, what happens to these businesses long term? Do they just become are they all in trouble.
estimate where the value is created in these applications and they just kinda get it completely wrong.

Today my guest is Sean Klaus.
Sean is chief product officer at Confluent.
Previously, he was chief product officer at MuleSoft, which is a billion dollar business within Salesforce.
Before that he was chief product officer of Metro Mile, a public auto insurance technology company.
And prior to that, he spent six years at Atlassian, where he ran the Jira Agile and also built the first ever B2B growth team.
He also created two of the most popular Reforge courses, one on retention and engagement, and one on data for product managers.
Sean is awesome because he is both very tactical and execution oriented.
While also being very philosophical and insightful.
About the craft of product and growth.
In our conversation, Sean shares why most PMs are not good.
What it takes to become a good or great product manager.
How he thinks about his career, like a bingo card, and why he indexes towards finding very different roles for every new job that he takes.
Why good data is the most important ingredient in AI tools
And for product managers working with AI.
Also, how to build a great B2B growth team. What he's learned about doing B2B growth.
and his really interesting take on how AI will and won't disrupt SAS tools out in the wild.
If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. It's the best way to avoid missing future episodes, and it helps the podcast tremendously.
With that I bring you Sean Claus.
This episode is brought to you by Interpret. Interpret unifies all your customer interactions, from gone calls to Zendesk tickets, to Twitter threads, to app store reviews.
And makes it available for analysis. It's trusted by leading product orgs like Canva, Notion, Loom, Linear, Monday.com, and Strava to bring the voice of the customer into the product development process.
Help you build best in class products faster. What makes Interpret special is its ability to build and update customer specific AI models that provide the most granular and accurate insights into your business.
Connect customer insights to revenue and operational data in your CRM or data warehouse to map the business impact of each customer need and prioritize confidently.
And empower your entire team to easily take action on use cases like win loss analysis.
Critical bug detection, and identifying drivers of churn, with Interpret's AI assistant wisdom.
Looking to automate your feedback loops and prioritize your roadmap with confidence, like Notion, Canva, and Linear, visit Enterpr.com slash Lenny to connect with the team and to get two free months.
When you sign up for an annual plan.
This is a limited time offer, that's interpret dot com slash Lenny.
This episode is brought to you by buildbetter.
Back in twenty twenty when AI was just a toy.
Build better bet that it could cut down on a product team's operational BS.
Fast forward to today, 23,000 product teams use purpose-built AI and build better every day.
First, Build Better uses custom models to turn unstructured data, like product and sales calls, support tickets, internal communications, and surveys into structured insights. It's like having a dedicated data science team.
Second, Build Better runs those structured insights into workflows, like weekly reports about customer issues, context aware PRDs, and user research documents with citations. It even turns stand-ups into action items that automatically get assigned and shared into your tools.
Plus, with unlimited seat pricing on all plans, Build Better ensures everyone at your company has access to this knowledge.
Truly no data silos.
In a world of AI demos overpromising and under-delivering, CY Build Better has a 93% subscription retention. Get a personalized demo and use code LENI for$100 credit if you sign up now.
at buildbetter.ai slash Letty.

Sean, thank you so much for being here and welcome to the podcast.
Thank you, Lenny. It's really awesome to be here.
I've had you on my radar for a long time, and I am really excited to finally have you here and big bonus points for having a very
beautiful sultry Australian accent that always
Helps with the ratings, I think. I don't know if it's causal, but it's correlative.
I'm I'm I'm glad to be a bit of a curiosity.
Yeah. So I want to start with something I totally
believe and I love that you have very strong opinion about this, which is
Just the state of the product management.
uh career and how it's
Most PMs are not that great.
And how there's a big opportunity to level up. You just talk about what you've seen there and you're just like
Thinking here.
Yeah, it's honestly like
A big conundrum for me. I think it's actually part of
I would it's grandiose to say so, but of my life's work.
Like why is it that
Um, product management is still such a relatively undeveloped discipline. Like we're like
Fifteen to twenty years into this thing.

You would have thought that
It would be less random than it is? Like the outcomes are random, the behaviors are random.
Individual performance is random.
Yeah, seemingly. Right. And so there's something about
the current state of product management that isn't getting at
The truly important things, the truly value added things, the
the right way to think about problems, the right way to think through problems, the abstract reasoning that's needed, the something that isn't working about it. I've spent a long time trying to put my finger on it.
And then be like, how do you reproducibly produce that? Like reproducibly produce people who can really be really great product managers.
The thing is that if you think all the way back to it.
Like I spend a long time as an engineer.
And people always talk about ten times engineers, right?
And I wanted to be a ten times engineer. You know, I'll leave it to others to decide to tell you whether or not I was or I wasn't, but certainly I wanted to be. And I tried to be a really great engineer.
And it must be true that if there's ten times engineers and I would argue they're definitely there must be ten times product managers too.
But at the same time those ten times product managers
Because product management is ultimately about leverage. So it's about helping other people have dramatically more impact than they would
you know, if they were unorganized that they didn't have somebody to kind of organise the goals and what we're trying to achieve.
Then that means that a ten times product manager has a hundred times return.
M or more. Because they're because they're ten times the return on ten times resources.
Right. So the outcomes are so wild like wildly distributed and the benefits are so good that you would have thought that
Kind of if it would have behooved us, there would there would have been a way that this had evolved and improved and really gotten way crisper than it has.
Uh
Here we are, you know. I'm not saying that we haven't gotten better, we one hundred percent have.
But I I think I think it we could all say that we're not
Reliably producing
You know, ten times product managers every day.
Every day of the week.
I love this point and it's especially painful that
When someone works with a PM that's
Not great.
There's just this like meme of why do I need PMs? PMs are useless.
P Sack.
And it just creates the like no one's ever like engineers are useless.
Or designers are useless, but ever there's so many people are like, I don't need product managers on our team never hire a PM.
And it just s sets the whole profession back.
When I first started out in PM somebody, you know, it's a tr it's obviously a chestnut, but he pointed out that like realistically when you're a product manager, your job is to say no to ninety percent of things that get that get brought your way. And so that kinda makes you the bad person pretty much from the start.
And so you're saying no to ninety percent, so you can say yes to ten percent?
And th that kind of puts you behind the eight ball right at the very beginning.
And so you have to kind of very quickly get runs on the board, you have to prove to be the to have the right insights, to have the right data, to make the right decisions. Or you don't get another go, you don't get another swing you don't get another swing at it.
So it makes sense that, you know, product managers are the easiest to kinda single out and
Uh and kind of criticize.
But that is also what makes it the funnest thing. Like if you think about like why do we do this? You know, somebody once asked me like
Yeah.
Would you retire? Like wh wh why do people do what they do?
Uh'cause certainly at some point it isn't just about the money.
And at the end of the day, product management is so damn fun.
Because it's about hope.
Trying to figure out an edge. It's like trying to look at the world.
Find the portion of the of the chessboard that isn't occupied.
But that is valuable.
And find a way to get into it, invade it, and destroy it. Like it's a it's a it's a really fun
Like it's decisions under uncertainty.
And that makes it unbelievably fun. Like really, really painful and very frustrating and very hard to convince people, but very, very fun. Um, so you know, in equal measures, basically.
What's the answer, Sean? How do we solve this problem?
I know you said it's your life's work. What el what do you find actually helps most in helping PMs level up and become Satanic's PMs?
Th I think the most important thing and the kind of the chestnut that I repeat to everybody
Is that at the end of the day
the time you spend looking inside the building doesn't really benefit you very much at all. Right. And you know, Steve Blank and people used to talk about you should be spending eighty percent of your time thinking about things going on outside the building. You might not be outside the building, but you should spend eighty percent of your time thinking outside.
And I would say that very small number of PMs do that.
They get dragged into internal politics.
They get dragged into Scrum management or Scrum execution or product delivery, like elements of the delivery thing.
Uh and you just can't win that way. Like you just you just can't win that way. Th you can never get an A.
Because you're because you're fundamentally not solving the job. Like the job is not about execution or anything, it's about
Finding reliable differentiated value, right? That you that you can uniquely deliver into the market. So I would say that
If there's one thing I you know, two things I would say actually that I generally guide product managers to do. One is to like always start from the point of view outside the building in every document, in everything, always talk from the customers' perspective, from the market's perspective, from the competitors' perspective. And the people who listen to me on that, I would say it get better almost immediately, because they're starting from a place that's easier to understand.
And then secondarily be data informed. Like kind of use
Use all of that view of the world, but don't just make up a bunch of statements, like support that statement with
You know, anecdotes and bits of data. Doesn't have to be a treatise.
But like
kind of bring in to bring uh kind of convince everybody of what the world really looks like and what the opportunities ahead of the company look like and good don't good things happen to you. And all of a sudden you go from a world where
Nobody wants to help you get anything done.
to where everybody is wants you to win. If they want you to win and they they may not give you everything you want, but they certainly will try because they're like, Well, of all the bets we could make, this is a good one.
I imagine many people listening to this are thinking, Oh, I am that person. I talk to customers all the time. I'm always interacting, looking at research.
Putting data together.
And I what you're saying is you're probably not
Doing that enough.
Is there anything that you could help someone recognize of no, you're actually not doing this enough? And
You think you are, but you're not.
It's one thing to say you're spending a lot of time looking outside the building. It's a whole other thing.
to like
hear from the places you don't normally hear from. So like so avoid ava avoid availability or confirmation bias. Like most of the time people go talk to the people they always talk to.
And they learn nothing particularly new.
They don't synthesize the results that they got from that conversation. They don't seek out the counterfactual. They don't seek out the proof that they're wrong.
They they they don't analyze what their competitors are doing and figure out what that must tell you about about the market.
They don't bring back the data of how their product is actually being used versus how people say it's being used.
It's it's like
You know, kind of all data no analysis is not very useful.
Like all kind of like you know, everyone can bring back an omnibus edition of like, you know
Random stuff I heard on a Tuesday.
But the but the competitive advantage is is extracted out of figuring out what other people don't see.
Figuring out what
You know, where we're wrong, figuring out where
A well placed bet could have dramatically um you know uh outlandish r returns.
And so people
You know, I think firstly people often say that they do a lot of this stuff, they but they actually don't.
Right, because they don't have any structured way of doing it. So what they really mean is like every now and then I get in a I get in a customer call or every now and then I get stuck into an escalation. And so they're kind of conveniently bucketing it.
So firstly they they don't do it in a very structured way.
Then they don't bring back analysis, they get true insights from that thing.
So they don't really gain very much at all. It's just it's just a more more activity
No outcomes. Activity is not people people do far too much activity with not enough outcomes.
Um and there just is enough time in the day to do that to be successful.
You as a
product leader is at the Venn diagram center of the
sweet spot of where this podcast has been going recently, which is
Product and growth and
How AI helps you with all these things.
And so to follow a thread there.
With
Synthesizing and understanding what
People are saying custom user
research and surveys and all these things.
Have you found any tools that you and your team have
found really useful to help you do this more efficiently versus, you know, traditionally just
Manually going through all the stuff and finding
Patterns.
Yeah, so firstly like stepping back a little bit just into like the motherhood and apple pie portion of like um of qualitative research or whatever.
Like I find that most people don't even understand
Well don't start with the rigorous foundation in what they what they're gonna need to do to get the answers that they want. So for example
Your listeners have probably heard about the Nielsen number before. But basically the idea is that once you interview between seven and fourteen people, you stop learning new things. Less than seven, you don't learn enough.
More than four fourteen, you start learning anything new.
And so if you interview two people, you probably don't have enough data.
If you interviewed twenty-two, you probably had too much. So like they don't even right size their efforts. So that's a problem. So they don't start that that way. Then they go into these conversations.
Asking leading questions?
Which really are designed to get the customer to say what they already want to be true. Which you think
So the so they haven't done enough research or they've done too much, and then they've blown up all of the results before they've even heard anything. Like so you know, if you don't if you don't right size your research and you don't kind of set this up to learn.
then you're going to lose. Like no amount of
No amount of applying LMs or any type of kind of structured reasoning is gonna help you.'Cause you just basically
You're reading back what you want to hear, or some more weird summarised ver version of what you want to hear.
But you know, stepping back from all of that, like
What I like to do is uh specifically getting to LMs, is like I think that
We live in just the most amazing time for product managers right now.
In terms of being able to analyze vast quantities of information
And see the common threads.
So give it let me give you a few a few examples of that.
Right, one might be you can do a bunch of uh custom interviews.
Yeah.
interviews into chat GPT and you can say, Hey, Chat GPT, this is my strategy.
Tell me where my strategy does not fit what these customers talked about.
It's all about the not, not where it does, where it does not. Like people spend far too much time looking for what they're hoping to see, not for what they're not looking to see.
So you can you can literally ask ChatGPT to help you find
W
the customers probing at the edges of what you're trying to do, where it's where it's wrong, where what you're saying is not what they believe. Um and you can ask it questions like that, you can ask it where
you what what your customers are saying would better fit what your competitors are saying. So you can basically say, hey, you can copy and paste one of your competitors' positioning documents into chat DPT and say, is this a better fit for what they have said than my thing, which is which is you can summarise your your own strategy.
I uh you can take your competitors but public documents
And you can ask it to summarize what the strategy probably is.
And it's actually surprisingly good at that because mostly your public documents
Uh actually a summary or at least a derivative of what your strategy is.
So it will give you crazy insights into what other people's literally their product strategy at times creepy, like oh, they will probably do this, they will probably do that. It's more likely they would do this than they than they would do that.
And so like normally that type of
Inside
was hard one.
Like, you know, it's like it took a lot of m sweat work. You you basically get to read a lot of stuff. You kinda had to use your brain as like this big
kind of summarization machine and eventually you knew what you felt about all the things you had read, but you couldn't summarise why. LLMs that let let you get to that.
really, really, really quickly in a very structured way.
But only if you
push at the edges, provoke the provoke the answers you don't want to hear, provoke the problems like
Try and try and
you know, prove to yourself that you're wrong, I think is the easiest way to start.
Um try to use some of these tools.
I love that. And it sounds like in your experience, you're just using straight up open AI, JATGBT, Claude, not like
Any specific tool for user research for this specific use case? Uh no, mostly I find that like the the straight up LMs themselves are good enough.
Um
And uh and we do have some internal tooling that we built um around
You know, I don't know if you've ever had um Sachin Reki on on the show. You may have, uh he was a product leader pretty g well known in the gross community and he was a leader at LinkedIn for a long time. And he kind of uh
He used to call he used to call this concept a a feedback river.
And he basically said like really smart
Product managers are constantly swimming in a feedback river. They set out to surround themselves by feedback river.
And uh and and I really deeply believe in that. It's like okay, how can I surround myself with
you know, user interview data with direct customer feedback, with NPS data, with
competitor information, like I'm always kinda trying to wash myself over with information.
And where I'm going with this is that uh LMs and tooling based on it can be exceptionally good for this.
So for example, we get a ton of uh content, we get a ton of inbound customer requests, as you can imagine, coming from the field or directly from customers.
We use LMs.
to take in the the those those ask.
To summarise what they're about.
to find other ask that are like that one, like really in a compelling way, like a real way, like a semantic way, not a not a other words exact exactly the same. Are these the same concept?
So that we can look across all of the inbound demand on us and say, Well, the most popular idea is this one and it's getting more popular. The least popular idea is this one, it is getting less popular in a r in a really deep, rich way, even across
hundreds or thousands of pieces of inbound feedback.
I I think you know it's a really great time to be a product manager.
If you can put these types of tools to work.
But they w they don't do the job for you.
They just help you do these things that are, you know, intricate in that job of finding the finding the gaps, finding the opportunities.
Finding the common threads without
You know, necessarily having to do all of it just inside your wear wear, just inside your brain.
I'm gonna stay in this A AI river that we're in right now and ask a couple more AI related questions.
And this may be what you just said, but I'm curious if there's more here.
You kind of have this hot take that the way AI will most impact product management is
Is it
data management and data versus like
models you're building or anything else is can you talk about what you've seen there?
Yeah, I mean I think there's two implications for people as they're building products based on AI and as they're thinking about um like AI and their workflow. So let's start with the first one because that's how product managers do product management things.
You just asked this question of like
Should it be specific tools built for, you know, to to make AI easier for product managers to use?
Or is it in fact?
like more general models being put to work.
At the end of the day.
Like these models are very, very, very smart.
But they're also like insanely dumb.
Like and everyone knows that, right? Insanely dumb. In other words
They really only know what they were trained on.
or what you bring to them right at that moment, like in that millisecond, and then they will forget it immediately.
And so and it's very easy to convey
convince yourself that that isn't true. But it it's actually what really matters. And let me add one extra piece that makes that really important.
At the end of the day, information has a decay rate.
So think about customer feedback.
It has a decay rate.
Or what your competitors are doing has a decay rate.
So any new piece of data decays in its value to your decision making very, very quickly. Very, very quickly. It's you can point your own decay chart if you want to, but the answer is very, very quickly.
And so when you think about the job, which is synthesizing all of this very complicated information to make good decisions.
What does that mean?
Well, you've got this synthesis machine, which is this LLM thing that's gonna help you do help you do synthesis.
But if it hasn't got all that data to do synthesis on top of, it's got nothing.
And so that means that
Like L and Ms can only be as good as
the r the the data they are given and how recent that data is, they're ultimately like information shredders. Like they're they're they are
And they uh
you know, limitless information eaters. Like they just can't be you can never have enough information
to give to an L and M to truly g g gain its value.
The more things you give it, the better it gets. Broadly speaking, that's the you know, kind of just not perfect, but that's close enough.
And so what that means is a as an internal product leader or you know, using putting LMs to work.
You need to figure out how to bring as much information about customers or their ask or your competitors, all of it. How can you find all of it and bring it together and give it to the LM, either in your tooling or even in just copying and pasting or whatever your flow is is gonna be. That's one thing.
But then if you take it beyond that and you go, Okay, well now I'm a product leader and I'm building an app
And I wouldn't put AI in my app.
What will make my AI experience really great?
It's definitely not gonna be the models.
Because like these models are mostly going to be somewhat replaceable. And you could say, Okay, well is it gonna be the prompts?
Maybe.
But
You know, certainly good prompts are better than others, and you certainly like that's kind of an ongoing investment you'd probably want to make to ask better questions to get the LM to deliver better answers.
But it's obvious that the real answer is the context, like all the context you're gonna give it, all the data you're gonna copy and paste.
And so th if you think about let's say I'm I'm building a
You know, I I I have no relationship to this, but let's say I was trying to build a human capital like H C M.
A bar, like an AI bar. Let's say I was working at work day and I was trying to bring a an A.
It's pretty obvious that the smarts of the bar
would really be related to all of the employee information.
But not just that, it would be the benefits information.
It would be the legal um situation in the country where that person is currently working. It would be the company's um policies and procedures that apply to the So you get what I mean by about like these.
These kind of like the jumps of logic and the jumps of data and the way data is all linked together.
If you want to have a smart AI experience
You'll convince yourself that all I really need to do is get a model and wire it in and build a little pipeline that will suck some data in it and it will whack it into the LM.
And if you think that way, you're gonna be very sad. Very, very sad for a very long time.
Because y you're constantly gonna be wrestling with how do I get data to this thing.
How do we get good data to this thing? How do we get timely data to this thing? How do we get well structured data to this thing?
And so you know, it's a data management problem. Like it's
Getting access to good data, getting access to high quality data, getting access to timely data, and getting it to the LLM to get the LLM to make a smart decision.
That's where ninety percent of the calories go.
Maybe it's a bit like Einstein's thing, you know, it's ten percent inspiration, ninety percent perspiration. Nobody wants to hear it.
Everybody wants to just think about what these really cool models and how smart they are and the next one will be even smarter.
But but really it's just the hard work of getting really good data data to the LLMs to get them to do good things.
It sounds really obvious as you make this case.
It makes me think about at the at the Lenin Friends summit
Mikey Krieger talked about how he had kinda the two types of PM groups within Anthropic.
One was focusing on user experience product and uh the other was working on the model
Research side.
And they realized that all of the success came from the model research work, like making the model
And and the data they provided the model was where all the value came from, not just like optimizing the user experience and they're just putting more and more of their product team on just that versus like
Tweaking UX and buttons and things like that.
Yeah, exactly right.
Something sort of related. I'm just gonna ask one more AI question. I don't want every talk to end up being just all AI.
But
Something that's kind of been a meme recently. I didn't I know you have a perspective on this.
is that AI makes it really easy to build products.
So in the future
If you can easily clone, say, a B to B SaaS app like Salesforce or Atlassian or
Uh, whatever whatever your favorite B to B SaaS app, what happens
to these businesses long term, do they just become
Are they all in trouble?
There gonna be a hundred salesforce competitors.
What's your sense and prediction of what might happen?
There.
Yeah, I think it's really weird. Um, I think people really underestimate
where the value is created in these applications and they just kinda get it completely wrong. And I'm not sure like why that is.
So they give you think about so I spent a long time at LCM so I worked a lot on GR, which many people will know.
And I spent a lot of time at Salesforce, so I spent a lot of time in the CRM ecosystem, the the marketing ecosystem and all the rest of it.
If you want it to be like not charitable.
You'd step back and you'd look at all those applications and you'd say they're all just forms on databases.
You'd say the Jira is a form of a database?
you know, workdays are form on a database, so Salesforce, they're all forms on databases. Like all all vertical SaaS or business SaaS is ultimately forms on databases. And you'd be like, well how hard can that be to replicate?
And the answer is like unbelievably hard.
Like unbelievably hard. And people just think you totally get it wrong because it's not actually just about
you know, the data model. So if you think about the f if it's forms on databases, it's these beautiful user experiences.
that sit on top of data models, right? So whatever the object is, it might be a customer object or a you know a campaign object or some or an employee object, right? You could say that, well, there's some element of lock in in the object, like the object itself, like the fields of the object. I'm like pretty boring, right? That's not very interesting. But sure, maybe
It certainly there's some value in being the system of record, like the default that everybody uses.
There's definitely some value in the UX. Like the the well, you know, I want to be the best
Um H you know, HR facing application for working with employee data. Yeah, there's there's some value there.
But the real thing which is staring everybody in the face is it's all about the business rules.
Like that is what drives
The locking.
Because like why why do you buy workday? You don't buy workday for its out of the box configuration. You buy workday'cause you want to configure it to be, you know, Lenny Inc.
HR processes like it becomes Lenny Inc. Workday. It's not it's not Sean Inc. Workday, it's Lenny Inc.'s workday.
And actually as you the the longer you have the software, the more it becomes that. The more it becomes less and less like work in more and more like your specific company. Which makes sense because it was built to be configured to meet the needs of any specific company and every company's their own precious snowflake.
And as that happens, those configuration pieces that bit that makes the application
um native and a fit for your organization makes it a fit for nobody else's organization and also makes it a black box.
To the point that you don't even understand how it works anymore.
Like if you went to, for example, Salesforce and you said, Hey, could you define all of the you know processes by which software was sold inside Salesforce?
They couldn't tell you that without reading the code.
Of the Self Force instance.
I'm not t that's not a proprietary secret. That's obviously true, because over time that's that literally how sales happen. There is no other way to do a sale other than through their internal tooling.
And so what that means is like is that
It's not the UI that matters.
And it's not the um the data model that matters, although those are both very useful.
It's the years and years and years
of evolution of the underlying workflows of the product to support the the customers, but also the customers evolving those workflows to make them work the way they do.
And so how does that impact AI AI companies, right?
You could say it's easier than ever to build an a a forms on the forms on a database application.
And so I'm like
Yeah, okay. That presumably drives the incremental value of every new one of those to zero, right? So
Probably leads to more power to the to the existing winning systems of record, because there'll just be a gazillion competitors who are just more forms on databases.
How would you ever choose between them? You may as well just go with the winner. You know, nobody ever gets fired for buying Salesforce or whatever, you may as well start from
From the kind of the pr premium vendor. That's one element.
You could go the other way and you could say, I've heard a few people mount this argument
Which I think is really interesting that
Um at the end of the day
Agents are gonna take away most of the u the use of that user interface.
So let's say for example your your salesforce for service cloud.
I've heard people say, Well
you know, a lot of those service agents might end up getting being being replaced with a genic workflows that will mean that, you know, there is no person operating the UI if the UI doesn't even exist anymore.
Then why do you even need Salesforce? You may sort of just have raw database tables and who even needs forms on databases? You can literally just have databases.
But that also doesn't make any sense either.
Because the agents have to operate
Against the rules of the system.
And the rules are defined by the business processes. So think about Salesforce without a head. Imagine Salesforce had no UI.
It would still have those business rules I was talking about.
And those business rules are what define what the agent should do. They're always telling the agent what what it what it should do and how the world can operate, how what is possible, what is allowed.
And so from my perspective
Like this idea that this just completely destroys like the the differentiation of
of these kind of business uh business process as applications.
This seems like a fantasy, crazy fantasy. The only way I could really believe it is if you said
Well like
You know, you could have a new startup that introspected all of the rules that are configured into a salesforce.
to try and reverse engineer what your actual business processes are and then kind of operate on top of that.
But the best place people to do that would be Salesforce themselves.
Or j or A Bassian in Abasian's case, or or Workday in Workday's case, you know, I don't I just can't see a world in which this like
I I think one of two things could happen.
All this moving to AI makes makes those applications even better.
Like even more even more um
I'm a salable. Like they basically get stronger. It makes the strong stronger.
Or it could enable
some new level of like applications that come from a more platform based thing, so less
A domain specific thing like uh you know A C M or ERP or
Um you know uh
engineering or you know, less less of the domain specific stuff. It could enable a more platform like play where
You have more business objects and business objects have rules and you could imagine a world in which
Like there there's kind of a whole evolution of new, more platform like SaaS applications that do more than one business function worth of worth of the business rules and the way things move around in enterprise.
But that doesn't exist today. So you could say that that could that could exist and you could say it could be
Way better than it than we've ever thought of because of AI.
Or you could say that the rich are gonna get richer. Like the the most likely outcome is that the is that the currently dominant companies are gonna get more dominant.
But I don't think this idea that it would just cause a spring up of a whole bunch of new
No, uhstone will more easily challenge challenge the incumbents makes any particularly
It's not straightforward to me how that would happen, basically.
Wow.
That was uh extremely fascinating and there's so much
There I can go in so many directions.
One is
Uh, I thought you would actually go in this direction, which is distribution advantages become even more important.
If it's easy to like today I could sit there and hire team clone salesforce might take
Um
A while.
But I could copy it. But by the time I'm done, they've evolved, they're moving, they're adding features, they're
ahead, right? You're getting to where the puck was.
And so if that's the case
One of the advantages one of the ways to get
anywhere is to have some kind of distribution advantage. Like it's one thing to have Salesforce as a product clone.
Another to get anyone to know about it.
To s adopt it to
Sell it, procurement, all that stuff.
So Phil, do you have a sense of like distribution advantages being even more valuable in that world?
Yeah, I mean it certainly makes sense, like ultimately at the end of the day, distribution's always an advantage,'cause the hardest problem is to even be in the consideration set.
For any given problem, like the world is full of problems.
It's just when people have that problem, they first they don't think they're gonna solve it at all. And when they do think of solving it, they don't think of you. So th the distribution is always like
An incredible advantage.
But it again, like in the world of AI
It seems like distribution is more likely to get hard than easy.
So so like uh you know, if you think about, for example, diminishing returns on cold email, because cold email is getting easier and easier to send even worse spam.
Like it sounds better, but it's
You know, effectively causing everybody to become desensitized everything.
You know, I don't know if you've noticed like
Half the LinkedIn reach outs now are all basically clearly LLM generated spam.
I mean like to some degree it's actually uh
Worsening the signal to noise ratio?
And so I think that a lot of the kind of breakthrough distribution mechanisms that startups often use.
seem to be getting crowded, more crowded just in general.
And more expensive, that doesn't bode well for
Kind of
You know, I'm I'm the not as good salesforce. I'm the not as good salesforce, but I'm cheaper. It kinda has to be something different. You have to there has to be some
Angle upon which you are materially better.
And what I saw happening and what I've been seeing happening, and I think it's been really interesting.
It's a lot of modern, you know, next gen applications
uh bringing data as a first class citizen into the workflow. And I think that that's pretty compelling, right?
So if you look at the next generation of
um you know applicant management um uh products that deal with you know inbound job applicants.
A lot of them now, like the the latest cool ones.
They include your you know, um your time to fill data. Like uh uh they include outcome data of like who's got the best hiring outcomes, who's who over what period of time has the worst attrition, you know, like literally all the way back to the interviewers and that where the interviewers were in the interview cycle. Like so it basically embeds data into the whole um life cycle.
And I think so I think that there are
Kind of these ways in which
Startups can bring
these experience benefits by just bringing a kind of different approach to the world.
that does enable them to capitalize on traditional disruptive innovation. Like at the end of the day at the end of the day, this is just disruptive innovation. It means that it most most companies have overshot their utility, like the average utility
So you can win by meeting the average utility
And being different.
You know, meet the bar and be different. Meet the bar and be different is is the way to cut through.
So that makes sense like that's a half decent playbook.
But if but if you know, if even for those companies
Now they're gonna have all these AI competitors who are using AI to engineer faster to build a competitor just like them, as quickly as possible, and start jamming it into the into the channel.
And it's gonna be interesting to see how this whole thing evolves.
Um you know, it kind of got raced to the bottom.
uh you know characteristics around it. You're probably right the distribution is still.
The hardest part in software, particularly when you're getting started.
Right. So if you have some kind of
Uh clever
uh fair advantage it feels like that becomes e even more powerful.
Say have a platform of of an audience or something like that.
Um you mentioned this ATS product they really like. Is there one you want to give some love to that you think is really cool?
That you like or you want to keep it anonymous.
Yeah, yeah, it's it's Ashby, it's the one all the cool kids are talking about now.
And it's funny'cause like people literally talk about it in comparison to all the even the l the you know, the last generation of of
Modern SaaS ATSs or whatever.
And they talk about it in glowing ways because of the way they put data in inside the actual workflow. So that's
the actions and its outcomes are directly tieable to each other in the application you're doing the work in. I think that's a pretty compelling user experience.
So just to maybe close this thread before I move in a different direction, this point you're making about how
valuable data is and how that's like at the core of being successful and differentiating
In the future, especially with AI.
tooling and product. Any advice you'd give to someone that wants to
Do that.
Make sure you have a
Is it like half proprietary data? Is it like make it a first class citizen? Like what's the
advice you'd give to founders who are trying to do do this, what you're suggesting.
Yeah, I mean I think at the end of the day it's kind of all of those things, isn't it? Like if you have first party data but you can't bring it to bear, then it's not very much use.
If you have third party data and you bring it to bear in interesting ways.
Like the problem with data is like we're all surrounded by all data all the time.
Excuse the that is everywhere.
Right. What really matters is the right data at the right time in the right place.
'Cause we're all humans, right?
And so and so to me
Like there are obviously data advantages and there are even data network effects if you can end up in a situation where you have very valuable sh first party data.
But you but you know, uh in any case it's still about being able to bring the right data at the right place at the right time for those users to for them to be able to get advantage from it.
You know, you know, like a a little um kind of segue, I guess, on on that one is um
Is I spent um
I know I've done a lot of my career. Like uh weirdly actually I'm you know, I've been a product person for a long time.
But weirdly I've ended up inheriting data teams. I've actually run data teams and a lot of different companies, which is weird because they because product managers don't normally own own data teams.
I think it's because I have just like a really massive affinity for data. I've always been really um
Data is calling so data driven?
It was kind of my kind of my jam.
And uh because I
And and in hindsight I look back and I think
Thank you.
Data is a data is the opposite.
Data is more like a compass than a GPS.
Right.
Like if you look at data
As a way of like giving you the answer?
You're always wrong.
You're always wrong, or you're slow.
Wrong or slow, or sometimes both.
Because mostly data doesn't give you the answer. It just tells you if what you just said is like ridiculous.
Oh
You know, there's potentially something there. So it's more like about disproving what whate whatever you think.
And you end up being slow because
If you try and use data for everything
Your brain is ultimately a data sifter or whatever. So the reason your intuition tells you something is because you've seen a ton of data that tells you that this is the most likely answer.
And so
And so being like data driven
Bing Data.
Obsessed?
It's like
It's something you can easily overdo. Very very very, very easily overdo. So it's about right sizing data.
Having the right diet at your fingertips.
having the right kind of view on data.
Rather than
kind of like trying to expect data to give you the answer, or trying to use data as a weapon, or trying to use data as a way to kind of force people to believe you or to go to go in your direction.
But data is kind of at the center of everything and about how to influence and be successful in products you're building and uh arguments you're mounting internally and everything else.
I I love that you went there. This is I definitely want to spend time on here.
You it's
Interesting you say that. That used to be uh data driven.
Like I was I'm Mr. Data Driven.
You created the Reforge course data for product managers.
And also the retention engagement course and Reforge. And by the way we'll link to these you're still
You're still helping with these courses, by the way, they're still running. They're awesome. I love them.
Yeah.
Great.
So we'll point people to those. Uh I love that you're also saying you're like, I think the way you described it to me before this is your reform data driven PM.
I lotta people say this, they're like don't you know, don't just tell don't just do what data tells you to do.
But use your intuition, use it as a guide.
It's hard like on the ground to it operationalise that advice.
What's your s say like, you know, to your PMs and your teams when they have
data telling them, Hey, this exper this experiment is a huge
Success or
There's a huge onboarding opportun uh conversion opportunity here.
I guess just like
What's your tactical advice to folks that have data telling them one thing and
And maybe something else telling him something else.
I I think the first thing I always encourage people to do is to like look at a piece of data. If you're looking at a piece of data
And the result tells you something that your intuition tells you is like insanely wrong. Like probably not right.
First believe your intuition and go and prove yourself.
Right. Like d don't just take it at first glance because most of the time
It's like Occam's razor, the most likely explanation for something that is insanely not intuitive is that it's just wrong. That there's there's a problem somewhere. Now occasionally, sometimes you actually will be right. Like there will now those will be pay to at moments.
Th those are the moments that make it all worth it. Like there are times when you do find the g the the nugget of gold. Like you you're like you're staring at it and like this is it, this was the problem, this was the thing we were looking for this whole time.
But you have to be very diligent about like following it through, like really understanding what you're looking at. Is this data representative? Is this data like a a good sample of the audience we care about? Is it already um subject to some sort of selection bias? Like oftentimes when I see analysis from different product leaders, you're even data teams.
You can drive a truck through it. Like literally drive a truck through it. Like
And and if you if you present data
With authority.
And that data is like ridiculous.
Or the analysis is just full of holes.
You don't just not get benefit for that.
Like you'd lose a whole bunch of brownie points. Like it would be better not to show up.
With an analysis that isn't clear.
Then it would be to show it with an analysis that's dumb.
And I see people self
Am I like on this? Actually relatively regularly.
Um, because they just bring a g a knife to a gunfight or whatever, like they just bring in an analysis that is just not it doesn't hold water.
And they present it and then get shot down live, which is no no nobody's idea of a good time.
Um so so kind of if I give a little bit of additional tactical thing things about that, it'd be okay. Uh if I'm looking at a piece of data.
What was upstream of this piece of data?
And does that look normal?
Like so so this thing happened or whatever, which you're very, very sad about.
What happened before that?
And does that match what you think should have been, right? Right. So what what happened before this moment's situation? And then okay, for that thing that you're looking at, what happened after?
If you if you have an idea of what happened before and after
That gives you some idea of whether or not this thing is at all worth interesting i interesting to talk about.
And then go one click above the um above this data that you're looking at. So it's like okay, these things, let's say it's it's um
Yeah, I'm looking at
Onboarding success. Let's say I'm looking at onboarding success to second week retention or something like that. I'm like, I have found this thing that totally crushes it. This intervention uh crushes it.
If you go upstream and you find out that this intervention only applies to two percent of the inbound onboarding stream, it's meaningless. It's most likely just a random aberration, but even if it was not a random aberration, it's not a useful tool.
Right. And so you and so you're gonna go up and then you you might go downstream and you might find yep, they last for two in the second week, but in the third week they all turn. So basically pointless. Why are we even talking about this?
Or or then then you might step all the way back and go, Okay, yes, those those people do get retained for longer
But their average ASP is smaller. Cause what we really care about, we do care about engagement and we care about more customers, but we want to keep the customers
at a high ASP to reach a certain revenue goal. Like the final goal is happy customers paying us money.
So that's what I mean about like getting uh going a click up.
If you go a click to the left, a click to the right, so before and after, and then a click up.
And you still see the the the thing that tells you the story that you want to tell?
then now you've got something that's very compelling because people want to hear about that. They want to hear well what did happen before, what did happen after, and why is that why is that outcome happening?
But you have to really do your homework and really be rigorous about it.
To to avoid f fighting false gold.
I love that advice.
Uh ASP. What does that stand for, by the way?
Oh, every sale press.
Got it.
This point you made about how a lot of times experiments show positive and then they end up not have being anything.
Uh, I had the head of growth from Shopify on the podcast and they do this really cool thing where they keep holdouts for years.
Of course.
And then uh and then it auto emails them I think a year or two later, hey, check this and see if
Those these cohorts are still
Uh this is still higher or not? And forty percent of the time turns out neutral after a positive experiment long term.
Interesting. It's really funny because uh last thing we did something similar, we had a global hold out group actually that was held out of all experiments. There was a it and so the experiment platform couldn't target that group at all. So ten percent
Of all people never saw anything ever.
Turns out to be really, really helpful because you can always compare them against whatever the experience was for any of the same vintage of of cohort. I agree with you.
But the other thing is I I don't really love some of that thinking process just in general. It's like, hey
You know, um let's say an experiment does show a temporary benefit.
If an experiment shows a temporary benefit, but that benefit does not persist forever.
Does that mean the temporary benefit was never worth it?
Or does that just mean the temporary benefit was an opportunity to reach another level? You just didn't capitalize on. Like there's no I I don't think there's a perfect answer, is what I'm trying to say. It's like
I don't think that the fact that a benefit doesn't last forever.
Mean C failed.
But I agree with you that like not trying to understand well what what is a net benefit been, what is a net lift been
It's also really important too.
Why growth is so hard. Like growth as uh as part of product is so so especially hard.
Marketers, I know that you love TLDRs, so let me get right to the point.
Wix Studio gives you everything you need to cater to any client at any scale, all in one place. Here's how your workflow could look.
Scale content with dynamic pages and reusable assets effortlessly. Fast track projects with built-in marketing integrations like Meta, CAPI, Zapier, Google Ads, and more, A B test landing pages in days, not weeks, with intuitive design tools.
Connect the tracking and analytics tools like Google Analytics and SERush, and capture key business events without the hassle of manual setup.
Manage all your clients' social media and communications from a unified dashboard, then create, schedule, and post content across all their channels.
If you're working on content rich sites, Wix Studio's no code CMS lets you build and manage without touching the design. And when you're ready for more, Wix Studio grows with you.
Add your own code, create custom integrations with Wix made APIs or leverage robust native business solutions.
Drive real client growth with Wix Studio. Go to WixSudio.com.
So you built
The first B to B growth team when you're at Lasian.
Correct.
Yes.
Yeah, yeah, it was.
Makes me feel like an old person, but yes, was a very long time ago. Slash uh maybe, you know, it's a new thing. We it's yeah. It's either a long time ago or it's just we've just recently figured out this is a thing that you could do in a B to B is like focus on growth.
Yeah, it is. It's like it's like when I so there was around about twenty twelve.
Um, and at that time kind of gross hacking was a thing. I don't know, you people don't really use that term anymore. But in in um B to C it was a very big deal'cause people could see Facebook doing their ten friends in seven days and they could see this kind of thing that was working for people and they're like, Man, that's amazing.
And that's when we set out to go, Okay, well, do those techniques work in B?
And honestly they know it it's kind of obvious now that a lot of them do and that it's worth doing
But at the time it wasn't that obvious because it for a lot of B to B companies, I mean you summarised it earlier, Lenny.
Distribution covers all faults.
Like all almost all ills.
can be, you know, filled in by really great distribution. Like if you have a really good ground game.
Really good marketing, a really good ground game. And in kind of jamming your product into the channel, like you're jamming your product in front of people and you're papering over the ugly parts.
With
You know.
uh customer success people and s uh services and consulting and whatever. Then people will buy almost any software.
Um, or you can certainly be successful with with a lot of different software. But back in twenty twelve it wasn't clear of like, okay
Well, if instead you wanted this differently.
And you tried to make software that sell itself.
Is the juice worth the squeeze?
Right. And you know, now I would say that
Pretty clear that the juice is worth the squeeze to the point that people lots of people think about this all the time. Um, but it was a bit of an interesting kind of time at that time.
And that was essentially the beginnings of product like growth. Is that a simple way to think about it? Yeah, the basic thing it's now called POG, but yeah, at that time we didn't even know what to call it exactly. Just growth.
But
So kinda based on that experience, a lot of B to B companies now have growth teams that are investing in growth.
What makes a great
growth team in B to B.
Any
Things any pitfalls you often find folks fall into that you think they should try to avoid.
Ultimately a lot of these
types of endeavors are a matter of balance.
So what I mean by that is um is growth teams tend to go through a set of phases.
Right.
The first phase is proving their value at all.
Right. So so that they call that the gold rush phase. That's the
That's the this thing's probably not worth even doing.
Why are we doing this?
Merry band of people.
out there trying to prove that there's some growth growth vector somewhere, right? So so that's the proof of phase.
And so you know
The advantage of that phase is that it's like life's good because there's usually a lot of gross to be found because nobody's gonna look it before, so life's good.
Uh, but it's pretty random because you're just literally searching across a random search base going, Have we tried X, have we tried Y? Have we tried Z?
Then then kind of once you get that that model going, then it starts to be, okay, um, you know, how do we scale this thing? Like is this just a flash in the pan? Do we just find a little bit of, you know, low hanging fruit and there's nothing else here there? Like was this just a project we should have done rather than rather than an ongoing thing.
So you have to kind of make it um a system. Like you have to prove that that it can be repeated.
And then you have to scale it. Like it has to become a thing, it has to become part of your DNA. You have to be taking a POG lens to everything you do, all the way from you know paid acquisition to um activation, retention, engagement, cross product expansion, upsell.
I mean you name it, like all the different ways you can grow a product by revenue or engagement.
There's many different ways ways to go about that. And so you end up having to scale out and be able to do all of those different things.
And then you have to figure out how you fit in with the rest of the organization because there's other people who build products all day every day. There's other people who sell that product all day all day. There's other people who mark market that product all day, all day. And so you know, gross organizations
are in this interesting space, they're in between everybody else.
kind of in everybody else's sandpit in a l in a little bit. In a in a little way.
And they're kind of at the edge of everybody's kind of full time job.
And they are very valuable, but they can be, you know, complicated because of all those relationships and because of the way they sit amongst amongst all of the other parts of the organization.
So so many organizations fail because they don't they don't really find much that wins, or when they do find wins, it just seems totally random, or they do find a lot of wins.
But they all can't understand them because they're just they seem like they're just a random walk through a bunch of the a bunch of potential opportunities.
Because there's many different ways to
can fail to fit as you go through your growth phase from trying the ideas to
Success to scaling to operationalizing.
One of the biggest memes along these lines is
A lot of
Companies
claim there's like just PLG rarely ever works.
You always
Event.
Either
You try it and it just doesn't work or it eventually just peters out. I guess
Any thoughts on just like what are signs that your product
has a chance to work.
P.
product led growth versus you're just just go straight to sales immediately and don't even worry about this.
First let's examine the counterfactual, right? So start with the opposite of your question and say hey
You know, how would the world be sadder sadder?
If we all just gave up on PLG. Like if we just said, Hey, just there's no point in doing it in in B to B sense.
The problem is that there is not
A natural
uh force
That pulls
Companies
Towards thinking about
Um the end users
Enjoyment and success.
earlier in the early in their journey. There there is no natural force, there's no natural countervailing force. Why is that?
I mean
101, the buyer is the most important person, the economic buyer is the most economic person, that their their needs are the number one thing. They're usually the person driving the RFP. They're usually the person dealing with the sales organization. So the needs of the person who you hear are usually all feature driven.
And they're not from the end users. And so you're kind of sowing a seed of your own demise
If you don't think about that end user.
But it's one thing to say that you should think about the end user. It's a whole other thing to have a system by which you do that.
Because people people pay lip service to all sorts of things, but you know
My uh you know
I'm sure you've heard this one before, but in economics, like people only do what their incentives told them to do. Like broadly speaking, that is what they do. That is what happens. You get what you set out to measure, you get what you give people incentives to do. If there is nobody in the organization whose true incentive
is to measure that success, the the MGs of success, their enjoyment, their happiness, their retention, their engagement early on.
It will not happen.
Or m or at best it will be a hobby.
Thanks.
And so then by extension, if I start from there.
Let's say okay, let's say it doesn't exist.
P O D doesn't exist and therefore it's a hobby.
And therefore there will be a bunch of hobby people who care about this.
Then you ask yourself, okay, w will that mean that
there will be many products for which like those experiences really suck.
And does that mean that there will be an opportunity for competitors of those products to be better at that? And is that a differentiation differentiate differentiated competitive advantage?
Yeah, I say it is. I'd say it is.
And so they're kind of working I just work my way backwards.
And it and I go, Okay.
You can say that your P or G investment might be too high. You could be like, Well
If I invest more
I won't get any more juice. Like this is not it like I can't spend my life
Just experimenting in the onboarding. And that's very, very true.
But it's very hard to argue it should be turned to zero.
And so so to me therefore it's about the balance. It's about okay, how does POG fit
with the other different ways that that I grow my business. So a conf, for example, we have a POG function, we do grow
Um with um, you know, self serve sign ups, people who sign up.
Literally their credit card.
Like lots of them sign up and they're very successful, never speak to us, you know. We also have like a uh enterprise sales team.
That sells you know directly to very big companies, you know, some of the biggest banks in the world, you know, the people you would definitely know of.
I don't think it has to be one or the other.
I think that you know
It's about a balance, it's about getting the motions to work.
And for really sophisticated companies, the people who really nail this
It's about making both motions work together.
Like if you can if you can get a POG motion work to feed your sales team.
And a sales team motion work to feed your POG funnel when the sales leads aren't aren't ready yet?
And kind of you can get those motions into playing with each other.
you can make a lot of money. It can be an extremely successful way to go to to build a very resilient business.
Why? Because you get a lot of customers.
And you get a lot of revenue.
Like, you can't be that successful as a company if you have a lot of revenue but a small number of customers because
You're captive?
Everyone everyone knows that.
You can't be that successful as a company if you have a lot of customers but not enough revenue because you just don't have enough money to sustain operations. So the magic is in having both a very large number of customers and a very large amount of revenue, it's very hard to knock over a company like that.
You know, if if I look back on my time at atten
I think they they shared their most recent numbers. I can't remember what it was, but it was in the public.
Data or whatever. Something eighty thousand or hundred thousand customers. Something like that.
Like that's a lot of customs. That's a lot of customs. You you you you let's say you're going up against uh Jira?
And you're like, Yeah, man.
I'm gonna pick off a thousand customers.
Right, from from a lot, right? That's a lot. Obviously a thousand customers is a lot.
You only have nineteen sorry, it's it's gonna be like uh
You know, eighty nine thousand to go or seventy nine thousand to go, or however many it is to to to go.
I can't remember their exact number of customers, but like
It's very hard to assail a company.
Which has a very large number of customers and a very large amount of revenue.
Uh and so that's why I think that POG as a
as a mechanism is incredibly important.
for almost any type of company, if you can make the motion work. Like obviously there are there are companies for whom the motion just is irrelevant.
But for those where it does matter.
It seems like the juice is worth the squeeze.
That was an awesome answer.
Uh I looked up last scene, they have three hundred thousand customers. Oh man, I'm so far off when when I left eight thousand customers.
They've done good work since then.
Yeah.
Also, you're talking about incentives and how the
The power of incentives. Charlie Munger has this great quote I looked up just to make sure I get it right.
Show me the incentive and I'll show you the outcome.
Yeah, exactly right. Exactly right.
You know, I've I've seen I've seen like um
Cases where like a sales team was
People trying to get a sales team to do like a POG emotion.
And you can beat them over the head as much as you like. You can get into a meeting and tell them that you really, really want them to do this.
But at the end of the day, like they're not gonna do it. And the same is true for every other
kind of like function. It's just the nature of things. I have some newsletter posts around the stuff if folks want to dig deeper. Also, um Elena Verna had an awesome podcast episode talking about product like sales.
And kind of the combination of these two things that we'll point to. Just like a whole other topic we can go deep deep on, but we're not gonna do that in this episode.
Maybe just one more question.
So you mentioned all
The companies you worked at.
So you've been at Salesforce Chief Product Officer
Uh Mule Soft specifically.
within Salesforce, uh
Metromile, Lassian, Confluent now.
A lot of really interesting and different roles.
How do you choose where to go work?
And how do you choose an o which opportunities to take, I imagine you have many options.
I have to think of my career and certainly in hindsight looking at it this way, Lenny, so like I'm just
I don't know, if forward looking it was obvious to me this way.
But looking back, it my career's been a little bit like a bingo card.
Like I've always been looking for
to fill in boxes I didn't
Have failed?
Because I felt like that would make me a better professional. Okay, it's it's like
If I didn't know anything about that specific type of sales model or that type of marketing or that type of
product management or that type of product or that layer in the stack or that
kind of thing is like well if I learn about that thing, I will become more versatile.
So actually two things. I will it's fun. It's fun to learn something new. It's fun to prove to yourself that you can do those new things.
And then it makes you more versatile because it means that any given problem you go up against
You've seen something that that
Pattern matches to it.
Like it kinda feels like
You end up bringing a gun to a knife in a way.
Because every problem you look at, you're like oh I have seen this from the other side, like I've seen this from some other angle.
And so I know that this is likely to work and this is unlikely to work.
And so, you know, when I when I joined, you know, early on early on in my career I was working for a big enterprise software company.
Sorry, small enterprise software company that sold to the Fortune One Hundred. Then I joined Atlassian and like I shared with you
We had no Salesforce at all, actually. At all. Literally nobody to sell the software. It sold itself or it didn't get sold at all.
And we grew to have eighty thousand c customers, like it was just pure product that growth and just an incredible company.
Then I was at MetroMar, which was a consumer company um that got acquired.
made an insurance product for end consumers. So they
Got nothing to do with technology products, like literally a
uh complicated, um
Internet of Things device you installed in your car, but ultimately it's an insurance product you'd sell to grandmothers in in Florida.
As much as you would urban millennials, so
And then uh we also have to totally back end software that's used by IT organizations and a consulate.
Uh you know, infrastructure that's used by the U.S.
Um developers everywhere to build really interesting data driven applications, data powered applications to do all sorts of things in real time.
And you look across all that and you go, It's all a bit random.
Right. Um
But like I didn't see it that way because
Like I learned you know, I actually was in sales for a bit. So so I was a I ran a pre sales engineering group.
Went around the world selling software.
So when I joined the Bastian, I wanted to kind of understand what it was to sell software at massive scale.
With no sales team. Like can it even be done?
Right. And so I learned a lot in my time at NASCAM.
When I went to MetroMarle, I'm like, Well, I've never built a consumer product before.
Like I can say that I've actually built a product that's touched many millions of people'cause Jira has, so I felt pretty good about that. But I've never built one that I could say, yep.
A consumer, your average consumer can use this thing. It's so simple, even my grandma can use it. I'd never build a product like that.
Uh so I got that experience at Benchmark which is really fun.
I'd never worked uh inside an organization as big as Salesforce.
Or an organization with with as good a sales motion, like the you know, you talked about distribution earlier.
Salesforce is an absolutely insane distribution machine, like just an incredible company.
within just an amazing distribution uh network.
And a fantastic marketing like approach that it's like
It's like a PhD in marketing, you know when when you spend your time uh at Salesforce.
You're like this company is just one of a kind, like it's a one of kind and it's so outlandishly good at one specific thing.
So looking back, you know
All of these jobs.
Have been
When I say a bingo card, like I've just got an outlandish education
In these areas that
You know, are not obvious at all. And once you've seen them
They're like superpowers. They they're superpowers to be able to bring that to bring that same experience to bear to bear on things.
And so one thing that I really
I'm trying to figure out as why often people don't do that.
And oftentimes people
Um stay in
A very specific domain? Like they prefer to stay in a domain.
They prefer to stay in a specific
kind of a type of company or a very s or a role that works in a certain way, like companies that have the same operating
model or they plan the same way, or they kind of they try to stay with
Things that are pretty similar.
But it seems obvious that
the most likely way to that to kind of really grow is the opposite.
It's to constantly be
Choosing things that are that are outside that. Not totally outside the lines. Like don't
Don't jump out of a plane if you've never parachuted before. Like obviously you want them to be in some way
And adjacency.
You know, that you want them to have something in common with with what you know.
But you want them to stretch you and change you?
You know, I uh
I had like a really kind of uh
Transformative experience.
Many many years ago when I was at um when I was at Alasian
And a guy called Tom Kennedy, he was our general counsel, so like chief legal officer, basically.
And a lifelong lawyer.
Very smart guy. I liked him very, very much.
But like just a lawyer. Corporate corporate council. I'm sure you know what they're like.
And really great guy.
And I remember so mostly in our meetings, like our meetings
He didn't
Talk that much, except about legal things, right?
But I remember um in one meeting we were having this vigorous debate
About a product strategy question.
About like what we should do. Should we go go should we go left or should we go right?
And like as usual, he's dead and he's mostly just staying silent.
And eventually the the conversation's been going on for fifteen minutes and he's like, Hey
Everybody.
Like a year ago we talked about X, Y, and Z, and he proceeds to lay out our product strategy at that time.
Anything.
Just recently we said the following things and that was a product strategy, whatever. Now you're saying this, isn't it obvious that that isn't this like what you guys are saying is not congruent with that.
And if you really meant what you said back then, we should be doing X.
Again, like the room went silent.
Everybody kinda turned.
To him?
Kinda nodded.
And then everyone
Yeah, okay, I guess we probably should be doing it differently. And so like the meeting stopped.
Like when the G C randomly mentioned that he like deeply understood our product strategy.
And he knew enough.
To make
be able to contribute in that way. And so the the the life changing part for me about that was just this realization that
If I'm gonna be a really great professional
If I'm like the type of professional I wanna be.
Is is that type of person the type of person who
can contribute to the whole company in all sorts of ways, like doesn't spend all of their time in everybody else's business.
But understands the business and has the
you know, mental horsepower and the experience
To be dangerous.
in all sorts of and I mean that in a compliment way, I don't mean that in a negative way, but to be dangerous in all sorts of situations. I think that when you have kind of like leaders like that behind you and with you,
Then you're just unstoppable. You're an unstoppable force in business when you kinda have that that uh motion happening.
Wow, that was an awesome story. And an awesome
Uh
Perspective.
Similar to the advice I always give PMs of
People always wondering, should I like go deep on a specific subject? Should I just
try different things and I find
just variety, especially earlier in career is really
powerful not just to help you discover the thing you like, but also to your point, just
using insights from all these different parts of the product and
Like internal tools and trust and safety and platform and
Consumer product side and
Growth and
Just course, uh
Like the more that you have, the stronger you get.
And I think I feel like another benefit of your approach is if you if you work at just B2B SaaS companies, you're never gonna like if you have too many of that on your resume.
It's very hard to get hired a consumer company.
And so just having it creates a a huge optionality for you if you do.
Which you did.
Yeah.
It it's interesting'cause people used to talk about people who are T shaped or whatever.
I've never really loved the analogy.
Because it's more like people are scribble shaped.
Like I mean like there's the really best people you've worked with, they're more like scribbles than they are um T shaped, because of course you wanna be horizontally capable, so you wanna be be broad.
And you do want to be deep.
You actually want to be deep in way more than one thing. Now obviously when I say deep I don't mean like
Like I'm not
able to do the job of like, you know, our finance function all day every day.
But I'm one hundred percent good enough to go like
three clic below like the simple financial analysis. Like I can
Go reasonably deep in our financials.
Because I want to and because it's partly like it matters. Like it's it's important to be able to do that.
And so
So maybe a different way to think about that bingo card is like
I've really regretted
Going deep in something that isn't quite my job. Like I've really regretted it. Like the worst case scenario is I've learned something new that I will never use.
Which you know.
I guess at least that made my brain slightly more agile. Like I don't know. There must be some potential benefit of that.
But the very best case scenario
Is that when I'm least suspected
At some point in the future, it will turn out to be the thing that matters. Like it will be it will be the tool that I need.
But I'm facing some important problem.
And I will be like, Oh my God, this was worth every cent.
And so like if you think about it on an ROI basis.
Doing things that aren't in your wheelhouse, like that aren't the things directly in front of you.
The ROI can really be outlandish, like it can be off the charts, great.
But of I guess it's speculative,'cause it's you know, you don't know you're gonna need it tomorrow, you don't know it's if it's gonna be something that's gonna be a regular tool you use.
It's interesting use of the bingo card as the analogy. What are you trying to
Is there a bingo moment?
At the end of this, it was a retirement.
Mm-hmm.
Is there Oh y you mean like you've got everything? You've got like a big Pokemon.
Yeah, we collect them all.
Yeah, I was I was working with um, you know, somebody at Salesforce
And you know, he was like a really he'd been there a long time, very, you know, um, very, very, very successful person, like honestly, uh, you know, didn't need to work anymore.
And and he said something that I found really useful. He's like, Well, now I'm at the point of my life
Where I want to work at the intersection of things that I am good at.
And things that will be valuable to the company to do. So basically like it feels like the reward of completing a bingo card.
is actually to just get to spend more time
doing things that are leverage that you enjoy and that are high leverage.
Uh and so that seems like a good outcome to me. Like if if
It's not as though you're gonna
I don't think most people are gonna like work and hopefully have some sort of great financial outcome and then go, Well that's it.
Picking up stumps, I'm retiring.
Uh I think for most people achieving some sort of financial outcome or some sort of um
You know, uh
independence or whatever is really just another stage. It'll be it at that point it will be Okay, well now what do I do? Like what do I do with my life? Like why
And so that was why I said earlier that at the end of the day
Product management is like
At times the worst job in the world.
And at times easily the best.
Like and and it's both.
And it can be both. And so, you know, I it's hard for me to think about what
Yeah.
if I think about the things that are the intersection of what I'm good at and and are valuable to the world
Product management is a pretty fun one to do and it's different every day.
So I think we're pretty privileged for those of you who listen, I mean, obviously your podcast reaches a lot of product people. Like I think we're pretty privileged to be able to operate at that um intersection.
But it's not easy because um
you know, you gotta show value. You know, it's like it's not it's a very complicated job to show value in and to
Demonstrate value to the world and uh
It's constantly being attacked, like you you mentioned, but it's still amazing that when when it all goes right, you know, when when a product is very successful in the market, it's hard to describe the joy you get from from that.
Kind of along those lines to close out our conversation before a very exciting lightning round.
I wanna take us to Failure Corner.
People here listen to these podcast episodes and everyone's always just sharing all these wins, everything's always going great. The CPO of this, CPO of that, just moving on up.
And they people really want to hear times when things didn't go right because that's those are stories
People don't share as often.
Can you share a story when something didn't go right, when you maybe had a failure in the course of your career, and if
You learn something from that.
Experience what you learned.
I mean, there's a lot of things that didn't go exactly to plan any. Um, like very early on in my career.
I uh you know I was a cell developer and I accidentally
Um
deleted like one of the core systems of the of the company that I was
Working at um so that's that's gonna go down in in infamy, but luckily that one's far in the rear view mirror, like that's it.
No, that was pre that was far pre Avasian, but very bad.
Uh yeah, you know, the the one I like to talk about, I wasn't I wasn't directly responsible for it, but I feel like responsible for it. I was at a company
And we launched a product
That was one of those products that um, you know, in hindsight should have been really obvious it was going to fail.
But for some reason we were all blinded by the potential.
It was a it was a product
That um
That was about it was basically for to measure the environmental impact uh of your company.
And to help you reduce the environmental impact of your company by doing think about it as like power management, building power management, managing the power drawer of computers, managing the power drawer of you know AC and all of that stuff. That was the vision, basically. It's like a kind of a manage your environmental impact of your business.
Kinda the idea was pretty cool at the time, and also it was the right time for that. And it's still a thing. It's still a area of active research and investment or whatever.
But it was like one of those things.
Talk about the wrong company.
Wrong place, wrong time, wrong distribution. Like we had literally no
right to win. No right to play. Like just absolutely no business in hindsight being in that.
In that business.
And I feel really bad because I again good idea, wrong wrong company.
And and and at the end of the day, uh we launched the product.
We actually kept the product in in market for two years.
And the and the final the final straw
was weird, the final straw was actually when a c when a customer finally wanted to pay for it.
Like it it had been a market for two years and we found ourselves with a customer who wanted to pay millions of dollars for it. They were ready to sign on the dotted line.
And that was actually the moment we decided to kill the product. Because where they if anybody if this per person signs this piece of paper, we are stuck with this forever. Like this this one this one customer will be bound by contract for however long or whatever. Like so we actually ended up killing it the at the moment
After two years of of um of failure when kind of somebody wanted to pay money for it.
And I I look back on that and I'm just like
Man, like that was a really big I feel really bad because um like it should have been obvious.
It should have been
It was obvious.
And we should have been able to call a spade a spade and I guess speak truth to power.
Um
But instead it kind of got through to the keeper and turned out to be a real accidental drain on resources for years and just a big mistake.
So is the lesson there
Uh just be real with yourself. Just
Yeah, I like I like that you have this forcing function of like, Okay, let's get for real now.
Is it like I wish we had a
earlier forcing function to force us to make a decision.
Yeah, yeah, I th I think if if I could do it differently, like I probably I might not have necessarily been able to one hundred percent change the decision, but I should have tried.
Like I mean, it was pretty obvious after six months, like this thing was like a bit of a zombie product walking.
And it would've it you know, it would the least I could have done is said, like this thing is dead. Like we should have called it dead way earlier, but instead we proceeded for another year and a half.
uh investing in it. And so that that's the bit that makes me kind of feel like a real bummer about it.
It reminds me of a recent episode of the Roz
Who is the CMO at Wiz.
And she joined as the first PM and a few weeks into it with doing tons of calls with customers, she's like
I I think I this I need a quick'cause I don't really understand what we're building. I don't get it.
And everyone's like
You know, like I don't even I don't either.
And it just not Yeah, the conference founders just had a vague idea what they were doing, but they didn't really have an idea.
And uh just spark the okay, wait, you're no one actually does. That's actually yeah, more concrete. And it helped them pivot and now
I don't know if you know about Whiz but they ended up being the fastest growing
Start up in history.
Yeah, see, isn't that amazing, right? You know, like it doesn't it's not it doesn't mean it's permanently fatal, but asking that question and kinda going through that, uh
Reckoning? Turns out they came out strong came out stronger.
Scary, but it turns out it's for the best often.
Before we get so very exciting letting you round, is there anything else that you want to mention or leave listeners with, maybe a last nugget.
Something that you think might be helpful.
Before we
Right.
Maybe a couple of different things that that that that I think uh sometimes well understood, but just repeating them, I guess, because they're very valuable to me.
One is um
that uh like if you let your calendar roll you then nothing good will happen.
Like, you know, I know people talk about that a lot, but it's surprisingly common in product management in particular that people end up ruled by their calendar. And so it's related to that whole look at spend eighty percent of your time thinking about things going on outside the bus outside the business.
Easy said, very hard to do.
And if you don't do it, no one's gonna do it for you.
And so like it's really hard to be successful unless you find a way to force that to happen.
Oh so repeat that.
Oh kind of also like somebody said this to me and I never actually looked up the quote.
But apparently Colin Powell said
That if you're making a decision with less than thirty percent of the available decision thirty percent of the available data
You're making a big mistake.
If you're making a decision only after you have seventy percent, it was either seventy percent or seventy seven percent, I can't remember the exact number, when you have seventy eighty seven percent of all the available data, you have waited far too long.
Right.
And that's weird. I've always found that very insightful and a bit relates a little bit to what we're talking about about data earlier.
But at the end of the day, they we get paid in product management to make decisions. Good decisions. Paid to make good decisions that will deliver business benefit.
And a decision with too little data is fatal.
A decision that takes too long and collects too much data is also fatal.
So like everything, it's about trying to find the balance of all of these different things to try and deliver business advantage.
A great way to circle back to all the things we've been talking about.
With that we've reached our very exciting lightning round. Are you ready?
Yes, let's do it.
Let's do it.
What are two or three books that you have recommended most to other people?
Yeah, um the the oldies but goodies is probably gonna be uh the main startup that I still find actually really good and the kind of key lessons in there I still think are very applicable to a lot of people, particularly the cohort analysis bit, which for some reason I still don't see people do anywhere near enough cohort analysis. So there you go, that's my little tip. And then uh Inspired, how to
How to build products that people love by Marty Kagan and the Silicon Valley Product Group. That's an oldie but a goodie
Think you know it's got a lot of the key lessons of product management in it, even though
Been for a long time.
Those are some classics. Very cool.
Do you have a favorite recent movie or TV show you really enjoyed?
I'm watching um program like just a
I I don't get to watch very much T V. Mostly at night I like to watch things that are extremely light.
that I like just don't at all
Inspire any element of stress.
And then they're very short. So I'm basically short and funny is basically my thing.
And there's a new program on Netflix, I think it's called Detroiters.
Oh I've been watching that.
Yeah, it's really funny. I really like that. It's so ridiculous, but very funny, so
Like that. That main guy, he's so funny. I forget his name, Tim Sweeney or something like that. Yeah, he's so good. Uh good one. I've been watching that. I'm loving it. It's like very quirky. I think the New York Times quote on there is like the very weird the quote so weird. Like in the first episode, I'm like, what is this show? It's not even clear what time it's set in, and like it's very weird. It's really cool.
Yes. Good way to describe it.
Uh next question. Do you have a favorite product you've recently discovered that you really love?
Uh yeah, this one's like some of your some of your listeners might be using it, but Glean, like it's a pretty well known startup now. They recently raised a ton of money.
We've been using um Glena consulate for a long time.
Uh and it's just
Ah oui.
Like it's just amazing.
Yeah, I can't I can't describe how good it is.
And I don't say this lightly because, you know, I think search
Like a business search is probably one of the hardest problems in computing, actually. Getting around is one of the hardest problems in computing. Amazing. Not often I use a product, and I'm like
This thing is like ten times better than anything that's come before it.
It's one of those for me.
What's the simplest way to understand what it does for you?
It searches all of our organization's knowledge.
So like you the the thing you were just saying before, you're like, um, you know, what does uh ASP mean?
Right.
If I had that in the meeting, I just open my n my new tab, it'll automatically take over my new tab, or just be like, what does ASP mean? And it will summarise back to me what ASP means and it will give me a link to all the documents inside our company that describe what ASP means. And then it will tell me who the expert in AST at our company is.
I it's I just
It's like having a second brain.
It's like an insanely cool um kind of organization searching.
Great tip. Okay. Two more questions. Do you have a favorite life motto that you come back to, share with folks, find useful and work your life?
Uh I think about this one a lot. Um, you know, when I started off in my career I was a very I was a you know uh an engineer's engineer. I used to very much about like technical correctness and what computers were capable of and
Kind of.
Technical righteousness, you know, the right answer rather than, you know, there is only one right answer and whatever.
It's a long winded way of saying that I often think about this phrase, which is um people don't care what you know until they know that you care.
And uh and so I've realized that really being able to influence people it doesn't matter about whether or not you're right. Oh whether or not you're wrong.
And at the end of the day, it's first about trust.
And about relationships.
And caring about what each other's outcomes are, what their incentives are, and all good things sit on top of that.
If you once you have those t kind of foundations, then you can build like really good partnerships and that's where you know good progress comes from.
Wow.
That is so good.
Yeah.
Connects with like radical candor similar.
Like in theory of just caring they need people need to feel like you care deeply about them before they
Take your advice.
And then also connects with this parenting book I'm reading called Listen.
that a previous guest recommended, which is all about
How your kids
Call it have problems when they feel like your connection to them.
It's weak.
And so the solution is to build a stronger connection for them to know that you care deeply about them. So this is really
Connected so much of what I've been reading.
Yeah.
It's a great one.
Final question.
You're born in Sydney, folks can maybe guess by your accent.
If someone were to visit Sydney, any tips, anything they think you think they should check out, favorite thing in Sydney?
Yeah, Sydney's a really beautiful city and like it's kind of famous for its beaches and it's a m basically a metropolitan city. People probably be very surprised when you visit it. It's a very big city, very metropolitan. A little bit like New York, but New York with really beautiful beach. If you want to think about it that way, it's kinda crazy.
Uh but there's actually like a ton of really cool nature and beautiful things all around to me.
And so if you wanna do something like off the beaten path.
You can actually go to there's an area called the Blue Mountains, which is like an hour and a half drive from Sydney.
And you can sail down a waterfall.
Which is well actually firstly you go canyoning through through a through a canyon full of water. And then you have sale of waterfall at the end.
And if you're looking for like just a really beautiful, fun kind of adventure like thing, an hour and a bit away from a massive metropolitan city.
That's my my sort of happy place, like really beautiful outdoors stuff, while also next to a beautiful city.
And you said you sail, what sort of sail off of waterfront? Repelling, I think. Yeah, lowering yourself down on a on a rope or
Got it. Okay,'cause when I hear sail, I'm like thinking a boat just
Jumps through over the waterfall. Oh no, absailing, which is also I think in the States you guys call it repelling. Repelling. Yeah. Wow. Yeah. Very cool.
Sean, you're awesome. This was extremely cool.
Thank you so much for being here. Two final questions. Where can folks find you online if they want to reach out? Also point folks to your reforch courses that
You created and uh
Final question, how can listeners be useful to you?
Sure, yeah, uh so my Reforge courses, you can check them all out at reforge dot com. As you mentioned, the retention engagement course and the um data from product managers course. So you know, love to see folks get some value from that. Lots of people have been through those courses already and
I really get a lot of value from it because like I said, one of my goals is to like
help all of us be better product people. I think our leverage could be massive.
Um, where you can get in touch with me, obviously LinkedIn, but also Sean M Clouds on X.
If you want to get in touch.
Uh in in terms of, you know, being useful to me.
I mean, j broadly speaking, they I'm always open to new ideas. Like if people have ideas about how to do better B to B
Um P or G better B to B um in a product led sales, for example.
better better ways of uh going about distribution and product led sales and product led growth inside enterprise companies. Hey, um
I'm hoping to learn myself. We're all we're all in one big journey learning how to do this better.
Mm.
So true.
Sean, thank you so much for being here.
Awesome.
Thank you very much. The name was great.
Hi, everyone.
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.
Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast.
You can find all past episodes or learn more about the show.
at Lenny's podcast dot com.
See you in the next episode.