This will make you a better decision-maker | Annie Duke (author of “Thinking in Bets” and “Quit,” former pro poker player)

Transcript from https://podmenti.com/t/53131019d76d68f4

It's so incredibly necessary in improving decision quality to take what's implicit and make it explicit. It's not that intuition is crap. Your intuition is sometimes right. If you don't make it explicit, then you don't get to find out when it's wrong. When you look at companies that have read your book, what do you find are the frameworks or tactics that really stick? People generally think the purpose of a meeting is for three things.
Discover, discuss, decide. The only thing that's ever supposed to happen in a meeting is the discussion part. Something that comes up in product a lot is this idea of pre mortem. So a pre mortem is great only if you set up kill criteria. Commit to actions that you're gonna take if you see those signal. You have a very interesting framework for how to think about decision quality when the outcome is very long term.
There is no such thing as a long feedback loop. And the way you choose to shorten the feedback loop is to say, what are the things that are correlated with the outcome that I eventually desire.

Today, my guest is Annie Duke. Annie is the author of the best selling book Thinking and Bets, and also her more recent book, Quit The Power of Knowing When to Walk Away.
She's also a special partner at First Round Capital, which we spent some time on and is incredibly fascinating.
Prior to this part of her career, she was a professional poker player, she's won over$4 million in tournaments, including winning a World Series of Poker bracelet, and she's the only woman who's won the World Series of Poker Tournament of Champions and the National Poker Heads Up Championship.
Currently, she spends her time helping companies make better decisions.
In our conversation, we cover the many lessons that she's learned from her friend Daniel Kahneman, who recently passed away.
What simple change she's found has the most impact on a company's ability to make
Better decisions.
How to make better quick decisions when the feedback loop is very long, and also why she doesn't actually believe in long feedback loops.
How she changed the way that the partners at First Arm Capital make decisions, which is incredibly interesting.
Plus, why when you're thinking about quitting, that probably means you've already waited too long and you should have quit a while ago.
I learned a ton from this conversation, and this is definitely gonna change the way I think about a lot of things. With that, I bring you Annie Duke after a short word from our sponsors.
And if you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. It's the best way to avoid missing future episodes, and it helps the podcast tremendously.
This episode is brought to you by Vanta.
When it comes to ensuring your company has top notch security practices
Things get complicated.
Fast.
Now you can assess risk, secure the trust of your customers, and automate compliance for SOC 2, ISO 27001, HIPAA, and more with a single platform, Vanta.
Vanta's market leading trust management platform helps you continuously monitor compliance.
Alongside reporting and tracking risk.
Plus, you can save hours by completing security questionnaires with Vanta AI.
Join thousands of global companies that use Vanta to automate evidence collection, unify risk management, and streamline security reviews.
Get one thousand dollars off Vanta when you go to vanta.com slash Lenny.
That's V A N T A dot com slash Lenny.
This episode is brought to you by User Testing.
Get fast feedback from real people throughout the development process so that you can build the right thing the first time.
Companies are being asked to do more with less. They need to move more quickly while building experiences that meet changing customer expectations, all while minimizing risk and costly rework.
With user testing, you have a trusted partner who provides user research teams with the skills, tools, and data to be able to articulate the value of user research to the business so that you can make an impact and build the best experiences with confidence. Get started today at usertesting.com slash Lenny.

Annie, thank you so much for being here and welcome to the podcast.
Thank you so much for having me.
It's my pleasure. I was telling a bunch of people that you were coming on this podcast and every single one of them was so excited that we were doing this.
So I'm
Quite excited that this is happening.
I wanna start with a question about Daniel Kahneman.
I know you see were very close. You did a podcast together. I know you grew to be
Good friends over the past few years.
Sadly he passed away about a week ago at this point. Yeah.
I'm curious what
Are some lasting lessons that you took away from your time with him? Maybe one or two?
Things that stick with you and you think will stick with you.
It's hard for me to describe properly
how humble this man was.
And it seemed so impossible because
His influence i was so huge, you know, and his intellect was so huge and his
Insight was
was so
Change making.
you know, in in the areas that he worked.
But he was so humble.
He really wanted to hear what you thought he
He said I don't know a lot.
He would change his mind?
You know, it's funny, I I got asked uh in a class that I was teaching
About um
Thinking fast and slow.
And someone was talking about, you know, how
There's
a lot of uh studies in there that didn't end up replicating.
And
What I've seen from a lot of other people in the field who who really who where their work is defined by some line of work that didn't replicate.
Where they're really
arguing against
The fact that it doesn't replicate just sort of
trying to justify or say but no, or this and that. Like this real lack of open mindedness to the idea that
You know what, maybe
Maybe it turns out not to be true and that should be okay.
But the thing about Danny was what when you talked about it
He was the first one to tell you, like
I wish priming wasn't in the book.
Mm.
You know, like he was just so open minded to the idea that that he kind of
He no
He knows what he knows now, but he knows that he doesn't know.
Everything.
Yeah.
A another thing that people don't know about him that outside of academics I think are that he was one of the pioneers in something called an adversarial collaboration.
So one of his favorite things and it he sort of in
Someone's invented it.
was find someone who really disagrees with you.
And then write a paper with them.
And try to figure out what with them
What's the study that would r resolve the issue?
So an example of that was Danny did work on happiness as it relates to
Well
Connuman.
Uh
And a collaborator, I I I
I guess
Dayton.
They published a result that got a lot of press.
About
the relationship between money and happiness and the idea was money doesn't buy happiness.
beyond seventy five thousand dollars. So that that got published in two thousand ten. I'm sure if you ran that exact same study it would be
more than seventy five thousand dollars now because you were to just for inflation.
But it was kind of like
You know, the idea being
Once your basic needs are met.
and you're not that worried about paying rent and so on and so forth, it's not gonna have that big of an effect.
But then a decade later, Matthew K Killings were
said
That's not true.
That
Actually
Money and happiness are kind of
Correlated.
through all the income levels. The more money you have, the more
Happy you
Tend to be.
Okay, so this is like a classic example where like
People will tear each other down sometimes and
That guy's wrong and whatever.
And instead it was
Kind of it's like, all right, let's do it together.
And they joined forces along with Barb Miller's
Um
to basically try to you know to try to sort of resolve
That issue.
He was just always seeking out. Like he he it was just like everything was like why am I wrong?
And he was eager and he was excited and he wanted to
You know, I remember having a lunch with him
Where
I realized after the lunch that
The whole lunch was him asking me questions about my work.
Like why is he asking me questions about my work?
Right? Like he's didn't gone a minute.
But he was just so curious about other people.
And he just made time for everybody.
And he just he just loved the people around him so much.
And I think that in in the end, like that that's
It's hard it's hard to believe. I I just think that that dwarfs his body of work, which is spectacular, but
She is a human being, which is so spectacular.
And I I do think you hear all the time like
About people who are
so incredibly brilliant. You know, it's sort of the brilliant but but asshole.
Kind of thing.
And it was just so the opposite with him.
You're so kind.
Mm.
It's amazing. And it feels like there's probably a strong correlation between people that are
Curious.
Always looking for how they're wrong.
And
Finding time to spend with people to kind of
Debate and discuss and ask.
feels like there's gonna be a strong correlation with that and and finding really interesting insights on covering
New things.
Yeah.
Yeah, yeah.
Well thank you. Thank you for sharing all that. Um I I wish I had known him.
Let's talk about
Decision making.
So there's kinda like two areas I wanna spend time on, as one would not be surprised decision making and quitting.
And I know that you've been on a lot of podcasts. I imagine many listeners have read your book or have heard a lot about the stuff that you teach. I'm gonna try to come at this from a bunch of different directions and do it do a little differently.
First of all, I know you're parents, as last I heard you have four kids.
I just had a kid, he's almost ten months now.
Oh congratulations. Thank you.
I'm curious what your
Frameworks.
You find most helpful in parenting?
And in raising your kids that have stuck with you there.
Well let me just first off say I
I get a lot of people who are pregnant.
Like who come and ask me, like, for my advice about parenting.
And
I'm I'm gonna I'll tell you the two pieces of advice I g I give people.
The first is this.
There's all sorts of parenting books, there's all sorts of
styles, you know, like
You know, a book needs to sell, so they need to sell you something new or different or whatever. Okay.
But
The only thing that matters really
Is that your kids know you love them.
like really know, like deep in their

No, you love them.
And like we can argue
breastfeeding versus not breastfeeding, or like co sleeping versus not co sleeping, or attachment parenting, or sleep training or
We didn't talk about all that stuff.
Right, do you home school, do you send them to private school, do you send them to public school?
Well, we can have lots of discuss
But it all dwarfs in comparison to
Your children.
No that
You would lay down in the street in the front of a bus and
If it meant that they would
be alive and happy.
Like
That's the number one thing I say.
Number two is
At some point you will drop your baby on its head.
So I I bet this has happened to you already.
Um well well yeah, sort of. Okay. Sort of, right.
So here's the thing about babies is
Babies can't move.
And so you get very used to, you know, like
You know, as a mom or a dad or
Like you're sitting on the couch and you're folding laundry and your baby is right here next to you.
And your baby can't move.
And then one day without warning
Your baby can roll over.
And you think it's fine'cause you like put a pillow there, right?
But like you turn your back for a second'cause you're like, Whatever and then your baby falls on its head.
This it happened with every single one of my children.
And you know, a and they may they don't fall that far and and whatever, but they're totally fine.
And you're not a bad parent.
And it's just this l like
There are things you don't know and like stuff's gonna happen and it's not gonna be perfect.
And every little thing isn't gonna have some lasting impression on your child.
'Cause at some point you won't drop your baby on its head.
Uh huh.
You will not mean to.
You didn't intend to.
But
You know, in my case it was always like, Oh my gosh, my baby can roll over. I did not
think that was gonna happen today.
And gosh, by the time I got to the fourth one, it's like I'm chasing after what baby to try to get the diaper on and this and that.
And then the third baby hits its head.
So
You know, I think there's just a lesson in that, which is like
It's a small thing, like don't sweat it.
And don't
Don't uh
get mad at yourself because you made a mistake. It's like the mistake would be
Oh, I realize my baby can roll over now.
So now you should make sure that you're taking extra precautions about your baby rolling over off the couch.
But it's just a fun it's just I mean, it's really just a funny way for me to say like
Your baby's gonna hit its head or
you know, walk into a coffee table or get cut.
Or
You know, whatever. It's like
They're very resilient.
And you didn't do anything wrong as a parent.
So don't beat yourself up about that.
And then the I do actually have one other thing I say. I go for it. Uh having four
Uh you have very little you have all no influence whatsoever basically on what your child's personality is.
Um
what you can do as a parent is make sure they say please and thank you.
Yeah.
So
You know, what I find with like parents of one
Is
Uh particularly if that one is very well behaved.
They think they're a great parent.
And they may be.
That would be resulting.
They may be.
But once you have four, what you realize is like, Oh, my parenting really didn't have anything to do.

But my children do say please and thank you.
This is really valuable, good advice. I I need to hear this. My wife is gonna need to hear this too.
She's always worried about the soft spot on our kid's head and getting hit in the It'll be fine.
It's actually kinda why it's that it's part of why it's there. It shall oh really your baby will be fine.
Like, I mean that's the thing. Like
You know, we used to cart babies around in the wilderness, you know, like
We're living in Gates.
We're fine. We're built for it.
We're built for a much rougher world than
The world that babies actually
Grow up in now.
I will say my my oldest child
When she was in high school.
Just for whatever reason.
She had no interest whatsoever in like alcohol or drugs or anything like that.
And I remember just feeling so smug about
How well I had raised this child and how
How great she was.
And oh these other parents who clearly weren't doing a good job.
I mean I didn't really, but like kinda.
'Cause it's hard not to.
And then I had m you know, and then my next child went through his own thing, uh
No that
That's just a temperament thing.
It's a good example of resulting in action. It is a good example of resulting.
But uh all my kids have turned out fine and they're all very different.
Mm-hmm. Um
But yeah.
So uh but as far as decision making is concerned, which is a totally different thing.
I'll actually quote Danny Carman on this one.
Nothing is as important as it seems when you're thinking about it.
That's a really important one.
Um
One of the things that I used to try to do with my kids all the time was mental time travel, which is actually a very good decision tool.
So they would be really upset about something.
And it could be
Something that happened at school, but or it could be like
They were grounded for something or
Whatever.
And
You know, and the thing is when you're in the moment, it just feels like so big.
And the thing that I used to say and they say all the time now is
This is gonna be so great for you when you're forty at Thanksgiving.
you're gonna be able to tell these stories to your children
And it's gonna be the best.
Like
You should be thanking me.
So that you can talk about your like crazy mother.
T.
to at the Thanksgiving table.
You know, and it allowed them to get some time and space from it to sort of realize
This is gonna be funny at some point.
Yeah, like at some point it's gonna be like a hilarious story that you tell people, like no matter how horrible it seems now.

When you're older.
You know, and it sounds like kind of a fun thing I was doing with them, but it's actually a really good decision making tool for us in general, right? Like
Look, there are absolutely things that
twenty years from now are gonna matter.
For sure.
Uh
But most things no.
Like if you think about most things that ever happened to you, like
you know, you're twenty one and your girlfriend with you or
Whatever, and you just think it's the end of the world and you're never gonna recover, you know, and all these things. But
Yeah.
If you could get yourself like, Well, how do I really think I'm gonna think about this in ten years?
Am I still gonna be heartbroken in ten year probably know, probably not. And I think that we need to get that perspective of time.
So that we can get out of the the moment, you know,'cause in the moment it just feels so important and the feelings are so big and it's like
The focusing effect on on this second and the feelings you're feeling right now are so huge.
That we s we forget the scope of time.
And I think that that's absolutely one of the best things you can do with your kids. And you can do it in small ways too.
Right? Like
You can choose to
play this video game or you can choose to study for your task.
A week from now when you get that test back.
How do you think you're gonna feel about those two choices?
Right.
So I mean that was a tool that I used a ton.
And it's just a generally an amazing decision making tool.
Just generally for people. It's one that I happen to use with my kids all the time.
And then I I would say the the other one was um
Using the word nevertheless.
Which is this is a great leadership.
Skill.
The word nevertheless.
So
Let's say my child got caught.
doing something wrong. I don't know. Like I found a bag of red solo cups in my backyard that they forgot to throw out after a weekend that I was away.
That's hypothetical.
Um so
You know, so I'm grounding them.
And
it's a lot of argument back, right? So th they think this is a debate and and they're giving me all of their input and their opinions on
why uh it's unfair and
all the other kids do it and the other parents don't get mad and
Whatever, whatever the argument is.
You have to have the balance between
them feeling heard, which I think is incredibly important that your children feel heard.
And
following through on what you know or believe is right. So it's
I hear you.
Nevertheless,
You're grounded for two weeks.
Like I hear what you're saying.
And I understand.
Nevertheless,
This is what's gonna happen.
And obviously, like the words that you can use for that might be different. You have so much more authority, obviously, over your children.
than you do in other places, but in the workplaces is very good'cause employees gripe all the time at decisions that you make it in leadership.
Because they they think they're right and they want their way.
And to have the ability to say, I heard you
And your input, trust me, it was incorporated into the decision.
Nevertheless, this is the path we're gonna take.
Right or wrong.
That's what we're gonna do.
So, you know, I think there's a lot of things that you basically you can say, like these are good things to do with kids, but they're actually just generally good really good decision strategy.
I'm taking notes on all these things. These are gonna be extremely Nevertheless is a really good one with children.
It connects to something uh Doctor Becky teaches if you know her of
Telling someone I believe you. When I when your kid says something and they're upset about something, start with I believe you.
Never go. Yeah, here's Nevertheless, exactly.
I think uh we're gonna start having to pivot this podcast, our parenting podcast,'cause there's so much
stuff here, but I'm gonna try to resist.
Uh something that
So we've been talking about decision making and frameworks and things like that.
Something I wanted to ask you is
How much better can someone get at making better decisions?
So for example
Say someone listens.
All of your
Podcast reads your book, studies it intensely.
What's kind of the delta you find in somebody being able to make a better decision and where this sort of comes from is
Daniel Kenneman of all people.
People ask him like he just lives such a
optimal life now that you know all of these biases we have and all these mental errors we make. And he's like, No, I can't
I think he said famously, like I I don't actually I can't use these in practice. This is just stuff that I
have learned but it doesn't actually impact my day to day.
So I'm curious just
How much better can someone get? What kind of delta have you found?
Terms of making decisions.

So
So it depends on whether you actually do the thing.
Mm.
I mean, I think that that's what the issue is.
So, you know, I mean I think you know, Kahneman uh did some work way back when like that sort of started him on this journey.
Uh no, it's basically work on hiring.
Uh and
You know, is taking it from completely unstructured
Lenny saying, I just know a great
product manager when I see one.
Whatever that means.
Two
Me going to Lenny and saying, Okay, I I understand that you think that you see a great product manager
you know one when you see one, but can you explain to me what that means? Like in in the abstract.
What are the things that you're looking for?
in someone that you want to fill that role.
Uh and we can then sort of excavate that, right? And and make what what
is implicit because you're you're applying some sort of implicit like model
to
ha you know, how you're thinking about the person that you're interviewing and and how they map onto the role
That they're gonna be in, but let's make that explicit.
So we can make that explicit, we can turn it into a decision rubric, we can create a structured interview process out of that.
And then, you know, what he found was after you've gone through that process, if the you then sort of use your intuition.
After having done that, not before.
That then you actually can really
Drastically improved.
Sort of your hit rate on hiring.
And in that case, uh, it was from about a fifty percent hit rate to sixty five.
That's pretty huge.
So that's a really big difference.
Problem is nobody does it.
So I I can tell you that
most of the conversations that that
I had with
Danny that were about my work in particular.
We're just ta you know, him saying like how do you get anybody to do it?
Now
That's not to say that I would be better at that than him. It's that he didn't do my job, right? So he he was an academic doing research and so on and so forth.
I'm like living in companies.
Like embedding for years.
Um
You know m I've I've been with
one client for five years, one client for four years.
uh one client for
Four years also, almost four years, three and a half years.
And then I just took on a new client who I've been with for six months, and that's my whole client roster.
Because I I stay with them so long.
So if you do it
The answer is quite a bit. Like at least in terms of
Danny's work, you know, i in relation to something pretty noisy like hiring.
Uh
You went from fifty percent hit rate to sixty five percent hit rate, which is huge.
Uh, it's enormous.
The problem is that
the way that you can make decisions in order to be better at them
is not supernatural to the way that humans make decisions.
I think we think a lot more highly of our intuition than we really ought to.
Uh we think uh
very highly over ability to notice things in the moment and act rationally toward them.
In a way that really we ought not to.
We tend to
think that we have insight that other people don't have when actually
the other people probably have more insight into our situation than we do.
So
You know, I think that the I think the answer is a lot.
With a big if.
Yeah.
With an if you can get people to do it.
And that I think that's where the issue comes in.
Now, the good news is that
That
You can then
reverse that and say
If I'm willing to do it.
Think about what an edge I'm gonna have over people who aren't willing to do it.
Right.
So we g it's like two
We can think there's people who don't know about it.
There's people who d do know about it but don't do it.
And then there's people who do know about it but do do it. And th it's that last group that's so tiny.
So the answer is I I think a lot, but nobody does it.
Mm-hmm.
So maybe following that thread, when you
When you look at companies that you've worked with or companies that have read your book and really dove in
And started to implement some of your
Piece of advice to make better decisions.
If you kinda look back at the ones that
had an impact. What do you find are the
The
mental models or frameworks or tactics that really stick
that most often have a an impact and the biggest impact.
I think the one that's easiest to implement is this.
And it's so easy and I just wish people more people would do it.
Um
The best way to get somebody's opinion
Is
Independently.
of other people's opinions, independently, asynchronously.
So
You know, the way that I put is like I I want people to stop talking to each other so much.
when we think about like w what did people generally think the purpose of a meeting is, they think it's for three things.
Discovery.
Which is I wanna discover
what your opinions are, what your judgments are.
Of something.
Right. So I wanna find out what you think about something. So like
For example, if we're if we're um
If we're develop you know, if we're we're if we're in product development and we're
Uh
We're trying to figure out a timeline.
You know, like so we've got like we're trying to develop a product roadmap and we're trying to figure out how long it's gonna take to release certain features or
Something like that.
We all come into a room and we start yelling it out.
Right. Together.
You know
That's it's it's just like it's so bad for decision making, like I can't even tell you.
There's cross influence. The loudest person in the room tends to then have a a m an outsized influence on the decision.
the most confident person in the room tends to have an outside influence outsized influence on the decision.
And that's great if the most confident person is also right.
Uh but the problem is
That's not always so.
Um
And so that's the discovery piece, right? And we tend to do that in a group.
Then there's discussion, which is I've now discovered the way that Lenny is modeling this problem or what his
judgments are about certain things, or his forecasts are, or his estimates are.
And now we're gonna discuss
those ideas. And we're gonna discuss discuss
Your ideas in comparison to my ideas and that's gonna happen in the group setting.
And then there's also now we're gonna decide.
In the group setting.
So we're gonna make some
set of decisions about
Say what the roadmap's gonna look like.
Here's the thing that I think is the easiest to change.
is to realize the only thing that's ever supposed to happen in a meeting
Is that
Discussion part.
So we should absolutely
Becoming together as human beings to discuss
Everybody's judgments and opinions and the way they're modeling the problem and their forecasts.
In particular, it's really good to come in and discuss the places where people are different.
So you've been in many meetings, I'm sure about eighty percent of the time is double clicking.
Oh, I agree. And I wanna now re
literally just use different words to say the exact same thing'cause I'm in agreement.
If we can if we can focus the discussion on places that people disagree, we're much better off.
But
We so now we can take this discover, discuss, decide.
So I'm saying disgust is only supposed to happen in the meeting. That's the only thing that's supposed to happen in the meeting.
So what's happening with the other stuff, the discover and the decide?
Well
And this is the thing that I think I I actually have been pu able to get people to do.
Is
Discover what people think before you get in a room.
independently of each other.
So how would you do that?
Well
Let's imagine that you're gonna have a meeting about the product roadmap.
So you would say to yourself, in this meeting, what are the opinions that I need to get?
From the people in the room.
It could be a brainstorm. What what are all the different features that we could develop?
It could so that it could be a brainstorm. Fine.
Write to them independently and say, Hey
Free for off.
Just
Come up with all the different features that you think would be reasonable for us to consider developing.
And then uh give me a fourth rank.
from best to worst of your own ideas with some three to five sentence rationale as to why you have these things in this order.
Okay, so I could do that.
But maybe we already have a list.
I can send that out for a force rant to everybody. Okay, so here's the final list of things we're really considering.
Uh prioritize those for me, just force rank them.
Uh and then again, give me a rationale, give me a little bit of free writing as to why you think this should be that you have these things in this order.
Maybe we've decided now on what our top five priorities are.
Great.
Uh
I could before the meeting.
send the top five priorities out. Hey, we've decided on this stuff.
For each thing, I'd like to understand what you think a reasonable timeline is, how many sprints it's gonna take.
So on so forth, right? So we can ask for that type of information.
uh so that we can start making estimates because that's gonna affect like budget and what we're going back to the board with and so on so forth. But
regardless, like just figure out what what uh what is the thing that we're gonna be discussing in this meeting.
And I'm gonna send it out to everybody independently and I'm gonna say don't reply all.
Right, just send it back to me independently. If it's a repeated decision, you can actually create a rubric that lives on like
air table or coda or
Google Sheets or whatever, I don't really care.
And people can input their decisions there.
where they can't see anybody else's decision. So like with Google, you could use Google Forms.
in order to do it and then it dumps into a spreadsheet that only you can see. That's a great way to do it.
So anyway, so you do that.
Uh and now you
can now see everybody's opinion.
that you then now send out to the group and say, Everybody look this over before we come in and discuss.
So notice you're still working as a group, but you're working as a group where you're not
in the same room together and talking at the same time.
And there's a word for that.
uh which is nominal group.
So it's a group that at that moment is working independently and asynchronously of each other.
Uh so if you can get people to do that.
And I ha I ha I do have companies where I I don't consult with them.
But I've just come in and talked to them briefly or whatever.
that do actually implement
That piece of it.
That is a huge piece of it.
That's
Ginormous.
And then you do the same thing for deciding.
So the decision should not not be made in a room.
It's uh made
Uh either
I love the one decision maker model, but not everybody's down with that.
But you can have like a vote forum where people go vote in private about the way they're leaning.
Uh you know.
You can the you know, you can do a variety of things with that, but just don't do it don't do it in in the meeting.
And then that just the last thing that I'll add.
Which which is a muscle that you really have to exercise.
Is
I think it's really important to
to understand
that
The word alignment
In terms of we're all aligned to the group.
Right.
The word alignment is stupid.
And it shouldn't be used.
And I know I'm saying that very harshly.
But it's true.
It's done.
Because it doesn't exist.
Like
You have ten people in a room
And they're all really different people with different opinions.
And
They're never gonna come out of the room agreeing with each other.
And it's really bad if the expectation is that they're supposed to.
And the the re it's really bad for a few reasons. One is it isn't reality.
And I don't like
coming out of things like without reality actually.
Kicking.
So it's not reality.
People don't actually agree. They're not actually aligned.
on the decision. That's just the thing that makes you feel better.
Right. So I I you know I think that that's problem number one.
But problem number two is that if the goal is alignment
If the goal is agreement.
then the meeting becomes coercive.
And you never want that.
So it the way that I'm supposed to
Talk about my ideas is to convey
Why I believe what I do.
Not to convince anybody that I'm right.
Because if I'm working to convince people that I'm right, it becomes coercive.
And that's horrible.
So
You have to get comfortable with walking out of the room. This is the nevertheless.
Walking out of the room, understanding that
Once you have that discussion, it's not that
Lenny, your your opinion might change, it could.
Right, I could say my thing.
And it could be just so damn brilliant.
That
You change your mind.
Somewhat
Right. So you come maybe you you're like oh I'm thinking about this differently and actually
But maybe you don't.
And you still believe a thing that's very different than me.
And
Leadership has to say
That's fine. I've heard both of you.
And I know that this isn't going Annie's way.
Nevertheless,
trying to think about what all of our goals are. This is what the decision is gonna be. And it's totally fine that you ended up not agreeing with each other.
Because it's reality and what it allows me to do is get a better sense of what the space of decisions is.
So those things I have been able to get people to do.
And they're actually quite impactful.
Imagine a place where you can find all your potential customers and get your message in front of them in a cost efficient way.
If you're B2B business, that place exists. And it's called LinkedIn.
LinkedIn Ads allows you to build the right relationships, drive results, and reach your customers in a respectful environment.
Two of my portfolio companies, Webflow and Census, are LinkedIn success stories. Census had a 10x increase in pipeline with a LinkedIn startup team.
For Wetflow, after ramping up on LinkedIn and Q4, they had the highest marketing source revenue quarter to date.
With LinkedIn ads, you'll have direct access to
and can build relationships with decision makers, including 950 million members, 180 million senior execs, and over 10 million C level executives.
you'll be able to drive results with targeting and measurement tools built specifically for B2B.
In tech, LinkedIn generate a 2 to 5x higher return on ad spend than any other social media platforms.
Audiences on LinkedIn have two times the buying power of the average web audience, and you'll work with a partner who respects the B2B world you operate in.
Make B2B Marketing everything it can be and get$100 credit on your next campaign.
Just go to linkedin.com slash podlenny to claim your credit.
That's LinkedIn.com slash podlene.
Terms and conditions apply.
There's kind of an implication here. You touched on it a bit that there's a DRI essentially. There's one decision maker.
Sometimes people start to feel like, Oh, my voice isn't heard. I don't have a lot to say. I can't be part of this decision.
I do talk a bunch about just how to make people feel included. You get feedback along the way, but any advice there.
If you try to move to this model of making
people feel like okay, I actually have impact on where this goes.
Yeah, so so here's the really wonderful thing.
About
Moving from a
a coercive model to a to a
Uh
Really, I guess you could say a model of curiosity, right? Like you want to be curious, not coercive.
Which means that the way that people in the meeting are are are com
are talking about what their opinions are is in the mode of conveying information, not trying to convince anybody.
So
Yeah.
Once we move away from that coercive model
And like and when I say coercive, like I'm I'm not saying anybody
is purposely trying to set up a culture of coercion.
Sometimes that's true.
But
For the most part everybody's trying to do a good job and nobody's trying to set up a culture of comvert coercion, but as soon as you say, Are we all in agreement or we're all in alignment
And as soon as you're allowing people and I'm sure you've been in these meetings, right? If you allow people to interrupt
If you allow people to say this, I think you're wrong.
I disagree.
Here's why
Those are all very coercive
Things to do.
To have happen.
You know, the silence
You know, interruptsing interrupting someone is silencing them.
Saying you're wrong?
Well that now you become tribal and people aren't gonna be open minded and they're gonna
stake their ground and it's it's all really bad. So let's move away from
That number one, that's that's already gonna help.
But
when we think about the way that meetings normally happen, again, there's like all this crosstalk and
Some people aren't speaking and some people are and so on and so forth, then then of course not everybody feels equally hurt.
But when you work as a nominal group.
What let's talk about something as simple as
uh
We're gonna make a forecast of how long it's gonna take.
to
Uh
to launch this product feature.
So
I'm gonna send out to everybody.
what's your point estimate, what's your lower bound, what's your upper bound in terms of timeline.
And you could do it in some way like how many sprints do you think is gonna take whatever you whatever language you wanna use.
Now everybody independently.
now gives their forecast.
With
a rationale for why they believe that.
And then you come into the room.
And you run a discussion where everybody's getting to say what their estimate was and why they believe that and people are are getting to ask questions.
I I have a question I always as a clarification. It's just I don't understand.
And as the leadership the way
that you would do it is um
Let me give you an example from a real one'cause it that'll be easy.
Um
I did one of these discussions for a question about
remote work, like what did the company want in terms of remote versus hybrid and that kind of thing?
So
Like there was a lot of disagreement.
About whether
Whatever the policy was.
It should be consistent across function.
Lots of disagreement.
So
You know, let's take somebody who was on the no, it shouldn't be consistent across function.
So they they now say why.
they believe it shouldn't be consistent across functions. And they say things like
Well, different functions have different requirements, right? So
There's some functions that have to be in the office, like if you're
IT.
As an example.
But there's other functions.
which are more uh you know collaborative and creative and and whatnot.
Uh
Where
it makes sense for people to be in the office.
Whereas like engineers
Doesn't really matter.
Right. So they're like there's different needs of different functions in terms of how much they need to be in the same space. So I'm not agreeing or disagreeing with this. I'm just saying what like somebody said.
So now as a facilitator
I never s would I never say
Oh, I agree.
What I say is
I just want to make sure I understood what you said.
And I reflected back.
So I say, so what I heard you say is that not all functions are created equal.
In terms of how well they work remotely or how well they work in person.
Or what the flexibility might be.
So what you're saying is there're some there're some functions that have to be in person.
Period.
And then uh
you know, other functions where
that sort of collaborative element
Be in the same office.
would be more important
versus
some functions where being remote is totally fine. Is that what you meant?
And then they have
they have the ability to say yes, that is what I met or
Actually
I I meant something slightly different let me say what that is and then I reflect that back.
Okay. So literally you're just going around, you're calling on people to do that, and then you're reflecting back what they say without offering your own opinion.
I don't know, Lenny, tell me how someone doesn't feel hurt in that situation.
Absolutely.
I would love I feel so good if somebody just clarified and made it clear they know what exactly what I'm saying.
Even if the decision ended up.
Doesn't end up the way I want.
And then what it actually ends up happening is that
The people in the room feel more uh
the the psychological term would be endowed.
I'm more endowed to the decision. In other words, they feel like they have ownership over the decision.
And whatever the decision is
They generally will see their their selves reflected in it.
Mm.
Because they were hurt.
You know, and
And
They also will generally understand that nobody n no one really ends up with exactly
Every single thing that they wanted to see in the decision because they also get to see the true spread of opinions.
On the team.
And and what you
see in that situation is that there's lots and lots and lots and lots of disagreement.
Which you don't see if you talk in a group.
It uh
Narrow's the thick
This
Base.
Right. So
Uh for example, like particularly if you're senior
If you say, I think this is gonna take
Three months to launch.
And I was thinking
Four weeks.
You're never gonna hear four weeks from me.
Right.
Ever.
But if we get those opinions independently.
you will actually hear that I think it's gonna take four weeks.
You may tend to be more right there, being more senior, but I may have something interesting to say.
Like you should hear what I have to say.
And this also allows me to learn from you too.
But because I I haven't heard your opinion first, I'm I'm not
I'm not gonna
conform my opinion to yours.
So it actually spreads the the um
sort of surface area of disagreement that you see on the team.
Which then makes peop people feel much better about the decision not being exactly what they want because
they recognize like, Oh, this is actually a hard problem. Like
People really disagree on this stuff.
Amazing. I think this is gonna be really helpful to a lot of people. Just to kinda close the thread on this little summary maybe is
Court Vice here is
Brainstorm separately. So I guess first there's discover.
Discuss, decide.

Completely independently.
Basically brainstorms. You're a big advocate of brainstorming independently sitting
And your own thinking through ideas.
And then bringing people together to discuss
all the things they've come up with and especially where they disagree.
And then
Ideally having one person
That makes the decision once
Sure, he has taken the floor. I mean, I I work with people where it's like a partnership and there's six people who are gonna vote.
Uh but they still do it independently. They go to a forum.
So I don't know what your vote is.
And we and all six people don't have to agree.
I mean I I I just think it's really important that like
Once you get above an N of
one, you shouldn't necessarily expect the people to agree.
Which I think is just really important.
And this is good for more than just brainstorming. It's for like forecasting.
Any kind of project planning?
uh budgeting
Um
Yeah, I mean we do this like
Uh for example uh
First round we do this for we have a structured
forum for evaluating a a company.
um in terms of whether you should invest in it.
Uh because
You know
There are facts and then there are the way that you model those facts and
So
uh it's a lot, you know, w what are the investors opinion of the founder and the product and
that kinda thing, and we just have structure around
Uh
how we're eliciting those opinions.
Um, which is really helpful. The the other thing that um
I just
kind of add to what you said is that
You can actually be in the same room together.
And still discover information independently.
So this will happen on the fly all the time.
where people are talking and they they start talk you know.
uh something comes up like someone suggests
some new feature or something like that.
And then people start saying, Yeah, but that's gonna take
And you go, Stop.
Okay.
Everybody take out a piece of paper.
So you you can still get that same independence on the fly.
As well.
is doing it in advance. But yes, that was a very nice summary.
If this is just the one thing people take away from this conversation already, I think that could have a big impact.
Speaking of first round.
One we actually just had Todd Jackson on the podcast talking about.
Left hide.
Talking about product market fit. I that episode will have come out before this episode.
Uh
Also, so I asked Brett Burson, your colleague at first round, what to ask you.
Oh.
Yeah, yeah.
Uh, by the way, your title of first round is amazing. Uh Special Partner.
I've never seen that before.
I am. It's a title just for me. I I adore Brett Person.
Me too. So I'm interested. I'm interested in what he said you should nothing too spicy.
Uh it's kinda along the lines of what you talked about.
So you said that you have a very uh interesting framework for how to think about decision quality.
in the short term when the outcome
is very long term. For example
Investing.
Also, many decisions we make in business, things you need to decide now that you only find out years from now.
Can you talk about your advice there and your insights here?
Oh my gosh, I'm so happy that that's the question that he asked.
Okay. So c can I give a tiny bit of background to it? Absolutely.
So prior to talking to first round, I have another client too who
They're amazing. I renegade partners at Razanne.
Uh, Winserk and Renata, Quintini, they're they're
Incredible.
Uh
Before kind of like
finally hooking up with them and they're they sort of work at different stages. First round of seed, obviously.
Um
Renegade is more like A B.
Tiny bit of Davulancy.
But uh
Before running into them I talked to
quite a few uh
Venture firms.
You're interested in talking to me.
sort of post thinking and bets having come out. So this would be like two thousand eighteen.
And
They were the theme.
There was a theme across
them all.
The the first one was
Well, the the kind of decision making you're talking about we don't need to do because
Uh we just know a good founder when we see one. So
I that is a sentence that came out of many people's mouths.
And as I just said to you, like the
Okay, I have no doubt, but don't you want to make that explicit?
There's all sorts of great things that come from making it explicit, not just in terms of
uh the increase in decision quality in the moment, but
It actually allows you to to close feedback loops much better.
So that was one thing I would that I found quite surprising.
But the one that I really
found very interesting was
Being told, that
Well
What you're talking about doesn't apply.
Because our feedback loops are
A decade.
And in poker you got an answer w right away.
You won or lost the hand.
Right away.
So the way that you're thinking about decision making doesn't doesn't really apply.
Until
I
Met first round.
And then
Uh Renegade.
where they actually heard what I had to say,'cause I gave the same answer to everybody.
So we'll just put
Aside that wouldn't you want to make that explicit?
The first thing that I would say is oh, poker's much noisier than you think because when I win a hand I have no idea why.
So I do actually have to wait a long time.
Because I have to have play many, many, many hands.
Before I actually know, like
Do I actually have an edge? Because I can't I I actually don't know very much.
On one hand, for one thing I almost never see my opponent's cards.
So I'm kind of left in in a dust of uncertainty.
But separately from that
The the main thing that I said was what how could you possibly think that the f the the feedback loop is ten years?
And this is what I think really caught round's eye.
Because when I was talking to Josh Capelman about it, he said
Well what do you mean? We don't get an exit for ten years.
I said
Oh, I'm sorry, do you invest and then you go to sleep like Rip Van Winkle?

And then you wake up ten years later and you go, How'd that go?
Or are there all sorts of things that happen in between?
The simplest
Thing.
The simplest thing is
Does it fund at Series A?
You know, and
You know, the little pushback that I that I sort of would get there is
But we're not investing for series A.
And I said, Well I know that.
But have you ever had a company that exited for more than a billion dollars that
did not find it serious A.
And the answer is no.
And I'm like okay.
So it sounds like that's necessary.
might not be sufficient, but it's necessary.
And it's certainly a signal.
That is actually more highly correlated.
With accident.
Uh well.
Than the investment at C.
Uh and then oh right, you have series B.
Right. And that's separate from all the other things that you can look at, like
You know what you talked to Todd Jackson.
What's going is it achieving product market fit?
We know that eventually for it to be
to be uh successful
Uh it's going to have to achieve product market fit.
Right. So you can look at what's happening with that. Um, you know, just general things about traction.
Uh
you know, what's happening with like NetNew ARR.
Ability to rate retain top talent.
Churn.
I mean, there's a there's so many different things that you can look at.
All of which
Are things that
You know must happen in order for the big thing to happen.
Okay.
So what that means is that
This this is the big I'm gonna make a bold statement.
Here.
There is no such thing as a long feedback loop.
You can make a decision, said
about how long the feedback loop is.
That is your choice, said
to live in a long feedback loop.
And you can choose to shorten the feedback loop.
And the way you choose to shorten the feedback loop is to say
What are the things that are necessary but not sufficient?
That's one thing.
for getting a good exit, or what are the things that are correlated with?
the outcome that I eventually desire.
And
What that means is that when you're at the decision point
Right. Like
In first round's case I'm gonna
invest in a company.
What you have to understand is that you are making a prediction about
How the world is going to unfold, how the future is going to unfold.
And those things that you're predicting.
You can track.
And you can track them back to the decision.
And you can do it pretty darn fast, mind you.
I mean think about like
Think about being in twenty twenty one.
They were companies that were raising an A six months after seed.
You know, today it's a little more sixteen month.
Yeah.
But but
Even so, the
Let's just say that that was the only thing that you decided to do.
I'm gonna forecast the probability that this company's gonna fund at series A.
And then obviously those companies start to fund or not fund at series A.
And you're finding that out in sixteen months.
Here's my question for you, Lenny.
Is sixteen months shorter than ten years?
Mm-hmm.
So I mean that that's you know, I mean it's probably why he
said to ask me that question.'Cause I just really do. I mean I I'm just in qu I have a very strong opinion about this.
The feedback loop is as long as you choose it to be.
Yeah.
And
If I take that back

Early on when I was talking to people.
What I would say is that
I think that there is a certain amount of psychological safety
And
Allowing the feedback loop to stay long.
Because
really of two main factors.
One is that
Look, if if I was early into Uber
And now I'm like
You know, uh
celebrity investor or something.
I don't really want to know if I'm good or not.
Do I?
Right. Like I don't really want the world to know that. I mean if I'm good, that's great, but it feels like they already believe that I'm good because I happen to be early into Uber.
So so since people already think I'm good, like I'm kind of just losing to that decision.
Psychologically speaking.
Not
investment quality speaking, but psychologically speaking.
Okay.
So here's the problem though.
Why was I early into Uber?
Did I have an insight?
into a real pain point.
in a developing market
Blah blah blah.
Or
Did my buddy start Uber and I was like, Sure, I'll give you some money.
Right. I mean, obviously I'm talking about the extremes here.
But we don't
We don't actually know what the decision quality was, right? Like
All we know is that you had a good result.
And given that you had a good result.
And people think very highly of you.
Uh what are you gonna get?
Right. So so it's like it's so nice to just sort of let that feedback loop sit there.
And allow people to have the opinion of you.
That is really
Nice. Feels good.
Right.
And not actually find out the answer.
'Cause why would I want to?
Unless you're
really super focused on decision quality.
Then you would want to do that.
So that's
You know, that's part of the psychological safety.
I and what that goes to is the real core of it.
Is
That
It's very, very difficult to
For human beings.
To deal with feeling wrong in the moment, even if it helps them in the long run.
It's just hard.
And the tighter the feedback loop.
the more that you risk finding out you were wrong in the short run.
Now that helps you to learn and improve your decision making if you're
If you're focused on it and you're good at it, right? Like
that's gonna help you and then in the long run you're actually gonna do better.
But human beings are notoriously good at trading off the long run just to feel good in the short run.
That's why we're all eating chocolate.
And cupcakes.
and stuff that we know is bad for us.
'Cause it feels good.
And w so much of our decision making is
trying to advance this positive self narrative.
And the idea that, yeah, we're gonna have a more narrat positive narrative ourselves in five years if we do some stuff.
You know, most of us are like, No, I don't really want to make that trade. I'd rather just kinda feel good now.
And
I can kinda use the fact that
Yeah, we're living in power law.
You know, under the influence of power law.
I can kinda use that to sort of
Just sort of confirm a lot of things that I wish to
to believe that that are true of me.
And if you kinda take that away from me and you take the uncertainty away from me
It's gonna be really hard.
And
I will tell you that's what I love about both Renegade and First Round is that they're just like
I wanna know.
It would be such a horror for me.

making good decisions when I actually wasn't.
And that's what really matters to me, and I I think that it's just like so special.
Hearing all this makes me uh makes me want to be an LP in first round. Not that they would let me in there, but it just knowing that you're in there.
Futsing with everything and the way they're thinking is uh
It's really inspiring.
I'm curious if you can share an example of anything that they tweaked.
as a result of this analysis that you did and how they evaluate
Well, first of all, let let me just say
Uh
They didn't really record a lot about their decisions.
Uh when I first came in.
They voted.
Uh and they had a record of the vote, so they they knew who said yes and who said no.
But they didn't they didn't really have any
They
have a lot of other information.
So the first thing that happened was just that.
Okay, what do we really think of
Is
the way that you would model like whether you should invest or not invest in
You know, broadly
very broadly you would say like you're you're rating the market the
the team, the founder, the product.
Broadly.
we're gonna make sure that those opinions aren't just like it's good or it's bad, but it's
you know, on a scale of one to seven, so that we can actually get some precision and
Some spread,
uh among the partners in terms of say strength of market.
We're gonna make sure that we have shared
uh definitions.

People don't.
Uh, so what I th what when I'm thinking about market and market quality, I might be thinking about something very different than you. So we wanna make sure that
uh we have a shared definition of that and that's reflected in in something that we're called mediating judgments, which are
Uh judgments that you make.
related to market
prior to actually judging what you think of the market in general. So you could think of of something like um
competitive landscape.
Uh so you might you you would judge that. So
So we ha that turns into what these mediated judgments are, which is a basically an implied definition.
So you create that and then you also think about what are the forecasts that are important. One thing that you already know is
Uh you're gonna forecast the probability the company funds it series that.
So that was like a huge change.
Uh just a very different way of making decisions.
what we've been able to do with that now,'cause it
I've been there for five years.
Is
we now can actually look, say at the partnership as a whole.
And look at these ratings that they're making of these the component pieces of of
you know, parts of how they're modeling like what makes a good investment, these forecasts that they're making.
We actually know how these companies have now unfolded.
So
In the simplest sense, we know whether the
Hundreds of companies.
have funded at series A or not.
And now we can actually look and say
How good are are the partners?
at actually forecasting this thing.
Right? Are they random? Are they better than random?
And we we actually know that and we can feed that back to them.
So that they can understand their own accuracy
Around these pieces of the decision because
The fact is that
Whatever a a a seed investor says to you
Whether that company is gonna fund its series A is part of their decision. It's included in the decision.
So they're making that forecast.
Whether they make the forecast explicitly or not.
So what we're saying over it first round is
Let's make it explicit.
Because you're doing it implicitly anyway.
And then we can actually start to look at your accuracy.
We can now feed that back to you and let you know.
How accurate you are.
Which will then help you to become
More accurate.
Right.
We could also look because we we know sort of, you know, in any given vintage
uh sort of what are the best companies or what are the worst companies. Remember, we're having people do these ratings like on a scale of one to seven of
say the quality of the market.
And we can look across the partnership and say
Look.
Lenny, when you're when you think the market is great
How does that map on to?
how well the company is doing.
In the future.
When you say the market is terrible.
How well does that map on to?
you know, how that company is doing how that company ends up doing in the future.
And I can come to you and I can say, Oh, Lenny, by the way
Your judgments about market are amazing.
You you really could you know a good market when you see one and it's really mapping on in a great way.
and to how that company unfolds.
But you're not so great with founder.
Or maybe you're great across the board, or maybe you're not.
So we can we can now start to give people
we can give the partners insight into their own decision making
Not only to allow them to improve the decision making, but also to allow them
To understand
that they not to over index on certain things that maybe aren't as predictive as an example for them. It allows us to uh we also can change the rubric based on the evidence now. So the first version of the rubric is always uh
taking the intuitions of the partners and making those explicit.
But um
then we can start to loop those back together.
And understand?
Uh
Well, maybe this thing that the partners thought was important actually isn't predictive across any partner. For example, we can start to
Develop the rubric based on on the data.
These were all things that weren't possible.
Because
Prior to that
If I had come in and said, Well let's look at decision quality,
How would I do that?
You know, I mean, I have no idea why people were I don't just know whether they said yes or no. And so it's it's very difficult to then s to start to to do some really serious refining of of the decisions.
If I don't if I don't have that information.
I desperately want to know which partner it makes the best decisions. I know you're not gonna share that. No, I'm not gonna
The partnership as a whole is excellent, as we know.
And this is what I will tell you is that
All partners have strength and all partners have weaknesses.
And they're not perfectly overlapping, which is wonderful, right? I mean that that's one of the things. It's like
What's really wonderful and I think it shows the power of
You know, why would you have a
more than one person having input into a decision.
Is
Some people are very strong on
you know, rating a particular aspect of market, or some people are very strong on rating a particular aspect of
The founder.
There's overlap and there's
You know, and then there's
Things where, you know, Todd is uniquely great at something.
Or Josh is uniquely great at something, or Bill is uniquely great at something.
So that's the wonderful thing about it is that everybody has strengths and everybody has weaknesses.
And they're not perfectly overlapping.
So this is where where you can see like getting that spread of opinions and really understanding
you know, breaking that decision down into its component part.
really shows you the
Value of
Um
Diverse opinions.
you know, it being as input into a decision.
I could talk about this thread forever. Maybe let me just ask one more question just because I'm super curious. Is there anything surprising that stands out that came out of this analysis so far? Just like oh wow, maybe
market isn't as important as we thought or this person is amazing at Yes.
So I think there have been c
Just generally speaking.
There, you know, when you're creating the initial decision rubric, there are people that there are things that people are really pounding the table about that they think is like especially important.
in making a decision.
And
One of the things that we found is that
Sometimes
Their intuition was absolutely right.
The thing that they were pounding the table about
is incredibly predictive.
Not just for them, but other pe other partners.
But sometimes
It's not at all predictable.
And
You know, and these are equal table pounding situations.
So
Let's say you're pounding the table about something.
Sometimes
The thing you're pounding the table about.
Mm.
It's predictive for you.
And for a all the other partners, that it's actually quite predictive about how the company does.
But sometimes when you're really pounding the table
About something.
It's not just it's not predictive for the other partners, it's not predictive for you.
And I think that what's really important to understand about this, and this is why it's so incredibly
Necessary.
In improving decision quality.
to take what's implicit and make it explicit.
Is that
Our intuition is sometimes right and sometimes wrong.
It's not that intuition is crap and you
your intuition is just completely brought. I mean, obviously that can't be true.
We would die, right?
So your intuition is sometimes right.
But it's also sometimes wrong.
And if you don't
Make
In explicit.
Then you don't get to find out when it's wrong.
You don't get to find out when it's off base.
А на це застер.
So so that's the thing that I think is is really interesting is that like you have
Equal.
VMware.
And competent.
That this particular factor is really important.
And
Sometimes it is and sometimes it isn't.
And it's so surprising.
Because we're talking about people who are true experts.
Or great.
And
You know, we j I think that we all just
have intuition about intuition, right? Like
You just intuit that
If they're so amazing.
Clearly their intuition about ha about what's important would have to be good.
But not necessarily, that's the thing. Not necessarily.
Yeah.
I think that was that was probably the most exciting thing.
Keeping it mysterious, but I still appreciate you sharing. Well, I have to keep it mysterious. You know.
Yes, I understand. I can't give away the trade secret.

I wanna touch on a different framework that I've heard a lot of companies actually using and something that comes up in product a lot is this idea of
Pre-mortem.
Which is essentially think ahead of time of what might go wrong.
You talk about this'cause I think this is something that's easy to implement, really powerful.
And
A lot of people are actually doing this.
Yes. Okay. So a pre mortem it's great.
But only if you attach
A pre commitment with it.
Mm.
Uh so I just want to be super clear about that.
What you find with pre mortems is actually work
I actually did this work with Marie Schweitzer and uh Linnea Gandhi who who are both a pen.
Then when you have people do pre mortemans
Uh
it generally doesn't actually change their plan very much.
or change their behavior. So
I think that we have the feeling that if you do a preemortem and you think like what are the ways that things might go wrong?
that that's gonna change your plan.
But
Uh probably not unless
You're specifically using it for that purpose.
And you say, Okay
Like we're gonna do this, but
Let's think about how we might change our plan in light of this information.
But I think what's actually more important than that.
is what a pre mortem allows you to do is to set up
Kill criteria.
So
Kill criteria are just
a a set of
Signals.
That you might see
That would tell you that it's time to
Pivot or or stop.
Because once we were actually once we actually launch something
We're very, very slow.
to
decide to quit.
I'm sure that everybody has felt that way before.
uh things go on way too long, even like they're over budget and you've blown the timeline and
when you finally shut it down, you realize you should have done it a lot earlier.
Um, and this is kinda true across the board because of variety of biases
the str the most well known and probably the biggest influence is something called sunk cost.
Which is that feeling, but then I'll have wasted it.
All the time and effort.
that I've put into this.
Already.
So it's taking into account what you've put in in the past and deciding whether to continue
uh uh on in the future.
Um, so what we wanna do is actually just get better at that thing.
So sort of understanding that when you've gotten to the pre mortem process
You probably are you probably are gonna
Watch it.
Like that's probably gonna be the case.
Use the premortem.
to set up kill criteria.
So I'll I'll give you an example from a sales team that I worked with
Uh basically what I I sent them out a prompt.
All the I see is a prompt.
Um that was
Imagine
that you got a lead through an RFP or RFI.
And you worked on it for six months.
And now it's six months later and the deal is dead.
Looking back, you realize there were early signals that that was gonna
be the case, what were they?
So this is a premortem. Like what are the things that you saw?
That would tell you that this was
Gonna go south.
And they came up with all sorts of ideas.
Uh
Notice this doesn't mean they're not gonna
start off pursuing the lead, right? But they came up they saw they they came up with all all sorts of signals.
So I'll just give you three of them.
Um
Uh
The R F R F I was clearly written with a competitor of mine.
So they felt that was a very bad signal.
Uh that was like Hm probably
gonna go badly.
Another one was uh the customer didn't want a demo, they only want to talk about price.
Obviously that's quite bad.
And another one was after the first few meetings they couldn't get a decision maker in the room.
Right. So
Uh they it was a much longer list than this, but that those are three.
So for each of those, you now that now becomes a kill criteria. If I see this thing.
And now you attach an action with it.
So in the case of price
They actually just said we should kill it.
If if they literally don't want a demo and they're only asking about price, they're just trying to beat up
somebody else on price. Like that's
There were a box checking exercise.
So there they just said, We're gonna kill it, we're not gonna pursue the deal anymore.
So this is great'cause salespeople
Will not
will pursue deals forever.
And
Leadership is like, Well, why did you stop pursuing that deal? And they get in trouble for it. So this is gonna help with that problem, right?
Uh, in the case of
The R FI was written with a competitor of mine.
They have an action associated with that as well, which is
Ask them directly.
If they're working with a competitor.
and how far down the road they are.
Depending on the answer you would kill or pursue.
In the case of we couldn't get a decision maker in the room, offer up executive alignment at the next meeting.
And if you they say sure, great. And if they say no
Kill.
So that's actually the what I feel is the best use of a pre mortem is
To say I'm gonna try to figure out
What those signals are along the way.
That things are going badly.
And now instead of just kind of hoping
that when I see those signals I actually act rationally, which is
I hope that
it will not come true. That's why there's many people who climb Everest in the middle of a blizzard, even though
They shouldn't be doing that.
Use the premortem to now create structure around those signals that you've spotted.
And
commit to actions that you're gonna take if you see those signals.
And I think that's the best use of a premortum.
That's really helpful. And it's interesting how many of your examples come back to
Just a framework that you often talk about, which is make it explicit what is implicit.
There's another example that the inve the first round example is a great example of that where it's just
Here's all our assumptions that just make them actually explicit and just shows how much it's then you can examine them.
You can examine them, you can
People can discuss them, you can figure out if they're wrong or right or
Whatever. It's like
I I wanna be very clear, like I'm not anti what
you know, anti your gut.
Or you in your intuition.
I think it's probably sometimes pretty good.
I just want you to make it explicit, that's all.
Okay, so we didn't have
Time to get into quitting, which is your more recent book.
Maybe we'll do a follow up episode specifically thinking about quitting but my answers are long.
I apologize to everybody. No apologies necessary. We'll have plenty of time in the future, hopefully.
Well let me just ask one question. I found this one quote from you.
Where you said, I you should assume that if you're thinking about quitting, it's already probably past the time that you should have quit.
Do you still believe that is that generally a good rule of thumb in just any
Take away
Tip lesson on quitting.
As our one question on quitting.
So
The data is pretty strong.
That
By the time you quit, it's probably long after the ship.
Uh
And it's it's really just because
Look.
When we start things, we're we're starting things under difficult circumstances.
Uh
which come from the uncertainty of the decision to start something.
Uh so when we start something, uh luck is gonna have an influence on the outcome, which obviously we have no control over.
'Cause luck isn't in our control.
And then there's also hidden information. So what happens is that after the fact
we know that we're gonna learn new information and it can make it very hard to start things.
'Cause we we sort of want to be more sure than we actually need to be.
It's why, you know, Bezos has the seventy percent rule to try to roll people back and be willing to accept that uncertainty in the starting decision.
Now the good news is
that
When you learn that new stuff.
And the new stuff that you learn is, ooh, if I had known this, I wouldn't have started it.
Uh, you have the option to quit generally.
So that's the good news. The bad news is that the same difficulties
that apply to the decision to start apply to the decision to stop. In other words, we're making that decision under uncertainty as well.
So we're we're not gonna know for sure whether it would have turned out well.
Or poorly.
Unless we continue.
to do the thing that we already started.
And we don't like to walk away from things unless we know for sure.
So as Richard Baylor put it.
Most people won't quit until it actually isn't a decision.
In other words
The the
Whole thing is blown up.
Or you know, the startup has no money.
Or you're up on Everest and the blizzard is literally upon you.
Um and you're stuck in it.
Um
uh or I think as he said, uh they won't
Until you've fallen in the crevasse already.
Then then you'll make the decision to quit.'Cause you then you know how it was really bad.
So, you know, people generally like for example
don't quit their jobs until they sort of feel they have no other choice or
Um
relationships are
Projects or
products that they're developing.
It all applies.
'cause we wanted to we're we're just sort of
We want to know for sure.
And then on top of that is the fact that
There is this issue of sunk cost, which is
uh when we walk away we feel like we'll have wasted everything that we've already put into
what we're doing. But of course waste is a a prospective problem, not a retrospective one.
Even if we treat it like a retrospective one.
Because it's the prospective one.
Well, if you wouldn't start this today
then that means that everything that you're putting into this going forward is the actual waste.
Right, but we do that all the time. We go forward with things.
uh that we ought not to be going forward with because we're trying to protect the the resources that we've already sunk into it in the past.
Then there's other issues that have to do with, for example, endowment.
uh the ownership over the things that we've built. This is particularly bad in product.
'Cause we're building things.
And once we build things, we own them.
And uh once we own them, we don't want to give them up and we actually value them more highly than identical things that we don't own.
And then there's issues of internal and external validity, which is really just a fancy way to say your identity.
How do other people view you? How do you view yourself? Do you feel like you failed?
And what that means is that by the time
There's so many biases against stopping.
that by the time you're actually even thinking it about quitting
Like that.
It's probably already past the time that you ought to have quit.
But we'll still continue on until we know for sure we didn't have any choice. Cause here's the thing.
When you walk away from something
And someone's like, Hey, why'd you stop that?
And you're like, Oh, I had no choice and you tell them everything that went wrong and
y nobody's gonna question you. They're gonna be like, Oh, well it sounds like you put in your
the your best effort.
But if you walk away early, people are like, What?
So just quickly, like I'll I'll tell you like I think just one of the best stories of this.
that I've got in my pocket here. Uh let me pull it out before the end.
Um so
Stuart Butterfield.
Creating a product called Glitch.
And glitch is a
a massive multiplayer online world building cooperative
Game.
Um
releases it. This is in the
But
Uh
And
It's like a huge hit with the critics. It's Monty Python meets Doctor Seuss. It's incredible whatever.
They're getting tons and tons of w great word of mouth and uh you know, PR
Not doing any p paid marketing.
They have incredible investors in Andreessen Horritz and Excel.
They have six million dollars in the bank.
And they have five thousand die hard users, meaning users
who use the game.
Who play over twenty hours a week.
The issue is that
That
Customer acquisition.
was a beast.
That for every one person
Who was playing over twenty hours a week.
There were like
Between ninety five and ninety nine people.
Who
Uh came for five minutes.
and left.
So
Obviously this is a customer acquisition problem, which which everybody kind of knows.
So they make an agreement in two thousand twelve.
that they're gonna do paid marketing, which they do for six weeks.
And during that six weeks there they're
acquisition of new users, it's growing like uh six to seven percent week over week.
Which is amazing.
And at the end of that six weeks
It's November of
of twenty twelve. At the end of that six weeks
That Monday morning.
Stuart Butterfield writes a note.
to his investors and co founders and says I woke up this morning
With the dead certainty.
The glitch was over.
Now notice nobody would
to do this in this case.
Right. Like
But this is what happened is that
The issue is that you really kind of have to see like is this worthwhile or not? Would I start this today?
That's a forecast of the future, right?
And what he did was some back of the envelope math and he said, Look
If we continue to acquire customer
at the weight that we've been acquiring them, at the cost that we've been acquiring them.
Uh, it's gonna be thirty one weeks till we break even.
But that's an a absurd assumption because
Customer acquisition costs it it's gonna go up.
Ca cap has to rise because we're gonna saturate the core gaming market.
So it's got a right.
So what he realized at that point was that this was not a venture scale business and he was in this for a venture scale business.
So even though nobody else saw that he was supposed to shut it down.
he saw he was supposed to shut it down. And not only that, he saw that he was supposed to shut it down for his employees who were working for equity.
And he had now realized that the equity wasn't worth their time.
And that he it wasn't fair to them to keep going with it.
So he shuts it down.
Obviously that feels
who does that, right? But that that and he will actually tell you
that he knows he should have shut it down before the marketing push.
But he needed the marketing push.
To prove it to himself.
That he would see in the future, clearly.
Now the coda to this story just quickly is
Two days later,
He's like, Well I'm a start up guy, I wanna start something.
And he's got this internal communication tool that his team is using.
in order to develop this product that everybody loves. He says
Actually they really like that. Maybe that should be the next product.
So he goes and talks to the investors, they roll their money over into that.
And that thing which had no name at the time.
Now gets a name which is searchable log of all company knowledge.
We'll just laugh.
And so I think this is the important thing to realize is that we get so focused on
But what about everything that I've put into it?
When what we forget is that when you're doing something
There's a
Not just the cost of doing something that's not worthwhile that's direct.
But there's also the cost of not being able to devote your attention to
to other opportunities.
that might be available to you.
And as smart as Stuart Butterfield is
He couldn't see Slack until he quit glitch.
And that is a true cost.
That he would have borne
I'm continuing with glitch.
If he had continuing with gl continued with glitch, he
Slack would not be something that we're all using today.
That is a incredibly beautiful way to end our conversation.
I feel like I have at least a billion more questions to ask you. And uh on the other hand, I feel like we've also helped a lot of people make much better decisions.
Through this chat.
So I'm really thankful they made time for this.
Two last questions.
Where can folks find you online if they want to learn about the stuff you're up to, in case they wanna work with you?
And
How can listeners be useful to you?
You can find me at antiduke dot com.
If you're interested in working with me.
Um, I have a sub stack in Kingdom Bet.
Uh, please go check that out.
I teach a class on Maven dot com.
twice a year on effective decision making.
So if you're interested in that, you can go to Maven.
And check it out. My next cohort at the moment is in September.
Uh although
I might do one. I'm not sure. But uh but in terms of how people can help me.
I co founded an organization called the Alliance for Decision Education. We're trying to bring uh the kinds of knowledge that we have about improving human decision making in adults
Two K through twelve education.
To make the world a better place.
So I would love it if people could go look at that.
Uh if you're interested in it.
Um get the word out.
Amazing. So we'll link to all those things in the show notes.
Any
Again, thank you so much for being here.
Thank you.
Thank you so much. This was so fun.
Same.
Bye, everyone.
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.
Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast.
You can find all past episodes or learn more about the show at Lenny's Podcast dot com.
See you in the next episode.