Godfather of AI: I Tried to Warn Them, But We’ve Already Lost Control! Geoffrey Hinton

0:00 They call you the godfather of AI. What would you be saying to people about their career prospects in a world of super intelligence? Trained to be a plumber. Really? Yeah. Okay, I'm gonna become a plum.

0:12 Jeffrey Hinton is the Nobel Prize winning pioneer whose groundbreaking work has shaped AI and the future of humanity. Why do they call me the godfather of AI? Because there weren't many people who believed that we could model AI on the brain. So that it learned to do complicated things, like recognize objects and images, or even do reasoning. And I pushed that approach for fifty years. And then Google acquired that technology. And I worked there for ten years. Well something that's now used all the time in AI. And then you left. Yeah. Why? So that I could talk freely at a conference. What did you want to talk about freely? How dangerous A I could be. I

0:45 I realize that these things will one day get smarter than us. And we've never had to deal with that. And if you want to know what life's like when you're not the apex intelligence, ask a chicken. So there's risks that come from people misusing AI and then there's risks from AI getting super smart and deciding it doesn't need us. Is that a real risk? Yes it is. But they're not gonna stop it because it's too good for too many things. What about regulations? They have some but they're not designed to deal with most of the threats. Like the European regulations have uh claws that say none of these apply to military uses of AI. Really? Yeah, it's crazy. One of your students left OpenAI. Yeah. He was probably the most important person behind the development of the early versions of chat GPT. And I think he left because he had safety concerns.

1:25 We should recognize this stuff is an existential threat. And we have to face the possibility that unless we do something soon When maybe yeah. So let's do the risk in what we end up doing in such a world. Yeah. Quick one before we get back to this episode. Just give me thirty seconds of your time.

1:43 Two things I wanted to say. The first thing is a huge thank you for listening and tuning into the show week after week. It means the world to all of us, and this really is a dream that we absolutely never had and couldn't have imagined getting to this place. But secondly, it's a dream where we feel like we're only just getting started. And if you enjoy what we do here, please join the twenty four percent of people that listen to this podcast regularly and follow us on this app. Here's a promise I'm gonna make to you. I'm gonna do everything in my power to make this show as good as I can now and into the future. We're gonna deliver the guests that you want me to speak to, and we're gonna continue to keep doing all of the things you love about this show. Thank you.

2:21 Thank you so much. Back to the episode. Jeffrey Kinson. They call you the godfather of AI.

2:32 Uh yes, they do. Why do they call you that? There weren't that many people who believed that we could make neural networks work, artificial neural networks. So for a long time in AI

2:43 From the nineteen fifties onwards. There were kind of Two ideas about how to do AI. One idea was that sort of core of human intelligence was reasoning.

2:54 And to do reasoning you need to use some form of logic. And so AI had to be based round logic. And In your head you must have something like symbolic expressions that you're manipulated with rules. And that's how intelligence worked.

3:08 And things like learning or reasoning by analogy, that all come later once we've figured out how basic reasoning works. There was a different approach, which is to say Let's model AI on the brain. 'Cause obviously the brain Makes us intelligent.

3:23 So Simulate A network of brain cells on a computer. And try and figure out how you would learn strengths of connections between brain cells so that it learned to do complicated things. Like recognize objects and images or recognized speech.

3:38 Mm. Even do reasoning. I pushed that approach for like fifty years. Because so few people believed in it. There weren't many good universities that had groups that did that.

3:49 So If you did that. The best young students who believed in that came and worked with you. So I was very fortunate in getting uh whole lot of really good students.

3:59 Some of which have gone on to create And playing an instrumental world and creating. Platforms like OpenAI. Yeah, so your suscover will be here. A nice example. A whole bunch of them.

4:09 Why did you believe that modelling it off the brain was a more effective approach. It wasn't just me believed it. Early on For Neumann believed it. And Turing believed it.

4:21 And if either of those had lived, I think AI would have had a very different history. But they both died young. You think AI would have been here sooner? I think neural net the neural net approach would have been Except it much sooner if I'd lived.

4:36 In this season of your life, what Mission are you on? My main mission now is to warn people. How dangerous AI could be. Did you know that when you

4:48 Became the godfather of AI. No, not really. I was quite slow to understand some of the risks. Some of the risks were always very obvious, like people would use AI to make autonomous lethal weapons. That is things that go around deciding by themselves who to kill. Other risks like the idea that they will one day get smarter than us. And maybe we'd become irrelevant.

5:10 I was slow to recognize that. Other people recognized it. Twenty years ago. I only recognised a few years ago that that was a real risk that was come might be coming quite soon.

5:21 How could you Not have foreseen that if With everything you know here about cracking the ability for these

5:28 computers to learn similar to how humans learn. And just You know, introducing any rate of improvement. It's a very good question. How could you not have seen that? But Remember neural networks twenty, thirty years ago.

5:41 We're very primitive in what they could do. They were nowhere near as good as humans. But things like vision and language and speech recognition. The idea that you have to now worry about it getting smarter than people, that seems silly then. When did that change?

5:55 It changed for the general population when ChatGPT came out. A change for me When I realised that The kinds of digital intelligences we're making have something that makes them far superior to the kind of biological intelligence we have.

6:11 If I want to share information with you So I go off and I learn something. Mm. And I'd like to tell you what I learned. So I produce some sentences.

6:20 This is a rather simplistic model, but roughly right. Your brain is trying to figure out how can I change the strengths of connections between neurons? So I might have put that word next. And so you'll do a lot of learning when a very surprising word comes. And not much learning when if it's when it's very obvious word. If I say fish and chips You don't do much learning when I say chips.

6:37 But if they fish and cucumber, you do a lot more learning. You wonder why did I say cucumber? So that's roughly what's going on in your brain. I'm predicting what's coming next. That's how we think it's working. Nobody really knows for sure how the brain works. And nobody knows how it gets the information about whether you should increase the strength of a connection or decrease the strength of a connection.

6:58 That's the crucial thing. But what we do know now from AI is that if you could get information About whether to increase or decrease the connection strength. So as to do better whatever tasks you're trying to do.

7:11 Then We could learn incredible things'cause that's what we're doing now with artificial neuron ads. It's just we don't know for real brains how they get that signal about whether to increase or decrease. As we sit here today, what are the big concerns you have around safety of AI? If we were to To list the

7:28 The top couple. that are really front of mine and that we should be thinking about. Um can I have more than a couple? Go ahead, I'll write them all down and we'll go through them. Okay, first of all I wanna make a distinction between two completely different kinds of risk.

7:43 There's risks that come from people misusing AI. Yeah. And that's most of the risks. And all of the short term risks. And then there's risks that come from AI getting super smart.

7:55 And signing it doesn't need us. Is that a real risk? And I talk mainly about that second risk because lots of people say, Is that a real risk? And yes it is. No, we don't know how much of a risk it is. We've never been in that situation before. We've never had to deal with things smarter than us.

8:12 So Really the thing about that existential threat. Is that We have no idea how to deal with it. We have no idea what it's gonna look like.

8:22 And anybody who tells you they know just what's going to happen and how to deal with it, they're talking nonsense. So we don't know how to estimate the probability. probabilities it'll replace us. Um some people say it's like less than one percent. My friend Yan Le Cart. who's a post start with me, thinks no, no, no, no, we're always gonna be We build these things, we're always gonna be in control.

8:42 We'll build them to be obedient. And Other people. Like Yuski. Say, No, no, no, these things are gonna wipe us out for sure. If anybody builds it, it's gonna wipe us all out.

8:54 And he's confident of that. I think both of those positions are extreme. It's very hard to estimate the probabilities in between. If you had to bet. On who was right out of your two friends.

9:07 I simply don't know. So if I had to bet I'd say the probabilities in between And I don't know where to estimate it in between. I often say ten to twenty percent chance their wipe is out. But That's just Got.

9:20 Based on the idea that we're we're still making them. And we're pretty ingenious. And the hope is that if enough smart people do enough research with enough resources We'll figure out a way to build them so they'll never want to

9:33 Thomas. Sometimes I think if we we talk about that second um path, sometimes I think about nuclear bombs and the the invention of the atomic bomb and how it compares. Like how is this different? Because the atomic bomb came along and I imagine a lot of people at that time thought our days are numbered. Uh yes, I was there. We did. Yeah. But but but what's what We're still here.

9:54 We're still here, yes. So The atomic bomb was really only good for one thing. And it was very obvious how it worked, even w if you hadn't had the pictures of Hiroshima and Nagasaki It was obvious that it was a very big bomb.

10:08 That was very dangerous. With AI It's good for Many, many things. It's gonna be magnificent in healthcare and education.

10:19 And more or less any industry that needs to use its data. It's gonna be able to use it better with AI. So we're not gonna stop the development. You know, people say, Well, why don't we just stop it now?

10:31 We're not Too good for too many things. Also, we're not gonna stop it'cause it's good for battle robots and none of the countries that sell weapons are gonna want to Stop it.

10:43 Like the European regulations. They have some regulations about AI. It's good they have some regulations. But they're not designed to deal with most of the threats. And in particular The European regulations have a

10:55 Uh clause in them that say none of these regulations apply to military uses of AI. So governments are willing to regulate regulate companies and people, but they're not willing to regulate themselves. It seems pretty crazy to me that they I go back and forward, but if Europe has a regulation but the rest of the world doesn't. Yeah. So we're seeing this already. I don't think people realize that when OpenAI release a new model.

11:23 or a new piece of software in America. They can't release it to the to Europe yet because of regulations here. So Sam Altman tweeted saying On you AI agent thing is available to everybody, but it can't come to Europe yet because there's regulations.

11:36 Yeah. Productivity disadvantage. Right. What we need is I mean At this point in history, when we're about to produce things more intelligent than ourselves What we really need is

11:49 is a kind of world government that works run by intelligent, thoughtful people. And that's not what we got. So Free for all. Well that what we've got is

12:00 Sort of We've got capitalism. Which is done very nicely by us. It's produced lots of goods. Goods and services for us. But

12:10 These big companies They're legally required to try and maximise profits. And that's not what you want from the people developing this stuff. So let's do the risks then. You talked about there's human risks and then there's So I've distinguished these two kinds of risk. Let's talk about all the risks from Bad human actors using AI.

12:30 There's cyber attacks. So between twenty twenty three and twenty twenty four. They increased by about a factor of twelve, twelve hundred percent. And that's probably because

12:45 These large language models make it much easier to do phishing attacks. And f fishing attack for anyone that doesn't know is it's they send you something saying uh Hi, I'm your friend John and I'm stuck in El Salvador. Could you just wire this money? That's one kind of attack. But the fishing attacks are really trying to get your log on. Credentials. And now with AI they can clone my voice, my image. They can do all that.

13:08 I'm struggling at the moment because there's a bunch of AI scams on X and also Meta. And there's one in particular on Meta, so Instagram, Facebook at the moment, which is a paid advert. Where they've taken my voice from the podcast. They've taken the my mannerisms and they've made a new video of me encouraging people to go and Take part in this. Crypto Punzi scam or whatever.

13:27 And we've been, you know, we spent weeks and weeks and weeks and weeks in end emailing Meta, telling please take this down. They take it down, another one pops up. They take that one down, another one pops up. So it's like whack-amole. And then very annoying. The the heartbreaking part is you get the messages from people that have fallen for the scam. And they've lost five hundred pounds or five hundred dollars. And they're crossed with you'cause you recommended it. And I'm I'm like I'm sad for them. It's very annoying. Yeah. A smaller version of that which is Pew some people now publish papers.

13:53 With me as one of the authors. Mm-hmm. And it looks like it's in order that they can get lots of citations to themselves. So cyber attacks a very real threat. There's been an explosion of those. Already

14:07 Obviously I is very patient, so they can go through a hundred million lines of code. Looking for known ways of attacking them. That's easy to do. But they're gonna get more creative. And they may Some people believe and I

14:21 Some people who know a lot believe that maybe by twenty thirty They'll be creating new kinds of cyber attacks. Which no person ever thought of. So That's very worrisome. Because they can think for themselves and decide to do it. They can

14:37 draw new conclusions from much more data than a person ever saw. Is there anything you're doing? To protect yourself from cyber attacks at all. Yes. It's one of the few places where I

14:49 Change what I do radically. Because I'm scared of cyber attacks. Canadian banks are extremely safe. In two thousand and eight, no Canadian banks Came anywhere near going bust.

15:00 Yeah. So they're very safe banks'cause they're well regulated. Fairly well recognition. Nevertheless I think a cyber attack might be able to bring down a bank. No.

15:10 If you have All my savings are in shares in banks. Held by banks. So if the bank Gets attacked.

15:18 And it holds your shares. They're still your shares. And so I think you'd be okay. Unless The attacker.

15:26 Sells the shares,'cause the bank can sell the shares. If the attacker sells your shares I think you're screwed. I don't know, I mean, maybe the bank would have to try and reimburse you, but the bank's bust by now, right? So So I'm worried about a Canadian bank being

15:42 taken down by a cyber attack. And the attacker selling selling shares that it holds. So I spread my money. My children's money between three banks. In the belief that

15:54 If a cyber attack takes down one Canadian bank The other Canadian banks will very quickly get very careful. And do you have a phone that's not connected to the internet? Do you have any like you know, I'm thinking about storing data and stuff like that. Do you think it's wise to consider having cold storage?

16:11 I have a little disk drive and I back up my laptop. On this hard drive. So I actually have everything on my laptop on a hard drive. At least, you know, if the whole internet went down, I had the sense I still got it on my laptop and I still got My information checked.

16:28 Then the next thing is Using AI is to create nasty viruses. Okay. And the problem with that is That requires

16:38 Just requires One crazy guy with a grudge. One guy who knows a little bit of molecular biology Knows a lot about AI. And just wants to destroy the world.

16:49 You can now create New viruses. relatively cheaply using AI. And you don't have to be a very skilled molecular biologist to do it. And that's very scary. So you could have a small cult, for example.

17:02 A a small cult might be able to raise him. A few million dollars. For a few million dollars they might be able to design a whole bunch of viruses. Well I'm thinking about some of our foreign adversaries. doing government funded programs. I mean, there's lots of talk around Covid and Wu the Wu Han laboratory and what they were doing and gain of function research, but I'm wondering if in, you know, a China or a Russia or an Iran or something

17:23 the government could fund a a programme for a small group of scientists to make a virus that they could, you know I think they could, yes. No. They'd be worried about retaliation. They'd be worried about other governments doing the same to them. Hopefully that would help keep it under control. They might also be worried about the virus spreading to their country. Okay.

17:42 Then there's um Corrupting elections. Yeah. So if you wanted to use AI to corrupt elections A very effective thing is to be able to do targeted political advertisements.

17:55 Where you know a lot about the person. So Anybody who wanted to use AI for corrupting elections would try and get as much data as they could About everybody in the electorate. With that in mind, it's a bit worrying what Musk is doing at present in the States.

18:14 going in and insisting on getting access to all these things that were very carefully siloed. The claim is it's to make things more efficient. But it's exactly what you would want if you intended to corrupt the next election. How do you mean'cause you get all this You got all this data on people. Yeah. You know how much they make, where they you know everything about them.

18:32 Once you know that, it's very easy to manipulate them. Because you can make an AI that's you can send messages Um that they'll find very convincing telling them not to vote, for example. So I have no no reason other than common sense to think this.

18:50 But I wouldn't be surprised if part of the motivation of getting all this data from American government sources. Is to corrupt elections. Another part might be That it's very nice training data for a big model.

19:04 But he would have to be taking that data from the government and feeding it into his And what they've done is Turn off lots of the security controls. Got rid of the some of the organization to protect against that. Um so that's corrupting elections.

19:19 Okay. Then there's creating These two echo chambers. Bye. Organizations like YouTube. And Facebook.

19:31 showing people things that will make them indignant. People love to be indignant. Indignant as in angry. What does anything mean? Feeling I'm

19:41 Sort of angry but feeling righteous. Okay. So for example, if you were to show me something that said Trump did this crazy thing. Here's a video of Trump doing this completely crazy thing. I would immediately click on it. Okay, so putting us in echo chambers and dividing us. Yes. And that's Um

20:01 the policy that YouTube and Facebook and others use for deciding what to show you next. Is causing that. If they had a policy of show you

20:13 balance things, they wouldn't get so many clicks and they wouldn't be able to sell so many advertisements. And so it's basically the profit motive is saying Show'em whatever will make'em click. What will make'em click is Things that are more and more extreme.

20:28 And that confirmed my existing bias. That confirmed my existing bias. So you're getting your biases confirmed all the time. further and further and further and further, which means you're d you're driving away. I'm not sure people realise that this is actually happening every time they open an app, but if you go on a TikTok or a YouTube or or one of these big social networks The algorithm, as you you said, is designed to show you more of the things that you w had interest in last time. So if you just play that out over ten years, it's gonna drive you further and further and further into whatever ideology or belief you have and further away from nuance and common sense and um parity. Which is a pretty remarkable thing. I I like people don't know it's happening. They just open their phones.

21:10 And experience something and think this is The news. Or the experience everyone else is having. Right. So Basically if you have a newspaper and everybody gets the same newspaper

21:21 Yeah. You get to see all sorts of things you weren't looking for. And you get a sense that if it's in the newspaper, it's an important thing or significant thing. But if you have your own newsfeed My newsfeed on my iPhone. Three quarters of the stories are about AI. And I find it very hard to know.

21:38 If the whole world's talking about AI all the time, or if it's just my newsfeed. Mm. Okay, so driving me into my echo chambers, um, which is gonna continue to divide us further and further, I'm actually noticing that the algorithms are becoming even more What's the word? Tailored.

21:56 And people might go, Oh, that's great. But what it means is they're becoming even more personalised, which was is means that my reality is becoming even further from your reality. Yeah. It's crazy. We don't have a sharity anymore. I share reality with other people who watch the BBC and other

22:13 BBC News and other people who read The Guardian and other people who read the New York Times. I have Almost no sharity with people who watch Fox News. It's pretty it's pretty um It's worrisome.

22:27 Yeah. behind all this is the idea that these companies just want to make profit and they'll do whatever it takes to make more profit. Because they have to. They're legally obliged to do that. So we almost can't pr blame the company, can we?

22:41 If the if it's Capitalism's done very well for us. It's produced lots of goodies. Yeah. But you need to have it very well regulated. So what you really want is to have rules so that When some company is trying to make as much profit as possible.

22:58 In order to make that profit, they have to do things that are good for People in general. not things that are bad for people in general. So once you get to a situation where in order to make more profit The company starts doing things that are very bad for society. Like showing you things that are more and more extreme.

23:15 That's what regulations are for. So You need regulations with capitalism. Now companies will always say Regulations get in the way, make us less efficient, and that's true. The whole point of regulations is to stop them doing things to make profit that hurt society. And we need strong regulation.

23:33 Who's gonna decide whether it has society or not? Because you know. That's the job of politicians. Unfortunately, if the politicians are owned by the companies, that's not so good. And also the politicians might not understand the technology. We've thought you've probably seen the Senate hearings where they wheel out, you know, Mark Zuckerberg and these big tech CEOs. And it is quite embarrassing because they're asking their own questions. Well I've seen the video of the

23:55 US education secretary. Talking about how they're gonna get AI in the classrooms. Except she thought it was called A one. She's actually there saying we're gonna have

24:06 All the kids interacting with A one. School system that's gonna start um Making sure that first graders or even pre K have A one teaching Yeah, every year starting

24:18 You know that far down in the grades. And that's just a That's a wonderful thing. And these are well these are the people that

24:28 These are the people in charge. Ultimately the tech companies are in charge because they were unsmart. The tech company's in the States now. At least a few weeks ago when I was there. They were running an advertisement about how it was very important not to regulate AI because it would hurt us in the competition with China.

24:47 Yeah. And that's a that's a plausible argument, there. Yes, it will. But you have to decide. Do you want to compete with China?

24:55 By doing things that will Two A lot of harm to your society. And you probably don't. I guess they would say that

25:06 It's not just China, it's Denmark and Australia and Canada and they're not so worried about and Germany. But if they kneecap themselves with regulation if they slow themselves down, then the founders, the entrepreneurs, the investors are gonna go. I think calling it kneecapping is sort of taking a particular point of view. It's take taking the point of view that regulations are sort of very harmful. What you need to do is just constrain the big companies so that in order to make profit They have to do things that are socially useful. Like Google search is a great example. That didn't need regulation because it just made information available to people. It was great.

25:42 But then if you take YouTube which starts Showing you adverts. And showing you more and more extreme things. That needs regulation. But we don't have the people to regulate it.

25:53 I think people know pretty well. Um That particular problem of showing you more and more extreme things, that's well a well known problem that the politicians understand. It they just um need to get on regulated.

26:06 So that was the the next point, which was that the algorithms are gonna drive us further into our echo chambers. Right. What's next? Lethal autonomous weapons. Lethal autonomous weapons.

26:19 That means Things that can kill you and make their own decision about whether to kill you. Which is the great dream, I guess, of the military industrial complex. So The worst thing about them is

26:33 Big powerful countries always have the ab ability to invade smaller, poorer countries. They're just more powerful. Mm-hmm. If you do that using actual soldiers. Yeah, but he's coming back in bags.

26:48 And the relatives of the soldiers who were killed don't like it. So you get something in Vietnam. Yeah. In the end there's a lot of protest at home. If instead of boding back in bags, it was dead robots

27:03 There'd be much less protest. And the military industrial complex would like it much more because robots are expensive. And Suppose you had something that could get killed and was expensive to replace. That would be just great.

27:17 Big countries. can invade small countries much more easily because they don't have their soldiers being killed. And the risk here. Is Uh.

27:26 These robots will be a little bit more. Malfunction or they'll just be more than No. That's even if the robots do exactly what the people who built the robots want them to do. The risk is that it's gonna make Big countries invade small countries more often. More often because they can. Yeah. And it's not a nice thing to do. So it brings down the friction of war. It brings down the cost of doing an invasion. Mm-hmm.

27:46 But and these machines will be smarter at warfare as well. So they'll be when the machines aren't smarter. So the lethal autonomous weapons They can make them now. And They I think all the big defence departments are busy making them. Even if they're not smarter than people, they're still very nasty scary things.

28:04 'Cause I'm thinking that you know, they could show just a picture. Go get this guy. Yeah. And go take out. Uh anyone he's been texting. And this little wasp.

28:14 So two days ago I was visiting a friend of mine in Sussex. Who had a drone that cost less than two hundred pounds. And The drum went up. It took a good look at me.

28:26 And then you could follow me through the woods. Mm-hmm. And it follow it was very spooky having this drone. It was about two meters behind me. It was looking at me. If I move over there, move over there. It could just track me. Mm-hmm. For two hundred pounds.

28:39 But it was already quite spooky. Yeah, and I I imagine there's as you say a race going on as we speak to who can build the most complex autonomous Autonomous weapons. There is a a risk I often hear that some of these things will combine. And

28:53 The cyber attack. Or release weapons. Sure. Um you can you can Get combinatorially many risks by combining these other risks. So I mean, for example, you could get

29:06 A super intelligent AI That decides to get rid of people. And the obvious way to do that is just to make one of these nasty viruses. If you made a virus that was

29:17 very contagious, very lethal, and very slow. Everybody would have it before they realised what was happening. I mean, I think if a superintelligence wanted to get rid of us. It will probably go for something biological like that that wouldn't affect it. Do you think it could just very quickly turn us against each other, for example, it could Send a warning.

29:36 on the nuclear systems in America that there's a nuclear bomb coming from Russia. Or vice versa, and one retaliates. Yeah. I mean my basic view is there's so many ways in which the superintelligence could get rid of us. It's not worth speculating about. What what is Well, you have to do is.

29:54 Prevented to. That's what we should be doing research on. There's no way we're gonna prevent it. From it's smarter than us, right? There's no way we're gonna prevent it getting rid of us if it wants to. We're not used to thinking about things smarter than us.

30:09 If you wanna know what life's like When you're not the Apex intelligence. Ask a chicken. Yeah, I was thinking of my dog Pablo, my French Bulldog, this morning as I left home. He has no idea where I'm going.

30:24 He has no idea what I do. Right. Yeah. And the g the intelligence gap will be like that. So you're telling me that if I'm Pablo, my French Bulldog?

30:34 I need to figure out a way to make My owner. Not wipe me out. Yeah. So we have one example of that, which is mothers and babies.

30:45 Evolution put a lot of work into that. Mothers are smarter than babies, but babies are in control. And they're in control'cause the mother just can't bear lots of hormones and things. But the Babe the mother just can't bear the sound of the baby crying. Not all mothers. Not all mothers. And then the baby's not in control and then bad things happen.

31:03 We somehow need How to make them not want to take over? The analogy I often use is Forget about intelligence, just think about physical strength.

31:14 Suppose you have a nice little tiger cub. It's sort of bit bigger than a cat. It's really cute. It's very cuddly. Very interesting to watch. Accept that you better be sure that when it grows up it never wants to kill you'cause if ever wanted to kill you.

31:28 You'll be dead in a few seconds. And you're saying the AI we have now is the target cup? Yeah. And it's growing up. Yeah.

31:37 So we need to train it as it's When it's a bean. It's not a safe thing to have around. But lions, people that have lions as pets. Yes. Sometimes the lion is affectionate to its creator, but not to others. Yes.

31:52 And we don't know whether these AIs. We we simply don't know whether we can make them not want to take over and not want to hurt us. Do you think we can? Do you think it's possible to train super intelligence? I don't think it's clear that we can. So I think it might be hopeless. But I also think

32:10 We might be able to. And it'd be sort of crazy if people went extinct'cause we couldn't be bothered to try. If that's even a possibility. How do you feel about your life's work? Because you were Yeah.

32:23 Um, it sort of takes the edge off it, doesn't it? Yeah. I mean the idea is gonna be wonderful in healthcare and wonderful in education. And wonderful. I mean it's gonna make call centers much more efficient. No one worries a bit about what the people who are doing that job I do. It makes me sad.

32:39 I don't feel particularly guilty about developing AI like Forty years ago. Because At that time we had no idea that this stuff was gonna happen this fast. We thought we had plenty of time to worry about things like that. They when you when you can't get the I to do much and you want to get it to do a little bit more, you don't worry about

32:59 This stupid little thing is gonna take over from people. You just wanted to be able to do a little bit more of the things people can do. It's not like I knowingly did something. Thinking this might wipe us all out, but I'm gonna do it anyway. Mm-hmm. But it is a bit sad that it's not just gonna be something for good.

33:19 So I feel I have a duty now to talk about the risks. And if you could play it forward and you could go forward thirty, fifty years and you found out that it led to the extinction of humanity. And if that does end up being Being the outcome. Well if you played it forward and

33:38 It led to the extinction of humanity. I would use that to tell people to tell their governments that we really have to work on how we're gonna keep this stuff under control. I think we need people to tell governments the governments have to force the companies to use their resources to work on safety. And they're not doing much of that because you don't make profits that way.

34:00 One of your your students we talked about earlier. Um Ilia? Yep. Ilya left. Open AI.

34:08 Yeah. And there was lots of conversation around The fact that he left because he had safety concerns. Yes. And he's gone on to set set up a AI safety company.

34:18 Yes. Why do you think he left? I think he left because he had safety concerns. Really? Um I still Have lunch with him from time to time. His parents live in Toronto and when he comes to Toronto we have lunch together.

34:31 He doesn't talk to me about what went on at OpenAI, so I have no inside information about that. But I know Ilya very well. And he is genuinely concerned with safety. So I think that's why he left. Because he was one of the top people. I mean he was He was

34:44 probably the most important person behind the development of Um chat GPT. The the early versions like GPT two. It was very important in the drawing to that. You know him personally, so you know his character.

34:57 Yes, he has a good moral compass. He's not like Someone like Muscle has no moral compass. Does Sam Waltman have a good moral compass? We'll see. I don't know Sam, so I

35:11 Don't want to comment on that. But from what you've seen. Are you concerned about The actions that they've taken. Because if you know Eliya and Eli's a good guy and he's left.

35:21 Yeah. That would give you some insight, yes. It would give you some reason to believe that There's a problem there. And if you look at Sam's statements

35:31 Some years ago. He sort of happily said in one interview, um this stuff will probably kill us all. That's not exactly what he said, but that's what it amounted to. Now he's saying you don't need to worry too much about it.

35:43 And I suspect that's not Driven by Seeking after the truth. That's driven by seeking after money. Is it money or is it Power.

35:55 Yeah, I shouldn't have said money. It's some some combination of this, yes. Okay, I guess money's a proxy for power, but I am I've got a friend who's a billionaire and he is in those circles. And when I went to his house and had uh lunch with him one day, he knows lots of people in AI, building the biggest AI companies in the world. And he gave me a cautionary warning across the

36:15 across his kitchen table in London. where he gave me an insight into the private conversations these people have, not the media interviews they do where they talk about safety and all these things, but actually what some of these individuals think is gonna happen. And what do they think is gonna happen? It's not what they say publicly. Yeah, one one.

36:33 person who I sh pr shouldn't name, who is the who's leading one of the biggest AI companies in the world. He told me that he knows this person very well, and he privately thinks that we're heading towards this kind of dystopian world where we have Just huge amounts of free time, we don't work anymore. And This person doesn't really give a fuck about the harm that it's gonna

36:51 have on the world and this person who I'm referring to is building one of the biggest AI companies in the world. And I then watch this person's interviews online. Trying to figure out which of three people it is. Yeah, well it's one of those three people. Okay. And I watch this person's interviews online and I I reflect on a conversation that my billionaire friend had with me. Who knows him? And I go, Fucking hell, this guy's lying publicly, like He's not telling the the truth to the world.

37:11 And that's haunted me a little bit. It's part of the reason I have so many conversations around AR in this podcast, because I'm like, I don't know if they're I think they're a lit some of them are a little bit sadistic about power. I think they they like the idea that they will change the world. That they will be the one. Yeah.

37:28 Fundamentally shifts the world. I think Musk is clearly like that, right? He's such a complex uh character that I don't I don't really know how to place Musk. Um He's done some really good things like um Pushing Electric cars.

37:43 That was a really good thing to do. Yeah. Some of the things he said about self driving were a bit exaggerated, but he That was a really useful thing you did. Giving the Ukrainians communication during the war with Russia. Starling. That was a really good thing he did. There's a bunch of things like that.

38:00 Mm. Um but he's also done some very bad things. So coming back to this point of The possibility of Destruction. And the motives of these big companies.

38:17 Are you at all hopeful that anything can be done? To slow down the pace and acceleration of AI. Okay, there's two issues. One is can you slow it down? Yeah. On the other is can you make it so

38:28 But it will be safe in the end. It won't wipe us all out. I don't believe we're gonna slow it down. And the reason I don't believe we're gonna slow it down is because this competition between countries and competition between companies within a country.

38:43 And all of that is making it go faster and faster. And if the US slowed it down. China wouldn't slow it down. Does Ilya.

38:51 think it's possible to make AI safe. I think he does. He won't tell me what his secret source is. I I'm not sure how many people know what a secret source is. I think a lot of the investors don't know what a secret source is, but they're given in billions of dollars anyway.

39:07 'Cause they have so much faith in eager. Which isn't foolish. I mean He was very important in Alex Met, which got object recognition working well. He was The main

39:18 The main force behind the Things like GPG two. Which then led to Chat GPT. So

39:26 I think having a lot of faith in Ellie is a very reasonable decision. There's something quite haunting about the guy that made And was the main force behind GPT too, which led rise to this whole revolution. left the company.

39:39 Because of safety reasons. He knows something that I don't know. About what might happen next. The company had No, I don't know the precise details.

39:49 Um but I'm fairly sure the company had indicated that would it would use a significant fraction of its resources Of the compute time. For doing safety research. And then it kept then it reduced that fraction. I think that's one of the things that happened. Yeah, that was reported publicly. Yes.

40:05 Yeah. We've got into the autonomous weapons. Part of the risk. Framework. Right. So the next one is joblessness.

40:15 Yeah. In the past, new technologies have come in. Which didn't lead to joblessness, new jobs were created. So the classic example people use is automatic telling machines. When automatic telemachines came in.

40:28 A lot of bank tellers didn't lose their jobs. They just got to do more interesting things. But here's I think this is more like When they got machines in the industrial revolution. And

40:42 You can't have a job digging ditches now, because a machine can dig ditches much better than you can. Yeah. And I think for mundane intellectual labour AI is just gonna Replace everybody.

40:55 No, it will May well be in the form of you have fewer people using AI assistance. So it's a combination of a person and an AI assistant. And I'm doing the work that ten people could do previously. People say that it will create new jobs though, so we'll be fine.

41:11 Yes, and that's been the case for other technologies, but this is a very different kind of technology. If it can do all mundane human intellectual labour. Then what new jobs is it going to create? You'd ha you'd have to be very skilled to have a job that it couldn't just do. So I don't th I don't think they're right.

41:30 I think you can try and generalize from Other technologies have come in, like computers. Automatic channel machines, but I think this is different. People use this phrase, they say, AI won't take your job. A human using AI will take your job. Yes, I think that's true.

41:44 But for s many jobs That'll mean you need far fewer people. Maj. Answers letters of complaint to a health service. It used to take twenty five minutes.

41:55 She'd read the complaint and she'd think how to reply and she'd write a letter and Now she just scans it into Um a chatbot. And It writes the letter. She just checks the letter. Occasionally she tells it to

42:10 Revised it in some ways. The whole process takes five minutes. That means she can answer five times as many letters. And that means they need five times fewer of her. So she can do the job that five of her used to do.

42:25 Now That will mean they need less people. In other jobs, like in healthcare. They're much more elastic.

42:33 So if you could make doctors five times as efficient. We could all have five times as much health care for the same price. And that would be great. This There's almost no limit to how much health care people can absorb.

42:45 Mm. They always want more health care. If it's There's no cost to it. There are jobs where you can make a person with an AI assistant Much more efficient and you won't need to

42:56 Less people. Because you'll just have much more of that being done. But Most jobs I think are not like that. Am I right in thinking the sort of industrial revolution?

43:06 W played a role in replacing Muscles. Yes, exactly. And this revolution in AI replaces intelligence, the brain. Yeah. So mundane intellectual labor is like having strong muscles. And

43:18 It's not worth much anymore. So muscles have been replaced. Now we intelligence is being replaced. Yeah. So what remains? Maybe for a while some kinds of creativity. But the whole idea of superintelligence is nothing remains.

43:33 Um these things will get to be better than us at everything. So what what do we end up doing in such a world? Well, if they work for us. We end up Getting lots of goods and services for not much effort.

43:46 Okay. But that sounds like Tim dang and nice, but I don't know, there's a cautionary tale in creating more and more ease for humans and in it going

43:55 Badly. Yes. And We need to figure out if we can make it go well. So the the nice scenario is imagine a company with

44:04 A CEO. Who is very dumb. Probably the son of the former CEO. Mm-hmm. And he has an executive assistant.

44:13 Who's very smart. And He says. I think we should do this. And the executive assistant makes it all work.

44:21 The CO feels great. He doesn't understand that he's not really in control. And in in some sense he is in control. He suggests what the company should do. She just make it all work. Everything's great. That's the good scenario. And the bad scenario? The bad scenario, she thinks why do we need him? Yeah.

44:42 I mean In a world where we have superintelligence, which you don't believe is that far away. Yeah, I think it might not be that far away. It's very hard to predict, but I think we might get it in like twenty years or even less.

44:54 I made the biggest investment I've ever made in a company because of my girlfriend. I came home one night and my lovely girlfriend was up at one a.m. in the morning pulling her hair out as she tried to piece together her own online store for her business. And in that moment, I remembered an email I'd had from a guy called John. The founder of Stanstore. Our new sponsor and a company I've invested heavily in. And Stan still helps creators to sell digital products, courses, coaching, and memberships all through a simple customizable link in bio system. And it handles everything: payments, bookings, emails, community engagement, and even links with Shopify. And I believe in it so much. Then I'm going to launch

45:35 A stanza. Challenge. And as part of this challenge, I'm gonna give away one hundred thousand dollars to one of you. If you want to take part in this challenge, if you want to monetize the knowledge that you have, visit Stephen Bartlett. Dot stan dot store to sign up. And you'll also get an extended thirty day free trial of Stan Store if you use that link. Your next move.

45:56 Could quite frankly change everything. Because I talked about ketosis on this podcast in ketones, a brand called Ketone IQ sent me their little product here. And it was on my desk when I got to the office. I picked it up, it sat on my desk for a couple of weeks, then one day I tried it. And honestly, I have not looked back ever since.

46:14 I now have this everywhere I go. When I travel all around the world, it's in my hotel room, my team will put it there. Before I did the podcast recording today that I've just finished. I had a shot of ketone IQ. And as is always the case when I fall in love with a product, I called the CEO and asked if I could invest a couple of million quid into their company. So I'm now an investor in the company, as well as them being a brand sponsor. I find it so easy to drop into deep focus work when I've had one of these. I would love you to try one. and see the impact it has on you, your focus, your productivity, and your endurance. So if you want to try it today, visit ketone.com slash steven for thirty percent off your subscription. Plus you'll receive a free gift with your second shipment. That's ketone.com slash Steven. I'm excited for you. I am.

46:58 So what's the difference between what we have now and super intelligence? Because it seems to be really intelligent to me when I use like Chat GPT three three O or Gemini or Okay, so it's already AI is already better than us at a lot of things. In particular areas. Like chess, for example. Yeah. AI is so much better than us that

47:16 People will never beat those things again. Maybe the occasional. When but Basically. It'll never be comfortable again.

47:23 Obviously the same in go. In terms of the amount of knowledge they have, Um, something like G V G four knows thousands of times more than you do. There's a few areas in which your knowledge is better than it's. I don't know.

47:36 Almost all areas, it just knows more than you do. What areas am I better than it? Probably in Interview. Cs.

47:46 You're probably better at that. You got a lot of experience at it. You're a good interviewer. You know a lot about it? If you tried if you got G V T four to interview a CEO, probably do a worse job. Okay.

48:00 Yeah. I'm trying to think if that if I agree with that statement. Uh GPT four, I think, for sure. Yeah. Um but I but I guess you could Yeah, I guess you could train one on this th how I ask questions and what I do and sure. And if you took a general purpose

48:16 sort of foundation model. And then you trained it up on Not just you, but every every interview you could find. Doing interviews like this. Mm-hmm. But especially you.

48:26 You'll probably get to be quite good at doing your job, but probably not as good as you for a while. Okay. So there's a few areas left. And then superintelligence becomes when it's Better than us at all things. When it's much smarter than you and almost all things it's better than you, yeah.

48:42 And you you you say that this might be a decade away or so. Yeah, it might be. It might be even closer. Some people think it's even closer. I might well be much further. It might be fifty years away. That's still a possibility. It might be that somehow

48:57 training on human data limits you to not being much smarter than humans. My guess is between ten and twenty years we'll have superintelligence. On this point of joblessness. It's something that I've been thinking a lot about in particular because I started messing around with AI agents. And we released an episode on the podcast actually this morning where we had a debate about AI agents with some CEO of a big A AI agent company and a few other people. And it was the first moment where I had

49:22 Yeah. It was another moment where I had a Eureka moment about what the future might look like, when I was able in the interview to tell this agent to order all of us drinks and then five minutes later in the interview you see the guys show up with the drinks. And I didn't touch anything. I just told it to order us drinks to the studio.

49:38 And you didn't know about who you normally got your drinks from. It figure that out from the web. Yeah, figure it out'cause it went on Ubreets. It has my My data, I guess. I mean I we put it on the screen in real time so everyone at home can see the agent going through the internet, picking the drinks, adding a tip for the driver.

49:53 Putting my address in, putting my credit card detain in, and then the next thing you see is the drinks show up. So that was one moment. And then the other moment was when I use a to called Replet. And I built software. By just telling the agent what I wanted. Yes.

50:07 It's amazing, right. It's amazing and terrifying at the same time. Yes. Because And if you can build software like that, right? Yeah. Remember that the AI, when it's training, is using code.

50:21 And if it can modify its own code. Then it gets quite scary, right? It can change itself in a way we can't change ourselves. We can't change our innate endowment, right? There's nothing about itself that it couldn't change.

50:37 On this point of joblessness, you have kids. I do. And they have kids? No. They don't have kids, they're no grandkids yet. What would you be saying to people about their career prospects in a world of super intelligence. What should we we be thinking about?

50:49 Um in the meantime, I'd say it's gonna as good at physical manipulation as us. Okay. And so

50:58 A good bet would be to be a plumber. Until the humanoid robots show up. In such a world where there is mass joblessness, which is not something that you just predict, but this is something that Sam Altman, OpenAI, I've heard him predict, and many of the CEOs, I mean Elon Musk, I watched an interview which I'll play on screen of him being asked this question. And it's very rare that you see Elon Musk silent for twelve seconds or whatever it was. Right.

51:22 And then He basically says something about he actually is living in suspended disbelief. I he's basically just not thinking about it. When you think about advising your children on a career with so much that is changing What do you tell them there's gonna be a value?

51:48 Well That is a tough question to answer. I would just say, you know, to to sort of follow their heart in terms of what they they find um interesting to do or fulfilling to do. I mean if I think about it too hard it can frankly it can be uh just dispariting and uh Demotivating.

52:07 Um Because I mean I I go through I I mean I I I've put a lot of blood, sweat and tears into building the company's

52:17 And then it And then I'm like, Well like should I be doing this? Because If I'm sacrificing time with friends and family that I would prefer to to to But but then Ultimately the AI can do all these things.

52:30 Does that make sense? I d I don't know. Um To some extent I have to have deliberate suspension or disbelief. In order to be to remain motivated. Um

52:40 So I I guess I would say just You know. Work on things that you find interesting, fulfilling and um And and that contributes uh some good to the rest of society. Yeah, a lot of these threats it's very hard to

52:57 Intellectually you can see the threat. But it's very hard to come to terms with it emotionally. Yeah. I I haven't come to terms with it emotionally yet. What do you mean by that? I haven't come to terms with What the development of superintelligence could do to my children's future.

53:17 I'm okay, I'm seventy seven. I'm gonna be out of here soon. But For my children and my My younger friends

53:26 My nephews and nieces And their children. Um I just don't like to think about what could happen. Why?

53:40 Because it could be awful. In in what way? Well if I ever decide you to take over. I mean it would need people for a while. To run the power stations.

53:54 Until Design better analogue machines to run the power stations. This So many ways it could get rid of people. All of which would of course be very nasty.

54:06 Is that part of the reason you do what you do now? Yeah. I I mean I think we should be making a huge effort right now. To try and figure out if we can develop it safely. Are you concerned about the midterm impact?

54:18 potentially on your nephews and your your kids in terms of Their jobs as well. Yeah, I'm concerned about all that. Are there any particular industries that you think are at most at risk? People talk about the creative industries a lot and it sort of knowledge work. they talk about lawyers and accountants and stuff like that. Yeah. So that's why I mentioned plumbers. I think plumbers are less at risk Okay. Someone like a legal assistant.

54:40 A paralegal. No. Um They're not gonna be needed for very long. And is there a wealth inequality issue here that will will

54:47 Yeah. In a society which shared out things Fairly. If you get a big increase in productivity Everybody should be better off.

54:58 Mm. But if you can replace lots of people My AIs. Then the people who get replaced will be worse off. And

55:09 The company that supplies the AIs will be much better off And the company that uses the AIs. So it's gonna increase the gap between rich and poor. And we know that. If you look at that gap between rich and poor

55:24 That basically tells you how nice this society is. You have a big gap. You get very nasty societies in which people live in wall communities and put Other people in Mass jails.

55:35 It's not good to increase the gap between rich and poor. The international monetary fund has expressed profound concerns that generative AI could cause massive labor disruptions and rising inequality. And has called for policies that prevent this from happening. Read that in the business insider. Have they given any idea of what the policies should look like? No.

55:55 Yeah, that's the problem. I mean if I can make everything much more efficient and get rid of people for most jobs. Or have a person assist you by doing many, many People's work. It's not obvious what to do about it. Universal basic income.

56:10 Give everybody money? I I think that's a good start. And It stops people starving. But for a lot of people, their dignity is tied up with their job. I mean, who you think you are is tied up with you doing this job, right? Yeah.

56:26 And If we said we'll give you the same money just to sit around. That would impact your dignity. You said something earlier about It's surpassing or being superior to human intelligence. A lot of people

56:39 I think like to believe that A I is is on a computer and it's something you can just turn off. If you don't like it. Well let me tell you why I think it's superior. Okay. Um

56:49 It's digital. And because it's digital You can have You can simulate a neural network on one piece of hardware. Yeah. And you can simulate exactly the same neural network on a different piece of hardware.

57:02 Yeah. So you can have clones of the same intelligence. No. You could get this one to go off and look at one bit of the internet. And this other one to look at

57:11 A different bit of the internet. And while they're looking at these different bits of the internet. They can be syncing with each other, so they keep their weights the same. The connection strength's the same. Weights of connection strengths. Mm-hmm. So this one might look at something on the internet and say, Oh, I'd like to increase this strength of this connection a bit.

57:28 And it can Convey that information to this one. So it can increase the strength of that connection a bit based on this one's experience. And when you say the strength of the connection, you're talking about learning. That's learning, yes. Learning consists of saying instead of this one giving two point four votes for whether that one should turn on, we'll have this one give two point five votes for whether this one should turn on. That would be a little bit of learning. Mm-hmm.

57:50 So these two different copies of the same mural net. uh getting different experiences, they're looking at different data. But they're sharing what they've learned by averaging their weights together. Mm-hmm. And they can do that averaging at like a

58:05 You can average a trillion weights. When you and I transfer information We're limited to the amount of information in a sentence. And the amount of information in a sentence is maybe a hundred bits. It's very little information.

58:17 We're lucky if we're transferring like ten bits a second. These things are transferring trillions of bits a second. So they're billions of times better than us at sharing information. And that's because they're digital. And you can have two bits of hardware using the connection strengths in exactly the same way.

58:34 We're analogue and you can't do that. Your brain's different from my brain. And if I could see the connection strengths between all your neurons, it wouldn't do me any good,'cause my neurons work slightly differently and they're connected up slightly differently. Mm-hmm. So when you die All your knowledge dies with you.

58:50 When these things die Suppose you take these two digital intelligences that are clones of each other. And you destroy the hardware they run on. As long as you've stored the connection strength somewhere. You can just build new hardware.

59:04 Executes the same instructions. So it'll know how to use those connection strengths. And you've recreated that intelligence. So they're immortal. We've actually solved the problem of immortality.

59:15 But it's only for digital things. So it knows It will effen essentially know everything that humans know but more. Because it will learn new things. It will learn new things. It would also see all sorts of analogies that people probably never saw.

59:31 So for example At the point when GPT four couldn't look on the web. I asked it. Why is a compost heap like an atom box? Off you go.

59:42 I have no idea. Exactly. Excellent. Most that's exactly what most people would say. It said, Well the timescale's very different. And the energy scales are very different. But then it went on to talk about how a compost heap, as it gets hotter, generates heat faster.

59:58 And an atom bomb, as it produces more neutrons, generates neutrons faster. Mm. And so they're both chain reactions. But at very different time and energy scales. And I believe GPT four had seen that during its training.

1:00:12 It had understood the analogy between a compost heap and an atom bomb. And the reason I believe that is If you've only got a trillion connections, remember you have a hundred trillion. Yeah. And you need to have thousands of times more knowledge than a person. You need to compress information into those connections.

1:00:28 And to compress information, you need to see analogies between different things. In other words, it needs to see all the things that are chain reactions and understand the basic idea of a chain reaction and code that and then code the ways in which they're different. And that's just a more efficient way of coding things than coding each of them separately. No. So it's seen many, many analogies, probably many analogies that people have never seen.

1:00:52 That's why I also think that people who say these things will never be creative. They're gonna be much more creative than us. Because they're gonna see all sorts of analogies we never saw. And a lot of creativity is about seeing strange analogies. People are somewhat romantic about the specialness of what it is to be human. And you hear lots of people saying, it's very, very different. It's a it's computer. We are, you know, we're conscious. We are creatives. We we have these sort of

1:01:16 in a unique abilities that The computers will never have. What'd you say to those people? I'd argue a bit with the innate. So The first thing I say is we have a long history

1:01:30 A believing people were special. And we should have learned by now. We thought we were at the center of the universe. We were thought we were made in the image of God. White people thought they were very special.

1:01:41 Mm. We just tend to want to think we're special. Yeah. My belief is

1:01:47 But more or less everyone. has a completely wrong model of what the mind is. Let's suppose I drink a lot or I drop some acid and not recommend it. And I Say to you

1:02:00 I have the subjective experience of little pink elephants floating in front of me. Mm-hmm. Most people Interpret that as There's some kind of inner theater called the mind.

1:02:14 And only I can see what's in my mind. And in this in the theatre There's little pink elephants floating around. So in other words, what's happened is my perceptual system's gone wrong. And I'm trying to

1:02:27 indicate to you how it's gone wrong and what it's trying to tell me. And the way I do that is by telling you what would have to be out there in the real world Four. It to be telling the truth.

1:02:40 And so these little pink elephants They're not in some inner theatre. These little pink elephants are hypothetical things in the real world. And that's my way of telling you. How my perceptual system's telling me Fibs.

1:02:53 So now let's do that with a chatbot. Yeah. 'Cause I believe that current multimodal chatbots have subjective experiences. Very few people believe that. But I'll try and make you believe it.

1:03:05 So suppose I have a multi modal chatbot. It's got a robot arm so it can point. And it's got a camera so it can see things. And I put an object in front of it. And they say point at the object.

1:03:16 Just like this. No problem. Then I put a prism in front of his lens. And so then I put an object in front of it. And I say point at the object and it goes there.

1:03:26 Okay. And I say no. That's not where the object is. The object's actually straight in front of you, but I put a prism in front of your lens. And the chatbot says, Oh I see, the prism bent the light rays. So Um the object's actually there.

1:03:41 But I had the subjective experience that it was there. Now, if the chapel says that, it's using the word subjective experience exactly the way people use them. It's an alternative view of what's going on. The hypothetical states of the world. Which if they were true would mean my perceptual system wasn't lying.

1:03:59 And that's the best way I can tell you what my perceptual system's doing when it's lying to me. Mm. No. We need to go further to deal with sentence and consciousness and feelings and emotions. But I think in the end they're all going to be dealt with in a similar way. There's no reason machines can't have them all.

1:04:13 But people say machines kinda feelings. And people are spiritually confident about that. I have no idea why. Suppose I make a battle robot. And it's a little battle robot.

1:04:25 And it's he's a big battle robot. That's much more powerful than it. It would be really useful if it got scared. Mm. No.

1:04:35 When I get scared um various physiological things happen that we don't need to go into. And those won't happen with the robot. But all the cognitive things, like I better get the hell out of here. Mm. And I better sort of change my way of thinking so I focus and focus and focus and don't get distracted.

1:04:53 All of that. Will happen with robots, too. People are building things so that they When the circumstances such they should get the hell out of there, they get scared and run away. They'll have emotions then.

1:05:05 They won't have the physiological aspects, but they will have all the cognitive aspects. And I think it would be odd to say they're just simulating emotions. No, they're really having those emotions. The little robot got scared and ran away. It's not running away because of adrenaline, it's running away because of a sequence of sort of neurological in its neural net. Processes happen which maybe.

1:05:28 So do you do you And it's not just adrenaline, right? There's a lot of cognitive stuff goes on when you get scared. Yeah. So do you think that There is conscious AI. And when I say conscious, I mean

1:05:41 That represents the same properties of consciousness that a human has. There's two issues here. There's a sort of empirical one and a philosophical one. I don't think there's anything in principle that stops machines from being conscious. I'll give you a little demonstration of that before we carry on. Suppose I take your brain. And I say one brain cell in your brain.

1:06:01 And I replace it by this is a bit black mirror like I replace it by a little piece of nanotechnology. It's just the same size. that behaves in exactly the same way when it gets pings from other neurons. It sends out pings just as the brain cell would have. So the other neurons don't know anything's changed. Okay.

1:06:20 I've just replaced one of your brain cells with this little piece of known technology. Would you still be conscious? Yeah. Now you can see where this argument's going. Yeah.

1:06:29 If you replaced all of them. As I replace them all, at what point do you stop being conscious? Well, people think of consciousness as this like ethereal thing that exists maybe beyond the brain cells. Yeah, well people f have a lot of crazy ideas. Um

1:06:45 People don't know what consciousness is and they often don't know what they mean by it. Mm. And then they fall back and saying well I know it'cause I've got it and I can see that I've got it. And they fall back on this theatre model of the mind. Which I think is nonsense.

1:06:59 What do you think of consciousness as if you had to try and define it? Is it'cause I think of it as just like the awareness of myself, I don't know. I think it's a term we'll stop using. Suppose you want to understand how a car works. Well, you know some cars have a lot of um. And other cars have a lot less umph.

1:07:15 Like And Aston Martin's got lots of umph. Mm. And a little corolla doesn't have much oomph. But um isn't a very good concept for understanding cars. Um, if you want to understand cars you need to understand about electric engines or petrol engines and how they work.

1:07:31 And it gives rise to um But umph isn't a very useful explanatory concept. It's a kinda essence of a car. It's the essence of an Aston Martin. But it doesn't explain much. I think consciousness is like that. And I think we'll stop using that term. But I didn't think there's anything.

1:07:48 Any reason why a machine shouldn't have it? If Your view of consciousness is that it intrinsically involves self awareness. Then the machine's gotta have self awareness. It's gotta have cognition about its own cognition and stuff. But

1:08:03 I'm a materialist through and through and I don't think there's any reason why A machine shouldn't have consciousness. Do you think they do, then? have the same consciousness that we think of ourselves as being uniquely

1:08:16 Uh Given as a gift when we're born. I'm I'm ambivalent about that at present. So I don't think this is hard line. I think as soon as you have a machine

1:08:28 That has some self awareness. He's got some consciousness. Um I think it's an emergent property of a complex system. It's not a sort of essence that's

1:08:39 Throughout the universe, it's you make this really complicated system. That's complicated enough to have a model of itself. And it does perception. And I think Then you're

1:08:50 We didn't get A conscious machine. So I didn't think there's any sharp distinction between what we've got now and conscious machines. I don't think it's gonna one day we're gonna wake up and say, Hey, if you put this special chemical in, it becomes conscious. It's not gonna be like that. I think we all wonder if these computers are like thinking like we are. on their own when we're not there. And if they're experiencing emotions, if they're contending with I we I think we probably you know we think about things like love and things that uh feel unique to

1:09:18 Biological species. Um are they sat there thinking? Are they uh do they have concerns? I think they really are thinking. And I think as soon as you make AI agents They will have concerns.

1:09:30 If you wanted to make an effective I agree, suppose you let's take a call centre. Mm-hmm. In a call center. You have people at present. They have all sorts of emotions and feelings. Which are kind of useful.

1:09:42 So suppose I call up the call center. And I'm actually lonely. And I don't actually want to know. the answer to why my computer isn't working. I just want somebody to talk to. After a while.

1:09:56 The person in the call center. We'll either get bored or get annoyed with me and we'll terminate it. Well, you replace them by an AI agent. The air agent needs to have the same kind of responses. If someone's just called up because they just want to talk to the air agent and we're happy to talk for hold the whole day to the air agent. That's not good for business.

1:10:16 And you want an AI agent that either gets bored or gets irritated and says, I'm sorry, but I don't have time for this and Mm-hmm. Once it does that. I think it's got emotions. No Like I say, emotions have two aspects to them. There's the cognitive aspect.

1:10:32 And the behavioral aspect. And then there's a physiological aspect. And those go together with us. And If the AI agent gets embarrassed, you won't go red. Yeah. Um start sweating. Yeah. But it might have all the same behavior. And in that case I'd say yeah, it's having emotion. It's got an emotion.

1:10:51 So it's gonna have the same sort of cognitive Thought. And then it's gonna act upon that cognitive thought. But without the physiological responses. And does that matter?

1:11:00 But it's doesn't go red in the face and it's just a different I mean, that's a response to the It makes it somewhat different from us. Yeah. For some things the physiological aspects are very important. My love. They're a long way from having love the same way we do.

1:11:14 But I don't see why they shouldn't have emotions. So I think what's happened is People have a model of how the mind works and what feelings are and what emotions are. And their model's just wrong.

1:11:27 What um Brought you to Google. You you have to Google for about a decade, right? Yeah. What brought you there?

1:11:35 I have a Son who has learning difficulties. And in order to be sure he would never be out on the street. I need to get Several million dollars.

1:11:47 And I wasn't going to get that as an academic. I tried. So I taught a Coursera course in the hope that I'd make lots of money that way, but there was no money in that. Mm-hmm. So I figured out well The only way to Okay.

1:12:00 Millions of dollars. is to Sell myself to A big company. And

1:12:08 So when I was sixty five Fortunately for me. I had two brilliant students. who produced something called Alexnet. Which was

1:12:16 neural net that was very good at recognizing objects and images. And So Elia and Alex and I Set up a little company and auctioned it.

1:12:27 And we actually set up an auction where we had a number of big companies bidding for us. And that company was called Alex Nett. No. The network that recognized objects was called out and test.

1:12:40 company was called DNN Research, Deep Neural Network. Research. And it was doing things like this. I'll put this uh graph up on the screen. That's that exact. This picture shows eight images. And

1:12:52 AlexNet's ability, which is your company's ability to spot what was in those images. Yeah. So it could tell the difference between various kinds of mushroom. And about twelve percent of ImageNet is dogs. And to be good at ImageNet you have to tell the difference between very similar kinds of dog. And it would go to be very good at that.

1:13:12 And your your company AlexNet won several awards, I believe, for its ability to outperf outperform its competitors. And so Google ultimately ended up acquiring Your technology. Google acquired that technology and some other technology. And you went to work at Google at age what sixty six?

1:13:31 I went at age sixty five to work at Google. Sixty five. And you left at age seventy six? Seventy five. Seventy five, I think. I worked there for more or less exactly ten years. And what were you doing there? Okay, they were very nice to me. They said they said pretty much you can do what you like.

1:13:46 I worked on something called distillation that did really work well. And that's now used all the time. In AI. In AI and distillation is a way of taking what a big model knows, a big neural net knows. And getting that knowledge into a small neural net.

1:14:01 Then at the end. I got very interested in analog computation. And whether it would be possible to get these big language models Running in analog hardware. So they use much less energy.

1:14:13 And it was while I was doing that work. that I began to really realise how much better digital is for sharing information. Was there a Eureka moment? There was a Eureka month or two. Um, and it was a sort of coupling of

1:14:29 Chat GPT coming out, although Google had very similar things a year earlier. Hell. I'd seen those and that had a big impact. Effect on me. The closest I had to a Eureka moment

1:14:40 was when A Google system called Palm Was able to say why a joke was funny. And I'd always thought of that as a kind of landmark. If it can say why a joke's funny, it really does understand. And it could say why a joke was funny.

1:14:55 Mm. And that coupled with realizing why digital is so much better than analog for sharing information. Suddenly made me Very interesting I safety. And

1:15:08 But these things were gonna get a lot smarter than us. Why did you leave Google? The main reason I left Google was because I was seventy five. Yeah. And I wanted to retire.

1:15:18 I've done a very bad job of that. The precise timing of when I w left Google. Was so that I could talk freely at a conference at MIT. But I left'cause I was

1:15:29 I'm old and I was finding it harder to program. I was making many more mistakes when I programmed, which is very annoying. You wanted to talk freely at a conference at MIT. Yes. At MIT organized by MIT Tech Review. What did you want to talk about freely? AI safety. And you couldn't do that while you were at Google. Well, I could have done it while I was at Google and Google encouraged me to stay and work on AI safety.

1:15:50 I said I could do whatever I liked on I safety. You kind of sense yourself if you work for a big company You Don't feel right saying things that'll damage the big company. Even if you could get away with it, it just feels wrong to me.

1:16:04 Mm. I didn't leave'cause I was cross with anything Google was doing. I think Google actually behaved very responsibly. When they had these big chat bots, they didn't release them. Possibly because they were worried about their reputation. They had a very good reputation and they didn't want to damage it.

1:16:19 So Open AI didn't have a reputation and so they could afford to Take the gamble. I mean there's also big conversation happening around how it will cannibalize their core business and such.

1:16:30 There is now, yes. Yeah. Yeah. And it's the older innovators' dilemmas to some degree, I guess, that the contending with. Make sure you keep what I'm about to say to yourself. I'm inviting ten thousand of you to come even deeper into the diary of a sea. Welcome to my inner circle. This is a brand new private community that I'm launching to the world. We have so many incredible things that happen that you are never shown. We have the briefs that are on my iPad when I'm recording the conversation. We have clips we've never released. We have behind the scenes conversations with the guests, and also the episodes that we've never ever released. And so much more.

1:17:05 In the circle, you'll have direct access to me. You can tell us what you want this show to be, who you want us to interview, and the types of conversations you would love us to have. But remember for now we're only inviting the first ten thousand people that joined before it closes. So if you want to join our private closed community, head to the link in the description below or go to D Oac Circle dot com I will speak to you then. I'm continually shocked by the types of individuals that listen to this conversation, um, because they come up to me sometimes. So I hear from politicians, I hear from some rural people, I hear from entrepreneurs all over the world, whether they are the entrepreneurs building some of the biggest companies in the world or they're Yeah, early stage startups.

1:17:43 For those people. That are listening to this conversation now. That are in positions of power and influence. World leaders, let's say. What's your message to them?

1:17:54 I'd say what you need is highly regulated capitalism. That's what seems to work best. And what would you say to the average person? N doesn't work in the industry. Somewhat concerned about the future. Doesn't know if they're helpless or not.

1:18:09 What should they be doing in their own lives? My feeling is there's not much they can do. This isn't isn't gonna be decided. Bye. Just as climate change isn't gonna be decided by people separating out the plastic bag from the

1:18:23 Um compostables. That's not gonna have much effect. the lobbyists for the big energy companies can be kept under control. I don't think there's much people. Can do.

1:18:35 Two Except for Try and pressure their governments. Two Force the big companies to work on IO safety.

1:18:45 That they can do. You lived a fascinating Fascinating. winding life. I think one of the things most people don't know about you is that your family has a Big history of being involved in tremendous things.

1:19:01 You have a family tree, which is one of the most impressive that I've Ever seen or read about? Your great great grandfather, George Ball, founded the Boolean algebra logic, which is One of the foundational principles of modern computer science.

1:19:15 You have uh your great great grandmother, Mary Everestball, who was a mathematician and educator who Made huge. Leaps forward in mathematics from what I was able to ascertain. Um I mean I can get the list goes on and on and on. I mean your great great uncle George Everest is what Mount Everest is named after.

1:19:35 Is that is that correct? I think he's my great great great uncle. His His niece Married George Poole. So Mary Mary Bull was Mary Everest Bull.

1:19:49 Um she was the niece of Everest. And your first cousin once removed Joan Hinton. was involved in the nuc a nuclear physicist who worked on the Manhattan Project, which is the World War Two development of the first nuclear bomb. Yeah, she was one of the two Female physicists at Las Alamas.

1:20:06 And then After they dropped the bomb, she moved to China. What? She was very cross with him dropping the bomb. And

1:20:15 Her family had a lot of links with China. Her mother. With friends with Shimma Mau. Oh.

1:20:23 Quite weird. When you look back at your life, Jeffrey We have the hindsight you have now and the retr retrospective clarity. What might you have done differently if you were advising me? I guess I have

1:20:40 Two pieces of advice. One is If you have an intuition that people are doing things wrong and there's a better way to do things. Don't give up on that intuition just'cause people say it's silly.

1:20:53 Don't give up on the intuition until you figured out why it's wrong. Figure out for yourself why that intuition isn't correct. Unusually. It's wrong. If it disagrees with everybody else, and you'll eventually figure out why it's wrong.

1:21:09 But just occasionally you'll have an intuition. That's actually right and everybody else is wrong. Mm. And I lucked out that way. Early on I thought neural nets are definitely the way to go to

1:21:20 Make AI. And almost everybody said that was crazy. And I stuck with it because I couldn't it such seemed to me it was obviously right. Now The idea that you should stick with your intuitions

1:21:34 Isn't gonna work if you have bad intuitions. But if you have bad intuitions, you're never gonna do anything anyway, so you might as well stick with them. And in your own career journey, is there anything you look back on and say, with the hindsight I have now? I should have taken a different approach at that juncture. I wish I spent more time with my wife.

1:21:54 Um And with my children when they were little. I was kind of obsessed with work. Your wife passed away. Yeah. From ovarian cancer?

1:22:11 No, that was another wife. Um I had two wives to have cancer. Oh really? Sorry. The first one died of ovarian cancer, and the second one died of pancreatic cancer. And you wish you'd spent more time with her. With the second wife, yeah. It was a wonderful person.

1:22:27 Why do you say that? In your seventies. What is it that you've you figured out that I might not know yet? Oh, just'cause she's gone and I can't spend more time with her now. But you didn't know that at the time.

1:22:42 At the time you think. I mean It was likely I would die before her just'cause she was a woman and I was a man. Um I didn't

1:22:53 I just didn't spend enough time when I could. I I think I I inquire there because I think there's many of us that are so consumed with what we're doing professionally that we kind of assume or more immortality with our partners because they've always been there. So we But she was very supportive of me spending a lot of time working.

1:23:10 And why'd you say your children as well? What's the what's the spend enough time with them when they were little. And you regret that now? Yeah.

1:23:22 If you um if you had a closing message for So my for my listeners about AI and AI safety. What would that be, Jeffrey?

1:23:32 There's still a chance that we can figure out how to develop AI That won't want to take over from us. And because there's a chance. We should put enormous resources into trying to figure that out,'cause if we don't, it's gonna take over. And are you hopeful?

1:23:49 I just don't know. I'm agnostic. You must get get bet in get in bed at night and when you're thinking to yourself about probabilities There must be a bias in one direction. Cause there certainly is for me. I mean imagine everyone listening now has a

1:24:06 Internal prediction. They might not say out loud, but uh how they think it's gonna play out. I really don't know. I genuinely don't know. I think it's incredibly uncertain. When I'm feeling slightly depressed, I think.

1:24:20 People are toast. AI is gonna take over. Well I'm feeling cheerful, I think. We'll figure out a way. Maybe one of the facets of being a human. Um is

1:24:29 Because we've always been here. Like we were saying about our loved ones and our relationships. We assume Casually, that we will always be here. And we'll always figure everything out. But there's a beginning and an end to everything.

1:24:40 As we saw from the dinosaurs. I mean. Yeah. And We have to face the possibility. That unless we do something

1:24:50 Soon. When near the end. We have a closing tradition on this podcast where the last guest leaves a question in their diary. And the question that they've left for you. Is

1:25:07 With everything that you see ahead of us. What is the biggest threat you see to human happiness? I think the joblessness is a fairly urgent short term threat to human happiness. I think if you make lots and lots of people unemployed. Even if they get universal basic income.

1:25:29 Um They're not gonna be happy. Because they need Purpose. And structural. They need to feel they're contributing something. They're useful. And do you think that outcome?

1:25:41 But there's gonna be huge job displacement. Is more probable than not. Yes. Thank you. That one I think is Definitely more probable than not.

1:25:50 If I worked in a call center, I'd be terrified. And what's the time frame for that? In terms of mass traffic, I think it's beginning to happen already. I wrote an article in the Atlantic recently. that said it's already getting hard for university graduates to get jobs.

1:26:06 And Part of that may be that people are already using AI for the jobs they would have Good. I spoke to the CEO of a a major company that everyone will know of. Lots of people use and he said to me in DMs.

1:26:19 That they used to have seven just over seven thousand employees. He said uh by last year they were down to I think five thousand. He said right now they have three thousand six hundred. And he said by the end of summer, because of AI agents, they'll be down to three thousand. So it's happening already. Yes. he's halved his workforce because AI agents can now handle eighty percent of the customer service inquiries.

1:26:39 And other things. So it's It's happening already. Yeah. So urgent action is needed. Yep. I don't know what that agent action is.

1:26:48 That's a tricky one because that depends very much on the political system. And political systems are all going in the wrong direction at present. I mean what do we need to do? Save up money? Like do we save money, do we move to another part of the world? I don't know. What would you tell your kids to do?

1:27:05 They said that like there's gonna be loads of just job displacement. 'Cause I work for Google for ten years, they have enough money. Okay, okay, fine. So they're not typical. What if they didn't have money? Trained to be a plumber. Really? Yeah.

1:27:21 Ha Jeffrey, thank you so much. You're the first Nobel Prize winner. That I've ever Had a conversation with, I think. In my life.

1:27:31 So that's a a tremendous honor. And you you you receive that award for a lifetime of exceptional work in pushing the world forward in so many profound ways that will lead to great And that have led to great advancements and things that matter so much to us. And now you've turned this season in your life to Shining a light on some of your own work, but also on the the the broader risks of AI and how um And how it might impact us adversely. And there's very few people. that have worked inside the the machine of a Google or a big tech company that have contributed to the field of AI that are now

1:28:03 at the very forefront of warning us against the very thing that they worked upon. The truck. Actually surprising number of us now. They're not as uh as public and they're actually quite hard to get to have these kinds of conversations because many of them are still in that industry.

1:28:19 So you know, someone who tries to contact these people often and ask invites them to have conversations, they often are a little bit hesitant. To speak openly, so they speak privately. But they're less willing to openly because maybe Maybe they still have something in It's some sort of incentives at play.

1:28:33 I have an advantage over them, which is um I'm older, so I'm unemployed, so I can say what I have. There you go. So thank you for doing what you do. It's a real honour and please do continue to do it. Thank you, thank you so much. People think I'm joking when I say that, but I'm not. The chummy fish. Yeah.

1:28:52 Yeah. Plumbers are pretty well paid. Uh

What you see above is a preview of the first minutes. One unlock costs 10 credits and covers this episode forever: full segment and word-level timestamps on this page, plus .txt, .srt, .vtt and word-level JSON downloads, as many times as you like.