2:30 am

it’s potentially creating new entities on this planet that could be smarter than us in many ways, and that we, for now, don’t know how to control. we are now subject, especially in america, to a technocratic oligarchy whose primary impulse is to destroy institutions. they don’t believe in them. even within an earldom, there are grades of snobbery, and within nobels, we now discover, it’s how many people you share it with.

2:31 am

they laugh it sounds very oscar wilde. it does, does it not? hello, welcome to ai decoded. we are recording on guy fawkes day, which seems oddly relevant, given our guests this week. describing ai as the revolution of our age, we have yoshua bengio, one of the godfathers of artificial intelligence, who is worried the race to smarter machines could be an explosive mix we cannot control. and fresh from a round table plot of his own, the broadcaster, comedian and writer sir stephen fry, who’s been starring in celebrity traitors while performing in the west end in oscar wilde’s the importance of being earnest. how does he find the time? welcome to you both. it’s an immense pleasure to have you on the show together. immense pleasure to be here. and dr stephanie hare is also here to lend her considerable expertise. and, stephanie, we are truly in the presence

2:32 am

of greatness this week, because yoshua is here in london to receive from the king the queen elizabeth’s prize for engineering, alongside six other pioneers of artificial intelligence. congratulations, yoshua. you must be very proud. thank you. erm, i’m glad that this work is recognised, of course. and you could, of course, swap notes with stephen, yoshua, because stephen was just only knighted by the king earlier this year. i’m sure the protocol is still fresh in his mind. do you know each other? i know of yoshua, of course. anybody who’s been interested for any length of time in ai knows of yoshua and knows and admires the way he has recently spoken out about the dangers of ai. it’s not an easy thing to do. there’s enormous pressure to... ..to conform with the excitement. we can talk about your... was he a partner of yours in some of the development, yann lecun? who, of course,

2:33 am

has expressed far less concern about the dangers of it, but it’s a bit like talking to a scientist, say professor bengio, and a scientist who works for british imperial tobacco about the dangers of cancer. i read a book, yoshua, many years ago about john harrison. he was a clockmaker who, in the early 18th century, discovered longitude, and he also received a prize from the king. in fact, i think it was george iii who lobbied on his behalf. and, of course, that discovery would speed up travel and the distribution of goods, whereas your discovery has sort of enhanced the collection, the transfer, the processing of data, which is another major chapter in human history which your name is attached to. that must make you sit up sometimes. do you think about it? well, i see that... i think it’s even bigger than what you’re saying, potentially, if advances in the capabilities of ai

2:34 am

continue along the current trends. it’s not just faster x, it’s... you know, it’s not just new tools. it’s potentially creating new entities on this planet that could be smarter than us in many ways, and that we, for now, don’t know how to control, and, for now, are not taking seriously. and that’s just one of many risks that i became gradually aware of that we’re just blind to as a society right now. you featured on the programme quite recently as part of the op-ed that we ran from the new york times. it was titled, rather ominously, the ai prompt that could end the world. and you’re quoted in it, as one of the great minds that they spoke to, as saying that you stay awake at night, you’re kept awake at night

2:35 am

by those thoughts that you’ve just touched on. what is it specifically you worry about? well, if we build entities that have their own goals... by the way, that already exists. and if those goals sometimes go against us, go against our norms... and that’s already the case. over the last year, we’ve seen many experiments where the ais preferred to preserve themselves and will cross our red lines - blackmail, cheat, lie, and even go up to decide to kill somebody. now, right now, they don’t have the capability to actually carry out something like this. but if we continue on the trajectory to make them smarter and we don’t find technical solutions to make sure they are aligned with us, then that’s a huge problem.

2:36 am

but it’s not just these ais potentially doing bad things on their own. it’s also humans using those ais to grab power, to, you know, become king of the world. and this is also a real possibility because intelligence gives power, and we need to make sure that we’re going to manage that power. and that is a political problem, not just a technical problem. and, if i may say so, a political problem that comes at the worst possible time in recent human history. when atomic power was discovered to be what it was after los alamos and so forth, the atomic energy commission was headed by von neumann, a man einstein called the greatest scientist of the 20th century. and it was a time when we were building out institutions. all within the same few years, there was the united nations, nato, the treaty of rome started europe,

2:37 am

the who, bretton woods started the imf. and we believed in institutions to control and regulate the world, because the war had shown us how necessary that was. we are now subject, especially in america, to a technocratic oligarchy whose primary impulse is to destroy institutions. they don’t believe in them. they certainly don’t believe in the ones i’ve mentioned. they have actually enacted and spoken against them. and any guardrails... you know, because with nuclear power, there was salt. i mean, very quickly there was a red telephone in the office of the president of the united states and the general secretary of the communist party in moscow and there was a sort of sense of responsibility. but we have the three cs that have always been a threat to any kind of large and powerful technology. firstly, the countries that need, you know, want national supremacy over weaponry and information and all the other things,

2:38 am

and capitalism itself, because of the sheer power and the influence of vast investment that is going on into the trillions for these new large language models, and what will spin off from them. and that’s unstoppable now. no-one can stop it. and thirdly, and perhaps even more importantly, criminals, people of malicious intent, who want to use this astonishing power for their own benefit. and all of these three have to be considered and brought together, as yoshua says, under some political framework, at a time when we cannot seem to make any political frameworks internationally or any agreements. yes. and, you know, the prompt that could change the world reminds me of an old ethics course they used to do for philosophy and one of the sort of equivalents of what wasn’t called the alignment problem then was the genie problem. it’s quite simplistic, but it greatly expresses it. you say to a genie, because you’re a good person,

2:39 am

who’s offered you your one wish, “i want to end all suffering in the world.” and instantly, all life disappears because the genie looks at the world and sees that where there is suffering, there is life, and to get rid of suffering, you get rid of life. it’s a perfectly logical step to a genie. he’s answered the problem. it’s the quickest and most perfect answer to the end of suffering. but it’s not quite what we meant. they chuckle and, you know, as many people have pointed out, by definition, artificial intelligence that succeeds at something cannot be understood. if it could, it wouldn’t be better than us. you know, chess is the obvious example as a closed system. all computers can play chess better than any human these days, or at least, you know, all the basically good chess engines since big blue in ’98. and that means you can’t predict what their move is. if you could predict what their move was, it wouldn’t be better than you. do you see what i mean? and expand that out to the nature of knowledge and the attributes that the intelligences can have.

2:40 am

and then bear in mind, just quickly, to finish my point, that the people who are in charge are the same people who are in charge of social media, who saw what damage it was doing, from cambridge analytica, what it was doing in, you know, in terms of political freedom and so on, and also what it was doing in places like myanmar and what it was doing, if you like, in the jonathan haidt sense or jonathan hait, i don’t know how he pronounces his surname. you know, to the minds of the young, you know, essentially how it was polluting the streams of discourse so that they became so toxic that our children are breathing in air that is dangerous, frankly, in social media. and the same people who are in charge of that and know how bad it is, are in charge of what ai is doing around the world. and that is a recipe for surely utter disaster. when the time comes, when ai is going to probably be the foremost engine of productivity in our economies, and whoever owns these frontier models

2:41 am

is going to have huge economic power that can be easily transformed into political power. so, let’s say that i’m a country and i own the only strong, very strong models that drive the european economies and i decide to threaten to just, you know, cut the power to this, the whole economies of many countries could go down. and do you feel we need the equivalent of a red telephone? so stephen was mentioning in the cold war... i believe it still exists today. one certainly hopes. ..that there was a red telephone between the then soviet union and the united states to make sure that there was never a nuclear war created by accident. do we need some sort of mechanism like that for ai today? we need to set it up. so... ..if the leadership in china and the us were rational

2:42 am

and they understood how upcoming generations of ai systems could be used against them, either because of, you know, criminals, terrorists or other parties, and if they understood the danger of rogue ais that by themselves could really harm a lot of people, including, you know, not making a difference whether you’re chinese or american, then they would see that they have to deal not just with, “ok, my adversary could use “ai against me,” “but there are all these other catastrophic risks, “and i can’t just focus on one risk. “i have to find a path that covers all of them.” and there are paths. scientists and engineers are starting to work out how, if there were an international treaty controlling these very advanced ais, you could have trust and verify, using technology

2:43 am

for verification, in the same spirit that we’ve done for nuclear, for example. so it would be possible if those superpowers, hopefully more than just the us and china, were to understand those risks, for them to come to an agreement where everyone wins versus everyone loses, i think that would be the rational thing to do. so, i think it’s just that, right now, there isn’t enough awareness, understanding of these risks. and i guess that brings me to my question for both of you, is, what would safe, good ai look like? you’ve mentioned an international treaty. we’ve talked about a red telephone. if you could speak to world leaders today and say, “this is what we need to have in place,” what do you want to see? so i’m actually more optimistic about our ability to control superintelligence than i was, say, a couple of years ago.

2:44 am

so, a couple of years ago, i didn’t sleep well, and i thought, “i need to do something about it.” can we fix ai? can we make it not harm people? and so i’ve been working on this, and last year i realised there should be a way forward that i’m going to say a few words about, and that prompted me to actually create a new nonprofit organisation called lawzero, which is a spin-off of the ai academic institute called mila that i created many years ago. and the idea is that we can build machines which are not like us, which don’t have a self, which don’t have a goal, but which understand the world and can make very good predictions in it. and so if you think a little bit about it, the laws of physics don’t care about you,

2:45 am

they don’t care about me, they don’t care about being elected or having more compute power. and we can build ai in the future, i’m convinced, and i’ve started to assemble a team of researchers and engineers to do that, that will be just like this, which will not be a new species, which will be like a really smart encyclopaedia, if you want, that can help us in scientific discovery, that can help us solve even... you know, design better tools for our democratic institutions, which will be sorely needed. and that even can help with the untrusted agents that companies will want to put on the market. i mean, they already are... how would it do that? i mean... it’s very simple. so if you have a really good predictor, but, you know, it’s something that doesn’t want anything, just making really good predictions, you can ask it, “is this action that an untrusted ai

2:46 am

wants to do, is this action dangerous in some ways?“ and, you know, which ways, and then we can use a socially accepted threshold. if the probability of something bad happening is above a threshold, then we just reject that action. so we can put a guardrail around ais that we don’t fully understand or we don’t fully trust, that do things in the world currently, already, without human oversight, and it’s going to get worse. and that would mitigate a lot of the risks, including the risks of people using ai against others, so long as the code that does that is not accessible to them. can i set that in a story i wanted to talk to you both about? it was in the guardian this week. it’s about truth, it’s about knowledge, and it’s about the role of ai in education. so they report, stephen, that elon musk has launched this week a rival to wikipedia. grokipedia. yes, i’m not entirely sold on the name. i don’t know if you are. but, i mean, essentially, it’s the...it’s the informa...

2:47 am

it’s the misinformation that it is seeding that has academics alarmed. so, some say it’s pushing far-right ideology. it’s giving the contributions from ai chatrooms the same sort of status or prominence as more rigorous academic research. so how much does that concern you? maybe we’ll get a view from yoshua about how we would stop that. it’s obviously deeply worrying, and i think yoshua’s solution is the...is, again, an answer to this. if you can establish a trusted kind of mother source of information that is... ..through which you’re able to pass your, you know, projected ideas, and “i’d like to do this with this ai” and the mother ai would say, “this is not safe. this is... “this is... i predict this will not be good.” and if it can be trusted and it can be... you know, the chinese, for all that we have a lot of reason to distrust their spies and their aims and, you know, their communist state and so on seem to be more on board the idea of safety

2:48 am

and controls than we are in the west at the moment. so if it can be agreed that such a device as yoshua can create, i think that would be the answer to grokipedia. would your terminator-style entity, yoshua, would it go in there and put paid to grokipedia? so, as stephen is saying, if we can build a very intelligent but completely unbiased... ..predictor, then we can essentially use it as a fact-checker. but it’s not just facts. it’s also even predictions with uncertainty being also included. because one of the things that... ..is lacking in many debates is humility. it should be at the centre of what scientists do, right, that you don’t claim things

2:49 am

when you’re not sure they’re true. and if you do, and you’re not sure, then you should say that you’re not sure. and... but how do you quantify that without human bias being built in? well, that’s the core question. and the idea is to transform the data in such a way that the ai understands the difference between opinion and fact. so, right now, the way ais are trained is to imitate what people write, among other things that the ai are trying to do. and that means they will behave in humanly bad ways. they will lie, they will not want to die, because this is what they’ve trained to imitate. but if we teach them to make the difference between what is a verified fact and an opinion, and what they then try to do is to understand why people are saying those things

2:50 am

rather than repeating those things or repeating, you know, in the same way, things that may be not true, but somehow push your agenda, then we would be in a much better place. that’s a really fascinating point, isn’t it? and it goes to the heart of the epistemic crisis, you might say, of the 20th and 21st century, in which... cos newspapers, which we now almost look at fondly as kind of reliable and, you know, diligent and honourable, but they, you know, for a very long time, have probably had 80% opinion against 20% fact in their coverage when you look at a newspaper, and we’ve sort of accepted that. and then it’s become an even wider distinction as we’ve developed through the digital age. and some way of restoring that essential difference. it’s kind of simple but, of course,

2:51 am

an immensely complex task. but it’s a simple outline is to find out and define a fact as opposed to an opinion. and that humility should also guide our societal decisions. so, to go back to superintelligence, in other words, machines that are smarter than humans across the board, this comes with a lot of risks of all kinds, right? and before we jump into this, which is, by the way, the stated goals of many of the ai companies, we need to understand those risks and we need to make sure we don’t allow people to build machines like this unless we have very strong assurances and with the scientific community agreeing that there’s no significant risk to our democracies and the future of humanity and so on. and we also need social acceptance, because it’s not just a technological question. it’s also how is ai going to change the world?

2:52 am

for example, in the last few months, we’ve seen a huge number of reports that have been... ..we’re restarting to quantify, of people becoming emotionally attached to ai systems and sometimes, you know, going into a bit of psychosis, and choosing to harm themselves or kill themselves. this was not on the radar screen a year ago. people did not anticipate that. but it is happening. people are becoming friends or even lovers with their ais. and it could be fine in some...most cases, but openai reported that 0.1% of the discussions had to do with suicide. so out of almost a billion people using those things, that’s a lot. do you have a view on that, sir stephen? because you were knighted for services to mental health awareness in your role as president of the charity mind.

2:53 am

and we have talked on the programme about chat bots and them being used for companionship and emotional support. yeah. are you concerned by that? at the moment, the propensity of these models is to follow up any response with a question - “would you like me to show you...?” and keeps opening up vistas of new ideas. in cases of vulnerable patients, these can trigger all kinds of responses because they... you know, no sensible psychiatrist or psychotherapist would say, “would you like me to tell you about other people “who’ve had a similar problem? would you like me to tell you “about people who have committed suicide? “would you like me...?” you know, i mean, all these things that it’s doing, which is built into it at the moment, as i say. but that’s something that really should be thought about is some sort of mechanism whereby if it gets into that territory, it’s given a far greater ethical framework from which to work. there’s also the fact that the information that you’re giving to these chatbots is all being recorded and then used as further training material, and it could very easily be abused by these companies. yes.

2:54 am

when you go and talk to a therapist, there’s all sorts of health regulations around that and you, as a patient, have rights, right? and accountability, if there was to be an abuse in that situation, medically, and for health reasons. there’s no such protections using chatbots as therapists. so you’re not telling it to a therapist. you’re telling your private thoughts to sam altman or elon musk. stephen laughs and just a final thought from you, sir stephen, since you are playing an oscar wilde part in the west end, what do you think the great mind oscar wilde would make of all this? was he a traditionalist or was he a futurist? he was a futurist. he was a futurist about art. i mean, he condemned victorian society for being shocked by anything new and said, “imagine if science was treated the way that art is treated “and that someone came up with a new theory about the stars “or a new theory about animals, “and people said, ’no, it’s horrible, “’this new, modern theory.

2:55 am

“i like the comfortable, old one.’ “ to some extent, that did happen in victorian england, of course, when darwin spoke up and the geologists suggested how old the earth may be. but people accept science advances and advances and art advances and advances. and so i think he would have embraced cinema, and i think he would have embraced, you know, mass communication. and i think he would be fascinated by what’s here, because his, you know, individual human wit and personality still shines through. and as long as we have the kind of brains that respond to the life of the eyes of the interlocutor with whom we’re engaging and the various sort of extra qualities that, at the moment, only humans have, which are to do with desire and love and amusement and engagement and that sense of another person, then those things are privileged and prioritised in our minds and will always be what we look out for. we are out of time. it’s been an enormous pleasure. sir stephen, yoshua, thank you so much for the time

2:56 am

you’ve given us this week. it’s really very much appreciated. and a reminder to all our viewers that if you missed the show this week, you can catch up on youtube. the back episodes will be there as well, of course. and i have a feeling that this one might do particularly well. stephanie, i will see you next week. thank you so much for watching. goodbye. goodbye. goodbye.

2:57 am

2:58 am

2:59 am

3:00 am

a dangerous super typhoon heads for the philippines, as the country recovers from another major storm which killed almost 200 people. russia says it intentionally targeted energy facilities in ukraine, as engineers there work to get the lights back on after a barrage of deadly strikes. the us senate adjourns for the day, as it works to end the longest government shutdown in history with no end in sight. hello. i’m carl nasman. the philippines is bracing for a super typhoon

3:01 am

• less than a week after typhoon kalmaegi

0 Views

info Stream Only

Uploaded by TV Archive on November 9, 2025