迈向未来

欢迎收听DeepMind播客，我们将带您探索人工智能的世界。我们将评估已知与未知的领域，审视我们试图了解的内容，计算未来将掌握的知识，规划前进方向，并确定何时抵达目标。我是汉娜·弗莱，数学系副教授。

Welcome to DeepMind, the podcast, where we're exploring the world of artificial intelligence. We're assessing what we know and what we don't know. We're looking at what we're trying to know. We're calculating what we will know, mapping out where we're going, and working out how we will know when we get there. I'm Hannah Fry, and I'm an associate professor in mathematics.

过去十二个月我一直在DeepMind工作。在本期节目中，我们将开始展望未来，探讨被称为人工通用智能的自学型AI。但首先，我觉得还没带你们好好参观过这个地方——位于伦敦国王十字区的DeepMind总部。

And I've spent the last twelve months at DeepMind. And in this episode, we are going to start looking forwards to the future, the self schooled variety of AI known as artificial general intelligence. But first, you know what? I don't feel like we've really given you the grand tour this place. DeepMind headquarters in Kings Cross in London.

跟着我吱吱作响的运动鞋走吧。注意关门，我们要穿过去了。

So follow me in my squeakiest of sneakers. Closed doors, coming through.

语言组专注于机器学习和语言理解。

So the language group, they do machine learning, understanding language.

与我同行的是极具亲和力的科莱·科瓦克格鲁，他是2012年就加入的DeepMind元老级成员。

With me is the extremely likable Koray Kovachoglu, one of the very first deep minders joining way back in 2012.

这个区域聚集了专注智能体研究的人员。更好的记忆能力，更强的规划能力——我们如何将这些能力赋予智能体？

This area is actually it's a group of people who are really focused on agents. Better memory, more planning. How can we get in get those into the agents?

所有会议室都以著名数学家命名：爱因斯坦室、高斯室、海蒂·拉玛室。

All of your rooms are named after famous mathematicians. Einstein room, Gauss room, Hedi Lemar.

哦，其实我认识她。我认识她。她实际上是个非常著名的演员。

Oh, actually, I know her. I know her. She's actually a very famous actress.

你知道吗，我最喜欢她在《与斯宾塞·屈塞共舞》中的表现。

You know, I liked her best in I Take This Woman with Spencer Tracy.

但她实际上还是个科学家，对吧？深度学习小组更专注于为任何数据领域设计算法和架构。

But she was actually also a scientist. Right? The deep learning group is more interested in just, like, coming up with algorithms and architectures on any data domain.

那边白板上有些非常美味的方程式。

Some seriously delicious equations on the whiteboard over there.

是的，到处都是。我是说黑板和白板。

Yes, everywhere. I mean blackboards and whiteboards.

如果你在走廊里有了想法，那很重要。

If you have an idea in a corridor, it's important.

静态数据转...所以这个区域是机器学习小组。他们正在进行的主要项目之一是模仿学习。

Static data to So this area is the machine learning group. One of the main projects going on here with this group is the imitation.

哦，等一下。这是我们认识的人。给你。办公室里只有丹尼斯和索菲。

Oh, hang on. Here's someone we know. There you go. Just Dennis and Sophie's in the office.

这是神经科学组的区域。他们经常思考哪些是重要问题，我们需要应对哪些挑战。使劲转它。

This is the area for the neuroscience group. They do a lot of thinking about what are the important problems, what are the challenges that we need to do. Spin it really hard.

哦，桌上足球。是啊，桌上足球从来不是我的强项。手腕要灵活。这是强化学习团队。

Oh, table football. Yeah, table football's never been my forte. Wig wrists. The reinforcement learning team.

这个领域一直是我们实现AGI目标的核心。如果你对智能体感兴趣，它必须是主动的。

This is one area that has been core to what we try to achieve with AGI. If you are interested in agents, it has to be active.

他们有些相当厚重的教科书。这可不适合睡前阅读，对吧？《分子电子结构理论》。这些单词我都认识，但连起来就不懂了。

They've got some proper weighty textbooks. That's not exactly bedtime reading, is it? Molecular electronic structure theory. I know what all of those words mean, but just not in that particular order.

睡眠舱。

Sleeping pods.

睡眠舱。我之前不知道有这个。

Sleeping pods. I didn't know about these.

能量舱怎么样？抱歉。

What about energy pods? Sorry.

不知道这些。我来这里一年了，没人告诉过我这些。里面有人吗？

Don't know about these. I've been coming here for a year. No one told me about these. There anyone in there?

对。哦，抱歉。抱歉。调低。

Right. Oh, sorry. Sorry. Turn down.

听着，我要辩解一下，她门上没有请勿打扰的牌子。好了，这个话题到此为止。回到简单的事情上，比如解开智力之谜。

Look. In my defense, she didn't have a do not disturb sign on the door. Okay. Enough of that already. Back to the simple stuff, like solving the enigma of intelligence.

在我们深入之前还有个小问题。首先，我们需要对智力进行准确定义，因为你看，智力是个相当难以捉摸的概念。在人类智力方面我们已经有了一些基础。虽然IQ可能不是每个人都喜欢的衡量标准，但它确实是我们最稳定的心理测试之一，能很好地衡量推理和逻辑等有限的智力标志。但IQ仍然无法帮助我们定义智力。

Just one tiny snag before we get there. First, we're gonna need a proper definition of what intelligence is because intelligence, you see, is quite a slippery beast to pin down. We've got a bit of a head start when it comes to human intelligence. Although it might not be everyone's favorite metric, IQ is one of the most stable psychological tests we have, and it does a pretty good job at measuring limited intelligence markers like reasoning and logic. But IQ still doesn't get us any closer to a definition of intelligence.

如果想在这方面取得进展，我们需要正确定义它。我们需要某种方法来捕捉智力的含义，这种方法对人类、狗、兔子乃至机器都同样适用。多年来关于智力的定义有过一些建议。1921年心理学家V·亨曼说智力是获取知识的能力和已掌握的知识，表面听起来不错，直到你意识到这也适用于图书馆。

If we're gonna get anywhere with this, we need to define it properly. We need some way to capture what we mean by intelligence that works just as well for humans and dogs as rabbits and machines. And there have been a few suggestions for what intelligence is over the years. In 1921, the psychologist, V. Henman, said intelligence was the capacity for knowledge and knowledge possessed, which sounds quite good on the surface until you realize that it also applies to libraries.

图书馆可以拥有知识。图书馆算有智力吗？可能不算。1985年认知科学家马文·明斯基认为智力是解决难题的能力，这个定义似乎更贴切些。它也涵盖了AI已经证明能做到的事情。

Libraries can possess knowledge. Do libraries count as intelligent? Probably not. In 1985, Marvin Minsky, the cognitive scientist, said that he thought intelligence was the ability to solve hard problems, and that seems a bit more like it. It also captures what AI has already proved it can do.

这位是DeepMind的高级研究员拉雅·哈扎尔。

Here's Raya Hadzal, senior research scientist at DeepMind.

过去几年里，程序能在大量特定细分领域达到人类水平——它们能像人类一样解析语音，能在英法互译中几乎达到人类水准，能近乎人类般识别物体和图像。这些都属于特定领域的专项能力。

In the last few years, there's a huge number of different narrow specific things that programs can do as well as a human. They can interpret your voice as well as a human. They can maybe translate from English to French and back again almost as well as a human. They can recognize things and images almost as well as a human. These sort of narrow specific things.

用'专项'这个词并非要贬低这类技术的变革性力量。仅在本系列中，我们就探讨过节能环保、医疗诊断、蛋白质折叠等领域，这些都充分展现了机器解决复杂问题的能力，也都属于狭义AI范畴。但真正的智能——通用智能——需要更本质的东西。有科学家将智能定义为学习能力或从经验中获益的能力，也有人认为关键在于适应所处环境并茁壮成长。

That word narrow is not to downplay the transformative power of this sort of thing. Just in this series, we've looked at energy conservation, medical diagnosis, protein folding, all of which certainly show the machine's ability to solve hard problems, and all of which are examples of narrow AI. But real intelligence, general intelligence, that needs something else, something more. Some scientists have described intelligence as the capacity to learn or to profit by experience. Others think it's about adapting and thriving in the environment you find yourself in.

无论问谁，科学家们普遍认同：智能本质上关乎与外部环境交互的能力。适应能力必然是其中一环——这意味着不能完全依赖熟悉的环境。真正的智能必须能应对突如其来的挑战。2007年，DeepMind三位联合创始人之一肖恩·莱格在梳理数百种对立观点后，与合作者发表了具有影响力的论文，试图精确定义智能概念。以下是他们得出的定义。

But whoever you ask, most scientists agree that somewhere along the line, intelligence is something about your ability to interact with an external environment. And being able to adapt has to be part of it too, so you can't be fully familiar with the environment. You've gotta be able to deal with unanticipated challenges that get thrown at you if you're intelligent. In 2007, after going through hundreds of competing arguments, Shane Legg, one of the three cofounders of DeepMind, wrote an influential paper in which he and his coauthor tried to pin down precisely what was meant by intelligence. And here is the definition that they came up with.

智能衡量的是智能体在多样化环境中实现目标的能力。而这正是这栋建筑里的人们追求的目标。让我们回顾第四集中高级研究员默里·沙纳汉的阐述。

Intelligence measures an agent's ability to achieve goals in a wide range of environments. And that is what they are aiming for in this building. Here's a reminder of what senior research scientist Murray Shanahan told us in episode four.

AI研究的终极目标是构建人工通用智能——能像人类一样胜任海量多样化任务的AI。我们人类并非专才，一个年轻成年人可以学习并完成无数事情，能适应各种挑战：学会烹饪、

The holy grail of AI research is to build artificial general intelligence. So to build AI that is as good at doing an enormous variety of tasks as we humans are. So we are not specialists in that kind of way. You know, a young adult human can learn to do a huge number of things and can indeed do an enormous number of things and can adapt to a huge number of different challenges. You can learn to make food.

创办公司、建造修理物品、进行对话、养育子女等等。我们真正希望构建的AI，正是具备这种广度的智能。这至今仍是未被攻克的挑战。

You can learn to make a company. You can learn to build things, to fix things. You can do so many things to have conversations, to rear children, so all all of those things. And we really want to be able to build AI that has the same level of generality as that. And that's really still an open challenge.

我们确实不太清楚该如何实现这个目标。

We don't really know quite how to get there.

但如果有人知道需要什么条件，那一定是DeepMind的联合创始人兼CEO戴密斯·哈萨比斯。我们将在下期节目中与他探讨通用人工智能（AGI），现在先来一窥他的想法。

But if anyone has an idea of what it will take, it's Demis Esarbis, the CEO and co founder of DeepMind. We'll be talking to him in the next episode about AGI, but for now, here is a little glimpse into his thinking.

我期待见证许多关键性时刻。比如，当AI系统做出诺贝尔奖级别的新科学发现时，那将是一个重大分水岭——某种意义上展现出真正的创造力。另一个重要节点是它能以自然流畅的语言与我们对话。

I'm waiting to see a lot of key moments. For example, I think a really big moment will be when an AI system comes up with a new scientific discovery that's of Nobel Prize winning level. That to me would be a big watershed moment. So, you know, capable of some kind of true creativity in some sense. I think other big points will be when it can use language and converse with us in a naturalistic way.

它具备学习抽象概念的能力。这些都是我们目前远未达到的高阶认知能力，我认为这些都将成为发展路上的重要里程碑。

It's capable of learning abstract concepts. These are all things that I think are high level cognitive abilities that we're nowhere near yet, and I think, will be big signposts on the way.

现在你可能会问：要如何着手如此庞大的任务？从何处开始？你是否完全相信通用人工智能是可能实现的？

Now it's reasonable to be asking yourself, how do you even approach such a colossal task? Where do you even start? Do you totally and completely believe that AGI is possible?

是的。

Yes.

这个回答真简短啊。不，我...我其实知道...

That's a short answer, wasn't it? No. I I knew I I knew

她开始骗我了。回来

She's starting to trick me. Back

我的导游是库拉伊·科瓦乔。他是DeepMind的研究总监。

to my tour guide, Kurai Kovachowo. He's the director of research at DeepMind.

我相信总有一天我们会达到那个阶段。但现在还没有。目前我们只能退一步，对重要问题提出假设，包括我们需要开发的关键算法、重要解决方案等那些核心要素，然后开始逐步构建更多内容。

I believe a day will come where we will be at that stage. Right now we are not. Right now all we can do is go back from that and then have a hypothesis about the important problems, the important algorithms that we need to develop, the important solutions, the like, those those key things we need to have, and then start building more and more and more.

那我们举个例子。假设DeepMind有人第一天决定研究Right问题。就是构建一个可以放入环境中的智能体。它能判断自己的位置。你们是不是会在白板上进行头脑风暴，列出所有可能影响构建这个智能体的不同方面？

So let's take an example then. Let's say the first day that someone at DeepMind decided they wanted to look at the problem of Right. So so building an agent that you can drop into an environment Yes. And it can work out where it's going. Do you just have like a big brainstorm on a whiteboard of all of the different possible aspects that might contribute to being able to build that agent?

是的，通常从这开始。因为如果有人想研究这类问题，这里可能会有相当多人对此感兴趣。我们会开始讨论：目标是什么？比如你举的导航例子就特别好。

Yeah. It starts with that. Because if someone wants to work on something like that, then probably there will be a good number of people here who would be interested in the same thing. We will start discussing, okay, what is the goal? Like, when we say navigation, you gave a particularly good example.

对吧？比如从这里到指定位置。我们如何定义它？这是什么类型的环境？智能体拥有怎样的控制空间？

Right? Like, going from here to a given location. How are we gonna specify it? What kind of environment this is? What kind of control space does the agent have?

所有这些都会影响我们该使用哪种算法。我们是纯粹基于视觉来做吗？是在简单网格环境中进行？还是从网格世界开始，再考虑如何过渡到三维环境？我们是否考虑实际将其部署到机器人上，比如使用真实视觉？

All these start affecting what kind of algorithms we should use. And are we going to do this purely from vision? Are we going to do this in a simple environment, in a grid world? Or we're going to start from a grid world, and then we need to think about the path towards going to a three d environment? Are we thinking about actually also putting this on a robot, like with real vision?

比如，所有这些讨论就开始出现了。

Like, all that discussion starts happening.

然后规模就变得非常庞大。我是说，光是那一个问题就非常巨大。

There's massive then. I mean, just that one problem is enormous.

确实如此，但这也正是研究的价值所在。很大一部分工作在于限定问题范围，比如能够明确写下并具体说明你想做什么，然后确保这确实是个具有挑战性的问题，并且有我们可以量化的良好指标。

It is, but that's why it's also research. A big part of it is trying to constrain the problem space, like being able to write down and specify what you want to do, and then making sure that it is actually a challenging problem, and there are good metrics that we can quantify

一种证明你成功的方式。

A way to say you're successful.

对，完全正确。正如我所说，这本身就是一个相当迭代的过程，获得批评意见、获取其他研究者的观点就是从那个时间点开始的。因为当研究还处于构思阶段、初始阶段时，正确提出问题并构建合适的语境实际上非常重要。

Right, exactly. And that itself, as I said, is quite an iterative process, and getting that critique, getting that view from other researchers starts at that point in time. Because, like, when the research is at the idea space, at the initial stages, it's actually quite important to formulate the right problems and and and sort of the right context.

通用人工智能不会一夜之间实现。这就是为什么——正如你在参观中听到的——这栋楼里有这么多不同的研究小组。你不能只从一个方向突破就解决通用人工智能问题。提出正确的问题意味着要超越导航等单一技能，深入挖掘智能的构建模块并同时攻克这些难题。

AGI is not going to happen overnight. It's the reason why, as you heard on the tour, there are so many different research groups in this building. You're not going to crack AGI by attacking on only one front. And formulating the right problems means going beyond individual skills like navigation and drilling down into the building blocks of intelligence and tackling those too.

从DeepMind成立之初，如果目标是通用人工智能，那就必须涉及控制问题。必须涉及主动算法。这就是为什么需要研究强化学习，这就是为什么需要研究智能体。

From the beginning of DeepMind, if the goal is about AGI, then it has to involve control. It has to involve an active algorithm. And that is why you need to do reinforcement learning. That is why you need to work on agents.

我们需要能够与环境互动的智能体，它们能通过试错学习采取何种行动。这就是Karai所称的策略。这是任何智能体——无论是人类、狗还是智能代理——都应具备的基本能力。如果科学家能在一个应用上取得成功，那么这些经验教训也应适用于其他领域。

We need agents that can interact with their environment, that can learn through trial and error what actions to take. What Karai calls its policy. It's a fundamental thing that you would expect to find in any intelligent being, humans, dogs, or agents. And so if the scientists can get it right in one application, the lessons learned should apply elsewhere.

训练智能体的核心在于，你可以从零开始，然后智能体逐渐积累知识，形成完成特定任务的策略，它会自主创造、构建并形成自己的策略体系。当然，随后我们会审视并试图理解：这个策略是否合理？为何采用这种策略？有时结果令人惊讶，因为它超出了我们的既有认知；有时你观察后发现，虽然策略能达成目标，但显然不是最优解。

The gist of training an agent is you can start completely from scratch, and then slowly the agent builds that knowledge, that strategy of how to achieve a certain task, and it creates, it builds, it comes up with its own strategies, it comes up with its own policy. So then, of course, at it and trying to understand does that policy make sense, why is it doing that policy? Sometimes it's so surprising because it's something that we haven't thought about before. Sometimes you look at it and you see that, nah, that doesn't make sense. Yes, it it it achieves something, but it's clearly not optimal.

你还记得自己第一次意识到或相信通用人工智能（AGI）可能实现是什么时候吗？

Can you remember when you first realized or or believed that AGI was possible?

我们曾在雅达利游戏上测试这些智能体，但它们表现平平。我们当时束手无策——尽管团队有好几个人，就是无法让它们运作。后来我们逐步简化问题，一而再再而三地简化。

So we tried these agents on Atari games and they didn't do much. And we couldn't we couldn't make them work. Right? Like, there was a there was a team of, like, like, several people there, and we couldn't make them work. And then slowly, we started simplifying the problem, and simplifying the problem, and simplifying the problem.

最终我们将其简化为一个极其微小的问题：就像10x5像素的图像里移动一个像素点，让智能体尝试控制它。当问题简化到这个程度——我称之为强化学习的MNIST基准——我们终于找到了可行方案，深度强化学习智能体开始正常运作。毕竟，这是个简单问题。

And we ended up with a very tiny, simple, trivial problem, like really just like a five pixel by 10 pixel image, and one pixel moving in that image, and the agent trying to control that. Once we reduced the problem to that, which I call the MNIST of reinforcement learning really, we could find a working solution. We started having these deep reinforcement learning agents working. Right? Because it's a simple problem.

当然，你可以直接编写程序解决它。但我们的理念是尝试用深度强化学习的方法，从像素级输入开始解决，构建一个我们认为能泛化到更多不同问题的系统。当我们看到成效时，实际上只用了几周时间就让10到15款雅达利游戏跑起来了。

Of course, you can write a program to solve that. But the idea was, like, try to do deep reinforcement learning and try to solve it that way. Try to solve it from pixels. Try to come up with a system that we think can generalize to more to to to different problems, to more problems. And once we saw that, actually, it was a matter of weeks we had 10 or 15 Atari games, like, being sold.

从那个微小起点开始，短短几周就扩展到整个雅达利平台——这是个重大转折点。正是这种进展推动着我们继续前进：我们不断选择更多样化的问题集，这些最终对实现AGI至关重要。

Like, from that tiny thing, in the matter of weeks, you go to Atari, that that was a big moment. That's what we keep on going. That, like, we select more and more diverse set of problems that we think are important at the end for AGI.

这是一个关键点。在这里，智能是指智能体在广泛环境中实现目标的能力。因此要实现通用人工智能（AGI），我们需要智能体能够解决更困难、更多样化的问题。您正在收听的是DeepMind播客，一扇了解人工智能研究的窗口。我们越接近AGI，这项技术就会变得越强大和精密，我们在日常生活中对它的依赖程度也会越深，而如果我们误解了算法的局限性，其后果可能会非常严重。

That is a key point. Here, intelligence is an agent's ability to achieve goals in a wide range of environments. So to get to AGI, we need agents to solve harder and more diverse problems. You're listening to DeepMind, the podcast, a window on AI research. The closer we get to AGI, the more powerful and sophisticated this technology gets, and the more we rely on it in our everyday lives, the more dramatic the consequences could be of us misunderstanding the limitations of the algorithms.

正因如此，在推进科学发展的同时，研究人员也在努力确保智能体具有可靠性、适应性，最关键的是不可被腐化——这项工作现在就在进行。

And that is why, in parallel to pushing the science forward, researchers are also working to ensure, right now, that agents are reliable, adaptable and crucially, not corruptible.

当你向神经网络展示一张公交车的图片时，它会识别出这是公交车，对吧？图片里有辆公交车。但事实证明，你可以对同一张图片进行细微修改——这些修改对人眼几乎不可见——神经网络却会坚称这是鸵鸟。这类对抗性攻击大多是人眼无法察觉的，它们不会大幅改变图像的实际内容，我们感知不到，但由于这些是算法，它们对输入数据的微小波动都很敏感，从而导致输出结果改变。

You show the image of a bus to a neural network, and it will say this is a bus, right? There's a bus in this image. Well, it turns out that you can take the same image, modify it a little bit, which is almost invisible to a human eye, but the neural network will say that it's actually an ostrich. These adversarial attacks are things most of the time that are invisible to the human eye that doesn't change the actual content of the image too much, we don't perceive it, but because these are algorithms and they are sensitive to even very small fluctuations in the input data, then it changes the output.

这为什么重要？为什么要在现实世界的人工智能应用中阻止这种情况发生？

Why does that matter? Why do you want to stop that from happening in in sort of real world AI?

有两个原因。其一如我所说，是鲁棒性问题。因为我们训练这些算法是希望它们能在现实世界中发挥作用。虽然训练数据来自现实世界，但你当然无法准确预知算法最终会处理什么样的数据。所以我们要确保算法对这些潜在的、类似噪声的干扰具有鲁棒性。

Well, for two reasons. One of them, as I said, is robustness. Because when we train these algorithms, we want them to be useful in real world. And we train them on datasets that we captured from real world, but, like, you cannot know exactly what's gonna happen, of course, what kind of data is going to be at the end of the day this algorithm is gonna consume. So we want to make sure that the algorithms are robust to these kinds of potential, noise like things that you want your algorithm to be robust to that.

对吧？从另一个角度看，这关乎安全性。要确保怀有对抗意图的人无法仅通过对输入数据进行微小调整就改变算法输出、改变神经网络的结果。我们实际上有整个研究小组专门从事这项工作，致力于构建更严谨、更鲁棒的人工智能，因为这些确实是真实存在的问题。毕竟，我们正在训练这些算法。

Right? And from another point of view, it's about safety. Being able to say that, like, someone maybe with an adversarial intent won't be able to change the output of this algorithm, output of this neural network just by making very small adjustments to the inputs. We have a we have a whole research group actually on that, on on on working on rigorous and more robust artificial intelligence, because, yes, these are real things. In the end, we are training these algorithms.

正如我所说，我们不仅要观察它们，更致力于开展可量化的研究，以理解它们行为背后的原因并尝试解释这些行为。其中部分工作也包括评估它们的鲁棒性程度。

And as I said, it's not just like looking at them, but we're trying to do more quantifiable research on understanding why they are doing what they are doing and trying to interpret that. And part of it is also understanding how robust they are.

我们已经见识过智能系统究竟有多脆弱的真实案例。研究表明，只需在停车标志上贴一小段黑色胶带，就能诱使无人驾驶汽车加速行驶。这种胶带对人类司机来说几乎难以察觉，却足以让车载算法将停车标志误读为限速45英里的标识。另有科学家发现，只要让人佩戴特制的玳瑁框眼镜，就能欺骗面部识别算法将其误认为米娅·约科维奇。而用于医疗诊断的图像，仅因使用不同品牌的扫描仪拍摄，就可能导致结果严重失准。

We have already seen real examples of just how fragile intelligence can be. Researchers have shown that you can add a little bit of black tape to a stop sign that will trick a driverless car into speeding up. The tape is so subtle that it would look innocuous to a human driver, but it's just enough to make the algorithms inside the car misread it as a 45 mile an hour speed limit sign instead of an instruction to stop. Other scientists have worked out how to fool facial recognition algorithms into thinking someone is Mia Yokovich just by making them wear a specially designed pair of tortoiseshell glasses. And the images that are used for medical diagnoses can end up giving wildly inaccurate results just if a slightly different brand of scanner was used to take them.

我们的目标是实现一种能切实保障此类事件不会发生的状态。这正是开展这项研究的核心理念——不是针对特定对抗性攻击或案例进行防御，而是构建出无论面对何种情况都能保持稳健的系统。当然，从智能体的角度来看，这还涉及整个安全性问题。

You want to reach a state where you can actually guarantee that things like that won't happen. That's the main idea behind doing this research. So not sort of try to defend against particular adversarial attacks or examples, but actually come up with systems that are going to be robust no matter what. From the other point of view, of course, if you are thinking about agents, there's the whole safety issue.

你具体指什么？当你们讨论安全性时，实际指的是哪些方面？

What do you mean? When you guys talk about safety, what are you actually talking about?

当智能算法自主决策时，我们需要确保其遵循自身策略行动时，仍能与我们的设计意图保持一致。考虑到这些算法持续学习的特点，我们还希望学习过程最终产生的智能体行为始终符合我们的预期目标。

If we have an intelligent algorithm making decisions for itself, you want to have some sort of guarantees that when it's acting with its own policy, it is aligned with what you intended it to do. If you think that these algorithms continued learning all the time, then we want that process to be also producing an agent that is aligned with what we have intended for it to do.

请再详细说明一下，举例描述你们试图规避的情况。

Fill in the gaps for me a little bit. Give an example of what you're trying to avoid. It

这涉及到相当技术性的层面——重点不在于防止智能体犯错。毕竟错误总会发生，对吧？就像我们训练的策略并非总是最优，它们不可能永远正确行事。

becomes quite technical in the sense that it's not like you are trying to avoid the agent from making a mistake. Right? Because mistakes happen, right? Like we train policies and they are not always optimal. They don't do the right thing all the time.

本质上这并非关于错误防范，而是如何确保学习算法在行为边界内运作，使其从根本上符合特定规范。

It's not about that really. It's more about you have a learning algorithm and you want to make sure that it sort of conforms to certain boundaries of behavior in essence.

如果我们能开发出安全、稳健且合乎伦理的人工通用智能（AGI），其影响将不可估量。但正如Demis所言，在实现AGI的征途上，随着智能体逐步攻克日益复杂的难题，相关发现同样可能震撼世界。Trevor Back是DeepMind科学项目的产品经理，他在我们第五集讨论的Moorfields眼科医院合作项目中发挥了关键作用。同时他也参与制定DeepMind的未来发展方向，决定接下来需要应对哪些挑战与机遇。

If we can develop safe, robust, ethical AGI, then the impact could be staggering. But as Demis mentioned, so too could the discoveries on the way to AGI as agents learn to solve increasingly hard problems. Trevor Back is a product manager for the DeepMind Science program. He played a key role in the Moorfields Eye Hospital collaboration that we talked about in episode five. But he also has a hand in deciding what the future holds for DeepMind and what challenges and opportunities lay in store for them to tackle next.

目前我们正处于探索新研究领域的阶段，可能性是无限的。以AlphaFold系统为例，其原理并不局限于蛋白质折叠领域——它本质上是理解原子相互作用方式，以及如何从基础概念构建材料。因此材料设计是个令人振奋的研究方向。想象一下通过AI算法实现高温超导体的设计，这难道不令人惊叹吗？

So we're really at the stage of exploring what other areas we should work on, but the possibilities are endless. If you look at the way the AlphaFold system works, there's nothing really specific in there to protein folding. It's around understanding the way that atoms interact, the way that you can build material from base concepts. So, looking at material design is a really exciting stage. You design or imagine a high temperature superconductor that's been sort of brought to life through an AI algorithm, right?

是否存在更高效的寻找这类材料的方法？

Is there more efficient ways of looking for those types of materials?

我们为何要关注高温超导体？

Why do we care about high temperature superconductors?

这正是科学研究的魅力所在。我们过去在医疗领域的工作都聚焦于具体问题，构建针对性的AI系统需要耗费大量时间精力。但若能致力于解决基础科学问题，就可能开辟全新研究领域，进而影响更广泛的问题范畴。超导问题为何重要？若能攻克超导难题，不仅可以通过增强聚变装置的磁场来解决能源问题，还能创造新型计算系统。

So this is the amazing opportunity of working in science. So our previous work in healthcare has been focused on very specific problems, and it takes a lot of time and effort and energy to build an AI system that works for those specific problems. If instead you can spend your time and energy solving a fundamental science question, then perhaps you can instigate a whole new field of interest and potentially impact a much wider array of problems. So why is superconductivity a problem? If you could solve superconductivity, not only could you solve a lot of the energy problems by having a larger magnetic field around fusion, but you could also create a new type of computing system.

要知道，这些领域的任何单项突破都可能带来无数机遇。

You know, there there's lots of opportunities that come from just a single breakthrough in one of these areas.

我认为绝不能低估这一点。试想若能在核聚变领域取得突破，对地球和人类文明的影响将是难以估量的，对吧？

I think it's important not to understate this, because the idea that you could have some kind of an impact on nuclear fusion, for example, the implications for the Earth and humanity are just enormous. Right?

确实如此。我认为这正是我获得能在世界上产生最大影响的机会的原因。AI作为一种革命性技术，不仅能影响我们当前所做的一切，还将改变我们对未来的构想。而探索机遇空间这类任务，恰恰是AI最擅长的领域，它本质上就是一种高效的搜索算法。

Exactly. I think this is the reason I came opportunity to have the greatest impact I could in the world. And I think AI is one of those revolutionary technologies that amazingly could impact everything we do, but also everything we think about doing in the future. And the sort of opportunity to explore and search the space of opportunities is really something that's well designed for AI to do. It's a very efficient search algorithm.

因此，如果你能把问题转化为可通过AI搜索的形式，那么在发现新型材料、识别天文数据异常、发现新类型恒星、定位更多黑洞等方面，机会可能会增加十倍甚至百倍。所有这些可能性，都只需通过AI的应用就能实现。

And so if you're able to set up the problem in such a way that is searchable via AI, you really could have 10 or 100 fold type of increase in the opportunity for finding novel materials or finding anomalies in astronomical data, finding new types of stars, finding more black holes, you know, all these types of opportunities are available simply via the application of AI.

说真的，现在人类面临的许多最大难题其实都是科学问题，这种说法并不为过。比如粮食水源获取、气候变化、医疗保健等，这些都是AI能够推动进步的领域。

And it's not stupid to say that actually lots of the biggest problems that face humanity at the moment are science problems. Right? Like, you know, access to food and water, climate change, health care. All of these things is stuff that AI can make progress in.

我完全同意。现实世界中存在的诸多物理性问题，如果能从基础科学层面取得突破，就能将能源成本降至近乎为零，或让全球粮食供应更加充裕，这将在多方面推动社会进步。

I think that's right. I think a lot of the the sort physical problems that exist in the real world are certainly things that if you can make a difference to some of the foundational science aspects, you could reduce the cost of energy essentially down to zero, or you could make food more readily available across the world. And so that would really help society progress in a number of ways.

这是个令人神往的前景。要实现通用人工智能，需要整合本期播客探讨的所有研究方向——记忆、推理、逻辑、学习、语言、具身认知等等，甚至更多。我们将在下期节目中与DeepMind联合创始人Demis Hassabis对话时深入探讨这些理念。

It's a tantalizing prospect. Solving intelligence and creating AGI will take the full range of research explored in this podcast and more. Memory, reasoning, logic, learning, language, embodied cognition and more. So much more. And we're gonna explore some of those ideas in our next episode when I meet with DeepMind cofounder, Demis Hassabis.

他将讲述如何创建世界顶尖的AI研究机构，透露令他夜不能寐的忧虑，并畅谈对未来的期许。

He tells us how he created the world's leading AI research outfit, reveals what keeps him up at night, and opens up about his hopes for the future.

那些看似无解的重大问题始终令我着迷又困扰——生命的意义、宇宙的起源、意识的本质，这些疑问如同我脑海中不断鸣响的警笛。而我尝试理解的方式，就是优先构建人工智能。

I'm just fascinated and also troubled by the things around us that we seemingly don't understand, all the big questions, you know, the meaning of life, how the universe start, what is consciousness, all these questions, which I feel like a blaring sort of klaxon in my mind that I would like to understand. And my attempt to doing that is to build AI first.

如果你想了解更多关于通用人工智能的知识，或探索DeepMind之外的AI研究世界，每期节目的注释中都有大量实用链接。如果你认为某些故事或资源对其他听众有帮助，请告诉我们。你可以通过Twitter给我们留言，或发送邮件至team@podcastatdeepmind.com。你也可以用该地址向我们提交关于本系列的问题或反馈。

If you would like to find out more about artificial general intelligence or explore the world of AI research beyond DeepMind, you'll find plenty of useful links in the show notes for each episode. And if there are stories or resources that you think other listeners would find helpful, then let us know. You can message us on Twitter or email the team@podcastatdeepmind.com. You can also use that address to send us your questions or feedback on the series.