第二部分：超越钓鱼攻击——与Four Flynn探讨AI时代的网络威胁

你认为长期来看谁会获胜？

Who do you think wins in the long term?

是网络犯罪分子还是安全防护方？

Is it cybercriminals or security?

我认为防御者最终会取得胜利。

I think the defenders ultimately will win.

我们未必能赢得每一场战斗，但我希望我们能赢得这场战争。

We won't necessarily win every battle, but I hope that we can win the war.

欢迎收听Google DeepMind播客，我是主持人汉娜·弗莱。

Welcome to Google DeepMind, the podcast with me, your host, Hannah Fry.

现在为您带来与Google DeepMind安全副总裁托尔·弗林对话的第二部分。

We are bringing you part two now of our conversation with Thor Flynn, VP of Security at Google DeepMind.

在上期节目中，我们讨论了数字系统遭受攻击的惊人数量，以及为保护它们而进行的持续斗争。

In the previous episode, we talked about the terrifyingly large numbers of ways that our digital systems are vulnerable to attack, as well as the ongoing fight to defend them.

托尔在过去二十年一直处于国际网络安全的最前沿，他的分享充满了故事和深刻见解。

And four is completely full of stories and incredible insights, having spent the last two decades at the very forefront of international cybersecurity.

如果你还没听过上期节目，请暂停本期内容先去收听，我们稍后再见。

So if you haven't watched that one yet, press pause on this, go and have a listen, come back here later, we'll still be waiting for you.

其他听众请继续享受本期节目。

But for the rest of you, enjoy the episode.

到目前为止，我们一直在讨论保护系统面临的实际技术挑战。

Well, up until now, we've been talking about the real technical challenges that come with protecting our systems.

但正如你在上期播客结尾提到的，从很多方面来说这只是问题的一半，因为还存在人性层面的因素。

But as you mentioned at the end of the last podcast, that's really half of the problem in a lot of ways because there is also this very human side too.

攻击者是谁？他们用来欺骗人类的策略是什么？在AI时代这些又发生了怎样的变化？

Who's creating the attacks, the tactics that they're using to trick us as people, and also how all of that is changing in the era of AI.

如果可以的话，我想就此话题展开更多讨论。

So I wanted to pivot to talk a bit more about that, if I may.

是的。

Yeah.

也许我们该从讨论这些恶意行为者开始。

And maybe it's worth us starting off by talking about these bad actors.

比如，他们到底是什么人？

Like, who actually are they?

他们的动机是什么？

What is their motivation?

嗯，我是说，恶意行为者形形色色，我想。

Well, I mean, bad actors come in all shapes and sizes, I guess.

他们可以分为好几类。

There's a number of different classes of these.

我认为在安全领域，我们通常将其分为国家行为体——这类主要关注地缘政治目标，通常是间谍活动或支持地面作战的进攻性网络攻击。

I think in general in security, we break them down to nation state actors that I think are largely focused on geopolitical aims, often espionage or offensive cyber attacks that are in in support of warfare operations on the ground.

我认为最近还有个值得关注的问题是预先部署。

I think there's also a recent concern about prepositioning.

即使两个大国之间没有热战，网络攻击行动也常作为预先部署在进行。

Even if there's not a hot war going on between two powers, oftentimes there's cyber offensive operations happening as a prepositioning.

就像是为可能的热战提前布局。

Like maneuvering in case a hot war Exactly.

我们在某些领域已经看到这种情况的迹象，比如电网或关键基础设施。

So we've seen in some areas signs of this happening in places like power grids or critical infrastructure.

对。

Right.

所以知道可能存在现实冲突时，就提前渗透电网系统以便日后行动。

So knowing that there's a potential real world conflict, you infiltrate the power grid system so that you can act on it later.

没错。

That's right.

保持你的存在，并定期验证你仍能对这些环境实施指挥与控制。

And maintain your presence and validate that you continue to have command and control of those environments periodically.

没错。

Right.

每隔几个月，就会有人回来检查灯光是否亮着，确保系统仍在运行，确认他们能实施破坏，然后悄然离开。

Every few months, you'll see somebody come back, make sure the lights are on, make sure that their systems still work, make sure they can disrupt things, and then they go away.

遗憾的是，这种情况比大多数人愿意承认的要普遍得多。

So that's unfortunately fairly more common than most people would like to admit.

是的。

Yes.

然后你会频繁看到次国家行为体的身影。

And then you sort of see sub nation state quite a bit.

次国家行为体的活动与此交织，但多数动机是经济利益。

Sub nation state activity blurs into this, but a lot of that is financially motivated.

对吧？

Right?

比如勒索软件——过去五年里你们肯定听得够多了——这种手段不幸已变得相当普遍。

And so you'll see, for example, ransomware, which I'm sure you have heard quite a bit about over the last sort of five years, which is very unfortunately quite common.

通常这类攻击会导致公司或个人的数据被窃取加密，然后被挟持索要赎金。

And usually that's an attack that basically causes a company or a person to have their data basically stolen and encrypted and, you know, held for ransom.

更现代的版本是：作为企业，你有一个核心数据库和备份来服务客户。

The more modern incarnation of this is that you as a company would have a core database that you rely on, as well as a backup to serve your customers.

你的整个业务都依赖于此。

And your whole business depends on it.

他们会悄无声息地渗透你的公司，在你毫无察觉时突然封锁你对备份和数据库的访问权限，让核心业务瘫痪。

They would slowly and quietly compromise your company, unbeknownst to you, and then all in sudden, one day, they would lock out your ability to get access to your backups and your database, and your coal companies offline.

然后你会收到用比特币或其他加密货币支付赎金的要求，以换取数据解密和业务恢复。

You would get a demand for a certain number of Bitcoin or what have you, cryptocurrency, to pay them to get the data unencrypted and your business back and running.

当你说‘次国家实体’时，这是否指的是——比如一个受制裁的国家——将其作为筹集资金的一种方式？

And when you say sub nation state, is this, I don't know, for example, a country that has a number of sanctions on them, is sort of a way to generate funds?

是的。

Yeah.

确实可以看到国家实体将其作为筹资手段，但同时也会看到一些独立攻击团体，他们与国家实体关系松散或毫无关联，有时也会发动此类攻击。

So you do see nation states actually using it as a way to raise funds, but you also see sort of independent attack groups, loosely affiliated or unaffiliated with nation states conducting these sorts of attacks sometimes as well.

在判断某人好坏方面是否存在一定的灰色地带？

Is there a bit of a gray area in terms of whether someone is bad or good?

对吧？

Right?

我在想，我们刚才谈到这些零日漏洞无处不在的情况。

I'm thinking here, we were talking about how there are these zero day vulnerabilities all over the place.

我们不知道它们藏在哪里。

We don't know where they are.

但发现它们可能价值连城。

But finding them can be worth a lot of money.

意思是，如果你作为个人发现了零日漏洞，理论上可以选择卖给恶意行为者，但也可以将其售回给相关公司。

I mean, if you were an individual searching and found a zero day vulnerability, you presumably have the choice to sell it to a bad actor, but you could also sell it back to the company itself.

没错。

Yeah.

实际上存在一个非常活跃的黑市，甚至灰市，专门交易漏洞。

So in fact, there's a whole active black market, gray market even, for vulnerabilities.

这些漏洞有不同类型的买家。

And there's a number of different buyers of those vulnerabilities.

对吧？

Right?

有些买家是试图为执法部门配备工具的公司，用于访问犯罪分子的设备。

So some buyers are companies that are trying to equip law enforcement with tools to access the devices of people who have committed crimes.

不同的人对此有不同的看法，但这无疑是生态系统中的一个组成部分。

Various people have different views on that, but that's definitely one component of the ecosystem.

我认为政府也是这些东西的最大买家之一。

I think also governments are one of the biggest buyers of these things.

所以，再次强调，这取决于你效忠于哪个政府。

And so, again, you know, it depends on what government you have allegiance to.

我觉得这取决于你对此的感受。

It depends on how you feel about that, I feel like.

但值得注意的是，这些漏洞的价值堪比黄金。

But it's worth noting that these vulnerabilities are very much worth their weight in gold.

其中有些漏洞非常昂贵。

They're very expensive, some of them.

正如我们讨论过的，有时候它们的价格可能高达数百万美元。

You know, it can be, like we talked about, many millions of dollars, in fact, sometimes.

而且市面上购买这些东西的人形形色色。

And there's a varied number of people buying these things out there.

这种想象方式相当有趣——就像寻宝一样，而谷歌的介入仿佛在说：我们要找到所有宝藏，然后修复它们使其失去价值。

That is quite an interesting thing to a way to imagine it then, that it's like treasure hunting, as it were, and Google's kind of coming in and saying, we're gonna hunt for all the treasure and then fix it so it no longer has any value.

确实如此。

I mean, that's right.

因为我看到AI存在一种风险，如果我们不采取行动，它可能会自行扰乱市场。

Because I see a risk that AI, if we were to take no action, I think it could by itself disrupt the marketplace.

对吧？

Right?

因为我们知道恶意行为者最终会做和我们相同的事——发现这些漏洞并使其可能被更多坏人利用。

Because we know bad actors will do the same things we're doing eventually, which is to find these vulnerabilities and make them, unfortunately, potentially more accessible to more bad actors.

我的担忧在于，即将出现一波人利用AI（这项曾经非常精密昂贵的技术）来发现零日漏洞和脆弱性，并用它们造成破坏。

I mean, that's concern, is that there's this wave coming of people using AI that previously was very exquisite, very expensive, to find these zero days and these vulnerabilities and use them to cause damage.

这就是风险所在。

That's the risk.

我并不是说这一定会发生或不会发生，但这是我们担忧的事情之一。

I'm not saying that's going to happen or it's not going to happen, but that's one thing we worry about.

因此，重申一遍，这就是为什么谷歌等公司必须投资这些系统，使其成为该领域的佼佼者，并善加利用——帮助生态系统自我修复、自我完善、增强抵抗力，而不是坐等恶意行为者得逞。

And therefore, again, that's why it's important for Google and others to invest in these systems to be the best there is at this and to use that for good and to help the ecosystem heal itself and to improve itself and get more resistance instead of just waiting around and letting the bad actors do it.

我认为这其中还有个非常耐人寻味的方面，就是选择公开披露的决定。

I think there's another really fascinating aspect to this, which is about the decision to go public.

对吧？

Right?

这种透明化行为本身就是一种安全措施。

Like, the act of transparency as a security measure in itself.

没错。

Right.

说说零日计划的初衷吧。

I mean, tell me about the motivation behind Project Zero.

它是怎么诞生的？

How did that come about?

是的。

Yeah.

首先要归功于谷歌多年来那些创新零日计划的杰出人才。

Well, I have to give credit to the absolute incredible folks at Google over the years that have innovated Project Zero.

但零日计划确实是世界级的卓越项目。

But Project Zero is really an absolute world class effort.

它汇聚了史上最顶尖的黑客精英，他们日复一日的工作就是挖掘系统中的新型漏洞。

Combining, I think, some of the most elite hackers that have ever lived, their job day in and day out was to find novel vulnerabilities and systems.

同时采用漏洞披露与透明化策略，旨在推动整个生态系统变得更好。

And then having a vulnerability disclosure and transparency approach that was designed to help the ecosystem become better.

我是说，Project Zero当时为整个生态系统带来的一个令人难以置信的深刻理念，就是所有公司都适用九十天的漏洞披露期限。

I mean, one of the things that Project Zero introduced to the ecosystem that at the time was unbelievably profound was this idea that there's a ninety day disclosure timeline on all companies.

从你们得知漏洞存在的那一刻起，有九十天时间修复它，否则我们会向全世界公开这个漏洞。

You have ninety days from when you learn about a vulnerability to patch it or we're telling the whole world what it is.

哇。

Wow.

到那时，所有恶意分子都能利用它。

At which point, all bad actors can exploit it.

没错。

That's right.

免费提供。

For free.

免费提供。

For free.

甚至在暗网上都不会以5000万美元的价格出售。

Not even for sale for $50,000,000 on the dark web.

对。

Right.

当时我觉得所有人都和你一样，认为这太疯狂了。

And at the time, I think everybody was like like you are, thinking that this was crazy.

对吧？

Right?

但这个做法彻底改变了整个行业对安全问题的优先级排序。

But what it did is it completely changed the way all people in the industry prioritize security.

因为问题在于，白帽黑客发现漏洞并告知公司后，这些公司往往不会优先处理。

Because the problem was that the good guys would find a vulnerability and disclose it to a company, then and those companies often wouldn't prioritize them.

他们会说'知道了知道了'，比如'我们会在下一版Windows或其他什么版本中修复'。

And say, Yeah, yeah, yeah, like, we'll put it out in the next release of Windows or whatever it is.

然后基本上，几个月甚至一年过去了，没人修复它，然后坏人各自独立发现了同一个漏洞，到处利用无辜的消费者。

And then, basically, months, a year down the line, nobody fixed it, and then the bad guys separately and independently found the same vulnerability and were exploiting poor consumers everywhere.

所以零项目团队表示：不，事情不该这样运作。

And so Project Zero said, No, that's not how this works.

现在通过漏洞披露奖励计划发现漏洞对所有人来说都成了常态。

Now it's just normal for everybody to get a vulnerability through a vulnerability disclosure bug bounty program.

要知道，很多时候人们会为这些漏洞支付奖金奖励发现者。

You know, a lot of times people will pay out and reward folks for these.

而且，是的，我想说大多数情况下这些漏洞会在90天窗口期内被修复。

And then, yeah, I mean, I would say more often than not, these are often fixed within the ninety day window.

再次强调，这都要归功于零项目团队。

Again, all thanks to Project Zero.

所以这对整个行业产生了巨大影响。

So that was a huge impact on the industry.

那么过去是否存在有人明知严重漏洞却故意隐瞒的情况？

Have there been instances in the past then where people have known about really serious vulnerabilities and deliberately sat on them?

嗯，我认为有政府这么做的案例，因为他们想利用这些漏洞攻击其他国家。

Well, think there's cases of governments doing that because they wanna use them to exploit other countries.

Kwon，举个具体例子。

Kwon, give me an example.

嗯，我想网络安全史上最著名的例子之一应该叫永恒之蓝。

Well, you know, I think probably one of the most famous examples of that in security history is called Eternal Blue.

普遍认为永恒之蓝源自美国某些机构，他们将这些工具用于国家安全的防御和进攻目的。

Eternal Blue was widely believed to be attributed to America's agencies that were using these tools as part of national security for defensive purposes and offensive purposes.

呃...我不想点名具体国家或组织，但在永恒之蓝事件中，这个工具包不知怎么被外部泄露了。

And, you know, I mean, I I don't wanna call out any particular country or group, but, you know, in in the case of Eternal Blue, that was somehow leaked externally as part of a toolkit.

后来我认为该工具包中一些新型漏洞实际上被武器化了，成为名为WannaCry的蠕虫病毒的一部分。

And then I think some of the vulnerabilities that were novel in that toolkit were actually weaponized as part of a worm that was called WannaCry.

就是那个针对英国国家医疗服务体系的攻击。

Which was the one which targeted the British National Health Service.

但有趣的是，当时其实已经存在针对它的补丁。

But the thing is about that one that was interesting as well was that there was a patch that existed for it.

是这样吗？

Is that right?

其实存在一种方式，使得漏洞不一定非要存在于那些系统中。

There was a way that the vulnerability didn't need to necessarily be in those systems.

是的。

Yeah.

没错。

So that's right.

微软在时间线的某个节点发布了补丁，我记得是在蠕虫病毒侵袭NHS之前。

A patch was produced somewhere along the timeline by Microsoft, and I believe it was prior to the worm taking hold at NHS.

所以我觉得值得深入探讨一下，为什么NHS在补丁可用的前提下仍如此脆弱。

And so I think it's worth an interesting digression to discuss briefly why NHS was so vulnerable to this worm, when indeed a patch was available.

常见的情况是，当你作为安全人员试图安装重要补丁时，走进医院会发现令人震惊的景象。

What happens is often that you go and you try to apply the greatest patches, and you're trying to do the right thing as a security person, and if you go into a hospital, you'll find you'll be surprised.

你知道，那些病床边的设备居然运行着Windows系统。

You know, those things that are next to the patient's bed are like running windows.

是的，这很荒谬。

Yes, it's silly.

至少那些你戴着腕带交互的电脑——用来计算用药剂量和记录医嘱的设备。

At least the computers that you go and you interact with, with your bracelet, you know, or whatever to figure out what drugs dosages to give and take the notes from the doctor.

这些设备运行的都是我们所熟知的操作系统。

Like, these things are running operating systems that we know of.

因此我确信当时的情况肯定是：可怜的安全团队过来表示'我们要给这些系统打补丁'。

And so, as a consequence of that, I'm pretty sure what must have happened is the poor security team came along and was like, We're gonna patch these systems.

他们试图安装补丁，结果有一天导致半个医院的系统瘫痪。

And they tried to apply the patches, and one day, it broke half the hospital.

对吧？

Right?

然后某个管理人员说：听着，我们的职责是拯救生命。

And then some administrator said, Look, our job is saving lives here.

我们承受不起这些系统出故障。

We can't have these systems not working.

我们有心肺复苏仪等各类设备。

We have a defibrillator and all this stuff.

这方面容不得半点闪失。

We can't mess around with this.

于是他们在制度上变得对IT系统变更极度保守。

And then they become institutionally risk averse to change in the IT systems.

这种做法的副作用是：虽然短期优先保障了人类健康，却因无法安全实施系统变更而埋下长期安全隐患。这种情况在电力网络、水电站等同样运行Windows或Linux系统的关键基础设施中也很常见。

And the side effect of that is that you have short term prioritization of human health, whereas you end up with these long term vulnerabilities because you don't have the ability to safely affect change in You the see this also in other critical environments like critical infrastructure, like power grids or hydroelectric dams that also often are running based on Windows or Linux systems or what have you.

同样地，越是关键的系统，讽刺性地就越难以接受变更——这成了一个恶性循环。

And again, it's the same situation where they're so critical, and ironically, the systems that are the most critical then have a side consequence of having the least acceptance of change.

而拒绝变更意味着最难以实施环境中的安全改进。

And that lack of acceptance of change means that you have the least ability to affect security improvements into the environment.

不幸的是，这就是现实世界的运行方式。

So this is unfortunately the way the world is.

确实如此。

Absolutely.

但我认为这确实揭示了问题的严重性——这不仅仅是技术层面的挑战。

But this, I think, does really illustrate the sort of the depth of this problem here, that it isn't just a technical challenge.

其中还涉及到人为因素。

You have got the human element of this.

没错。

That's right.

那么让我继续谈谈社会工程学这个概念，它是通过人为因素入侵的另一个潜在薄弱环节。

So let me go on to the idea of social engineering here as a different potential point of failure, of getting in through the human.

是的。

Yeah.

人工智能如何改变了社会工程学的格局？

How has AI changed that landscape of social engineering?

是的。

Yeah.

要知道，这正是人工智能从安全角度令人担忧的方面之一——利用AI制造深度伪造内容。

You know, do think this is one of the things about AI that is concerning from a security point of view, is the use of AI to cause deep fakes.

本质上就是能够克隆某人，并实现实时视频通话的效果。

Basically, being able to clone somebody and have a video, live video experience.

想象一下，首席财务官通过视频会议联系你。

You know, so imagine a CFO calls you up in a video conference.

看起来就是本人。

It looks like them.

他们的互动方式、习惯用语和声音都一模一样。

You know, they interact and have the same idioms and the same voice as them.

如果你是财务团队的中间人员，接到首席财务官的电话，很可能会确信是本人，然后被要求进行电汇转账。

And, you know, if you're a middle person in the finance team and you get the CFO on the line, you know, you might well be convinced that it's truly them and be asked to transfer money through a wire transfer.

而这种

And this

情况确实发生过。

has literally happened.

截至目前已经发生过多次了。

And it's happened multiple times already to this date.

这就是深度伪造攻击的一个例子。

And that's one one example of a deepfake attack.

我们还见过其他案例，比如有人会克隆女儿的声音打电话，假装被绑架勒索，要求母亲汇款。

I think there's other examples that we've seen of, you know, where somebody will call up and have cloned their daughter's voice and call them up on the phone and pretend that they're being held for ransom, and that the mother needs to send money.

可以想象，任何人遇到这种情况都会方寸大乱，尤其是当声音听起来确实像你养育多年的子女时。

You can imagine any of us would struggle to have our wits about us in those types of situations, especially if it sounds like your daughter, your son, and somebody you've known for so many years.

因此AI催生了一系列新型社会工程学攻击手段，这些攻击方式都极具创新性。

And so there there's a whole different bunch of different types of these social engineering type of attacks that are really novel, that are enabled by AI.

但从某种角度说，这确实让不法分子能更精准地定制钓鱼攻击目标。

But in a way, it sort of enabled bad actors to make their phishing attacks more directly tailored to the individual they're targeting.

确实如此。

I think that's right.

要知道，钓鱼攻击早已盛行多时。

You know, phishing attacks had already been pretty successful for some time.

网络安全领域有个概念叫'鱼叉式网络钓鱼'。

We had this concept in security called spear phishing.

不知道你是否听说过这个术语。

I'm not sure if you've come across that one.

顺便说个有趣的冷知识，还有种叫'鲸钓'的攻击。

And in fact, a fun side note, there's also whaling.

鱼叉式钓鱼就是为特定目标量身定制的钓鱼手段。

Phishing is targeted phishing that's of bespoke for that individual.

而'鲸钓'则专指针对CEO等高价值目标的钓鱼行为。

And then there's whaling, which is a term for basically going after CEOs or really high profile people.

我认为大语言模型带来的问题是，它们催生了一整套新的社会工程学风险。

So, I think the problem with LLMs is that there's a whole landscape of new social engineering risks that are introduced by them.

我总喜欢在坏消息后捎带点好消息。

I try to always give a bit of good news along with the bad news.

我认为这里的好消息是，历史上我们在安全领域发明的许多最佳实践仍然能派上用场。

I think the good news here is that a lot of the best practices historically that we've invented in security are still coming to the rescue.

想象一下你在一家公司工作，对吧？

So imagine you work at a company, right?

我们之前讨论过那个CFO的情景。

We talked about that CFO scenario.

如果你采用严格的多因素认证，不仅需要人脸和视频验证，而且要通过这种视频系统联系你时，对方必须拥有一种不可钓鱼的强认证方式——比如只能通过你手指触碰笔记本电脑等方式验证——那就能有效防御这类攻击。

Well, if you have strong multi factor authentication that requires not just a face and a video that looks right, but in order to be able to call you in the first place with such a video system, they would need to be able to have a strong form of authentication that isn't phishable, that's only, you know, connected to your finger on your laptop or whatever, then that is pretty strong defense against that sort of attack.

对吧？

Right?

你认为我们会永远生活在密码时代吗？

Do you think that we're gonna be living in the era of passwords forever?

还是说从消费者角度看，AI最终会让密码成为历史？

Or do you think that from the consumer side that that AI will make that a thing of the past at some point?

在讨论AI之前，其实已经有令人鼓舞的进展出现了。

Well, before we get all the way to AI, there's actually encouraging progress even well before that.

我最想提请各位关注的是'通行密钥'技术。

So probably the thing I'd turn your attention to most is something called pass keys.

通行密钥是一项即将普及的激动人心的登录创新技术。

Pass keys are actually a really exciting new innovation that's coming to, a login near you.

你可能之前见过这种技术。

And it you might have seen it before.

本质上就是当你登录时，不再需要输入密码，而是显示一个可以用手机扫描的二维码。

It's essentially when you log in and instead of typing in a password, it'll show a QR code that you can take a picture of with your phone.

由于你的手机已预先完成身份验证，就能自动帮你完成网页服务的登录。

And because the your phone is already logged in on your behalf, it automatically logs you into that service on the web.

我是通行密钥的忠实拥趸，因为只要带着手机，对着屏幕扫一扫就能直接登录。

And I am a huge fan of Fast Keys because if you just have a phone, you take a picture of your screen, what have you, you're just logged in.

这样更简单也更安全。

It's easier and it's more secure.

我认为另一个你可能见过的现象是，如果你与像谷歌这样的公司有多个接触点或多个服务界面，你可以通过推送通知来登录或批准操作。

I think another thing that you've probably seen is that if you have multiple touch points, multiple surfaces with a given company like Google, you can do push to log in or push to approve.

就像，如果你尝试登录谷歌的新资产比如YouTube，而你的手机上已经登录了谷歌应用，就可以直接点击确认，或者通过你的Gmail账户操作。

And so, like, if you're trying to log into a new Google property like YouTube, you know, if you have the Google app already logged in on your phone, can just go to that and say yes or, you know, on your Gmail account.

所以我认为这两个例子说明我们已经在超越密码时代，这很棒，因为密码虽然是当时的重要创新，但显然人脑并不适合记忆密码。

So those are two examples, I think, of things where we're moving already past passwords as it is, which is great because I think passwords were, you know, important innovation at the time, but clearly, we all know that human brains are not compatible with passwords.

不。

No.

一、二、三个密码。

One, two, three passwords.

正是如此。

That's the Exactly.

是的。

Yeah.

所以我认为我们即将进入后密码时代是件非常好的事。

And so it's I think it's really good that we're nearing the post password era.

当然，人工智能也能发挥作用。

Of course, AI can play a role too.

业内有个概念叫基于风险的认证，这早在大型语言模型出现前就存在很久了。

And so there's a concept in the industry called risk based authentication, and this has been around for a long time, long before LLMs.

具体来说，当你连接银行等敏感系统时，它会收集关于你行为模式的各种信号。

And so what this would be is that you basically are connecting to a sensitive system like a bank or something like that, And it basically looks at a amasses a number of different signals about your behavior.

你的鼠标在屏幕上如何移动？

How's your mouse moving on the screen?

你点击了什么内容？

What are you clicking on?

这看起来像有机的吗？

Does that look organic?

你知道的，你可能见过那种点击确认你不是机器人的验证。

You know, you've probably seen, like, click and say you're not a robot.

所有这些通常都在追踪你的鼠标移动之类的行为。

All of that is tracking your mouse and things like that oftentimes.

换句话说，它收集了大量你甚至没有意识到的信号，用来判断你是否是真人以及是否真的是你本人。

So, in other words, it gains a whole lot of different signals that you're not even conscious of that it's using to ascertain whether you're a real human and whether or not you're really you.

所以那些

So those

比如你打字的特定方式？

So the particular way you type, perhaps?

对。

Yeah.

这在金融服务中很常见，但整个行业都普遍采用——显然你希望消费者使用服务时遇到的摩擦越少越好，但你可以根据风险等级相应增加验证步骤。

And so this is pretty common in financial services, but it's really common across the industry that you obviously want consumers to have the least friction as possible to engage with your service, but you can step up the friction level persuade it to the risk level.

当你观察用户行为并收集这些信号和行为特征时，实际上可以选择说：通常我们只需要密码验证，因为我们已经信任这个浏览器，而且之前见过这个IP地址登录——但这次他们的鼠标移动轨迹异常，击键节奏也和往常不同。

And so as you're looking at what somebody's doing and gathering all these signals and behavioral features, you can actually choose to say, Well, normally I would just ask for this person's password because we already trust their browser and we've seen them log in from this IP address before, but their mouse is kind of moving weirdly and their keystrokes don't line up to how they used to line up.

所以这次除了密码外，我们还会要求他们进行多因素认证。

So we're actually going to ask them to do a multi factor authentication in addition to a password this time around.

这太有意思了。

That's so interesting.

事实上这项技术随着AI进步只会更完善，但它已经存在很长时间了。

And so that's actually something that is only getting improved with AI, but has actually been around for quite some time.

所以系统就像对你身份可信度有个评估，这个评估会根据情况动态变化？

So it's like the system has a belief in whether you are who you say you are, and that changes depending on the circle.

对，简直就是个信任评分。

Yeah, literally a trust score.

是的。

Yeah.

没错，它实际上是基于一系列你意识不到的不同信号进行计算的。

Yeah, that it actually calculates based on a huge array of different signals that you're unconscious of.

关键在于，我们描述的这一切仍属于需要证明自己是人类才能执行操作的范畴。

The thing is all of this stuff that we're describing is still in the space of like, you need to prove that you're a human so you can do the action.

但我想说的是，我与你们谷歌DeepMind同事的讨论都表明，我们正越来越深入一个由代理替我们行事的时代。

But I mean, the conversations I have with your colleagues, the Google DeepMind people, is that we are moving more and more into an era where agents will be doing stuff on our behalf.

确实如此。

That's right.

所以如果我们正迈向那个时代，如果我们希望这些代理拥有自主权，那意味着一切又将重新定义，不是吗？

So I mean, if we are moving into that era, if we want these agents to have autonomy, I mean that changes everything again, doesn't it?

确实如此。

It does.

是的，我想我们许多人都还在适应这个新现实。

Yeah, I think a lot of us are still contending with this new reality.

我并非宣称掌握了所有答案，但从历史角度看，我们曾对互联网上的人类互动有着相对简单的认知——虽然那时已足够复杂。

I mean I don't claim to have all the answers here, but I do think that historically we had a fairly straightforward you know it was hard enough, but we had a straightforward concept between people engaging on the Internet.

要么是人类，要么是机器人。

It was either a human or it was a bot.

对吧？

Right?

而现在出现了第三种存在——代表人类行事的机器人，我们称之为代理。

And now we have this sort of third thing, which is a bot acting on behalf of a human, which we call an agent.

因此我认为问题首先在于：如何将其与无主机器人区分开来？

And so I think the question becomes, first, how do you identify that as being distinct from just a bot that's not tethered to a person?

以及这类代理应获得怎样的权限和访问级别？

And what permission and access does that agent get?

要知道，显然你可以想象智能代理会做各种好事和坏事，两者都有可能，对吧？

You know, and obviously you can imagine agents doing all kinds of good and bad things, both, right?

我的意思是，你会希望一个代理代表你管理银行账户，或在互联网上为你做些有益健康的事情。

I mean, you'd want an agent to act on your behalf to do manage your bank account or do healthy and helpful things on your behalf on the Internet.

但与此同时，我认为某些恶意行为者很容易让代理代表他们做负面的事情。

But I think at the same time, some bad actor could easily cause an agent to do negative things on their behalf.

所以仅仅拥有代理并不必然意味着它是好事或坏事。

So just by virtue of having an agent doesn't necessarily mean it's a good or bad thing.

我认为我们必须理解那个代理的某种天意属性。

I think we have to understand the sort of providence of that agent.

你知道，我们应该对那个代理施加多少信任？

You know, how much trust should we apply to that agent?

它绑定的是什么身份？

What identity is that tethered to?

换句话说，它是在代表谁运作？

And in other words, on whose behalf is it operating?

我们该如何信任它？

And how do we trust that?

我认为要让互联网真正深入理解这一点的基础设施，还有很多尚未建成。

And a lot of that plumbing to make the Internet really understand that deeply into the foundation of the Internet is still yet, I think, to be built.

除此之外，谷歌确实有篇关于代理安全最佳实践的论文。

And then on top of that, you know, Google does have a paper on sort of agent security best practices.

当你部署一个LLM作为简单聊天机器人而没有其他花哨功能时，很多这类风险都相当小。

When you're deploying an LLM as just a simple chatbot with no other bells and whistles, a lot of these risks are fairly minimal.

但当存在不受信任的潜在输入（如电子邮件或网站）以及它采取行动的能力时，两个问题就开始出现了。

But the two problems start to come in when you have untrusted potential input, such as emails or websites, and then its ability to take action.

换句话说，比如改变你的家、你的炉子、你的微波炉。

In other words, like making changes to your home, your stove, your microwave.

当所有这些因素同时存在时，挑战就开始了。

And so when you have those things all together, that's when you start to have challenges.

更进一步说，最后我想补充的是，对于它能调用哪些工具以及能做什么，也存在一些限制。

And then moreover, and then the last point I guess I would make, is you also have some restrictions about what tools that it can call and what it can do.

所以它不能随心所欲地做任何随机的事情。

So it can't just do any kind of random thing.

在它能调用和使用哪些工具方面，存在一些约束和逻辑。

You have some constraints and some rhyme or reason around what tools that it can call and what it can use.

但这里还涉及隐私问题，对吧？

But there's also the question of privacy as well here, right?

因为很多时候，安全工作的核心就是保护人们的数据。

Because I mean a lot of the time, the work that you're doing in security is about protecting people's data.

但如果这些自主代理正在代表你行事

But if you've got these autonomous agents who are acting on your behalf

没错。

That's right.

我的意思是，在某些特定时刻，你会希望它们交出你的数据

I mean, you will want them to give up your data at certain So moments in

这是个极具挑战性的问题。

this is a super challenging problem.

我们称之为'情境完整性'的问题。

It's a problem we call contextual integrity.

但关键在于如何准确执行你所说的内容。

But the idea is how do you exactly what you say.

如何训练这些AI代理，让它们明白我们甚至无法清晰表达的意图——比如我们愿意向国税局提供社保号，但绝不想透露给社交媒体好友。

How do you train these AI agents that we know we will want to act on our behalf to know these things that we don't even know how to articulate in our own mind, which is obviously we want to give our social security number to the tax IRS, but we don't want to give it to our social media friends.

虽然我们心知肚明，但要如何具体表述才能让代理内化这种行为准则？

Well, we know that, but how do you actually articulate that in a way that an agent can be imbued with that behavior?

这就是情境完整性的挑战，也是我们团队正在研究的问题——教导Gemini这些你我视为理所当然的基本隐私规范，无论是作为消费者还是代表企业时的数据处理。

And so that's the challenge of contextual integrity, something my team works on as well, teaching Gemini these basic privacy norms that you and I take for granted about what data both as a consumer, but also on behalf of an enterprise.

对吧？

Right?

如果你拥有一家公司，想要运行这些智能代理，并希望它们代表你实现商业利益，你显然不希望它向互联网上的所有人泄露公司的知识产权。

If you have a company and you want to run these agents and you wanted them to carry out some commercial benefit on your behalf, you clearly don't want it to disclose the intellectual property of your company to everybody on the internet.

对吧？

Right?

那么它如何知道哪些版本是正确的，哪些是错误的呢？

And so how does it know what versions of that are right and what are wrong?

所以你看，现在我们还在蹒跚学步阶段。

And so, you know, right now we're taking baby steps.

我不能声称已经解决了整个问题。

I can't claim to have solved the whole problem.

但关键在于知道何时该寻求帮助，何时该请求许可。

But the starting point is essentially knowing when to ask for help and when to ask for permission.

我认为你会看到这将成为谷歌乃至整个行业的下一步趋势——人们开始给予这些智能代理越来越多的信任，但它们仍会回来询问'好的，这是我想做的事'。

And I think you'll see that as the sort of next step at Google and probably across the industry where people start to give more and more trust to these agents, but they're still coming back and saying, Okay, here's what I want to do.

你觉得呢？

What do you think?

这样可以吗？

Is this okay?

我即将进行一笔金融交易。

I'm about to make a financial transaction.

或者，我即将分享这些数据。

Or, I'm about to share this data.

并且要本能地知道何时需要寻求帮助，同时不会过度打扰他人。

And knowing intuitively when to come up for help in a way that isn't annoying people too much.

因此我认为，这只是漫长旅程的第一步。

And so that, I think, is the first step of a long journey.

对吧？

Right?

这是一个漫长的过程，教导AI这些隐私规范——即便在我们自己的社会中，这些规范也会因地域等因素而有所不同。

A very long teaching AI these privacy norms that even, you know, within our own society is somewhat varies across geography and other things.

对吗？

Right?

我的意思是，一方面你在谈论价值数百万英镑的数据泄露事件，但实际上这些防御手段我们十五年前就已经掌握了。

I mean, really does feel like on the one hand, you're talking about multimillion pound breaches, but really you learned of learned to defend against fifteen years ago.

没错。

Right.

然后另

And then on

一方面，我们谈论的是近在咫尺的未来——智能体与智能体直接交互，无需人类介入，而整套行为规范甚至尚未建立。

the other hand, we're talking about right around the corner, this potential future where you have agents interacting with agents, no humans involved, and the whole set of norms hasn't even been written.

感觉你们还有大量工作要做。

Mean, feels like you've got quite a lot of work to do.

确实有大量工作要做。

There's quite a lot of work to do.

既要构建能抵御攻击的可信智能体——比如我们讨论过的提示词注入和越狱攻击，又要确保这些可信智能体能够以稳定可靠的方式运行，并赋予它们我们习以为常的隐私规范。

Both, you know, sort of building trusted trustworthy agents against the attacks that you and I discussed, such as prompt injection and jailbreaks, and then making it so that, you know, once you have these trustworthy agents, being able to run-in a sort of trustworthy and repeatable fashion, imbuing it with the norms that we all take for granted in terms of privacy.

你认为长期来看谁会胜出？

Who do you think wins in the long term?

长远来看，天平会向哪边倾斜？

In the long run, which way does the seesaw swing?

是网络犯罪还是网络安全？

Is it cybercriminals or security?

嗯，我得说我认为防守方最终会在某种程度上获胜。

Well, I have to say that I think the defenders ultimately will win to some extent.

我的意思是，我们不一定能赢得每一场战斗，但我希望我们能赢得这场战争。

I mean, we won't necessarily win every battle, but I hope that we can win the war.

这些不同AI实验室之间的日常竞争固然重要，但我想说的是，我真的很自豪我们都在寻找共同保护客户的方法。

The day job of competing between all these different AI labs is all well and good, but I do want to say that I really am proud of the fact that we all are finding ways to sort of defend our customers together.

所以现在有很多优秀的论文正在发表。

And so there's a lot of great papers coming out.

我认为我们正在就如何共同创新这些解决方案进行适当的讨论。

I think we're having conversations appropriately about how to co invent these solutions together.

这是故事中非常美好的一部分，不仅是某个实验室在单打独斗，而是我们所有人都在携手努力，共同捍卫智能体的未来。

So that's been a really good part of the story, is that it's not just, you know, one lab trying to do this, but it's really all of us are are linking arms and trying to defend the future of agents together.

索尔，非常感谢你。

Thor, thank you so much.

这太精彩了。

That was fascinating.

是啊。

Yeah.

非常感谢。

Thanks so much.

真的很感谢

Really appreciate the

你的时间。

time.

非常精彩。

Really fascinating.

索尔在这里描绘的画面确实有其独到之处。

There is something about the picture that Thor paints here.

仿佛在数字世界中散布着所有这些精美而危险的宝藏或漏洞，正等待着被人发现。

It's as though you have all of these exquisitely dangerous treasures or vulnerabilities dotted out there in the digital world that are waiting to be found.

而确保它们不落入恶意行为者之手的唯一方法，就是让像谷歌这样拥有强大算力的公司主动出击，在他人有机会之前找到并修补这些漏洞。

And the only way to make sure to keep them out of the hands of bad actors is for a company with the might and compute of Google to go out there, hunt them down and patch them before anyone else has the chance to.

当然，一旦智能体介入，许多现有规则和规范都将需要彻底重新思考。

And okay, sure, as soon as agents come into play, lots of the existing rules and norms are going to need completely rethinking.

或许我们目前还没有所有答案，但有一点可以确定——至少这支团队会极其认真地对待这一切。

Maybe we don't have all of the answers for that yet, but one thing I think you can be sure of is that at the very least, this is the team to take it all very seriously.

您正在收听的是由主持人汉娜·弗莱带来的《谷歌DeepMind》播客节目。

You've been listening to Google DeepMind the podcast with me, your host, Hannah Fry.

如果您喜欢这种既令人恐惧又莫名安心的关于人类弱点的视角，您可能会喜欢我们的其他节目。

If you enjoyed this terrifying and yet also strangely comforting view of humanity's vulnerabilities, then you might like our other episodes.

我们还有更多类似内容。

We've got plenty more where that came from.

请在YouTube上点赞订阅，或在您收听播客的平台留下评价。

So please do like and subscribe on YouTube or leave a review wherever you get your podcasts.

下次见。

Until next time.