关于人工智能进步的思考（2025年12月）——我的博客文章音频版

我不明白，为什么有些人设定的时间表如此短暂，却同时对在大语言模型基础上扩展强化学习持乐观态度。

I'm confused why some people have super short timelines, yet at the same time are bullish on scaling up reinforcement learning atop LLMs.

如果我们真的接近于一个类似人类的学习者，那么这种基于可验证结果进行训练的整个方法注定会失败。

If we're actually close to a human like learner, then this whole approach of training on verifiable outcomes is doomed.

目前，实验室正试图在训练中期将大量技能嵌入这些模型中。

Now, currently the labs are trying to bake in a bunch of skills into these models through mid training.

有一整条供应链公司正在构建强化学习环境，教模型如何浏览网页或使用Excel构建金融模型。

There's an entire supply chain of companies that are building RL environments, which teach the model how to navigate a web browser or use Excel to build financial models.

现在，这些模型要么很快就会以自我指导的方式在工作中学习，从而使所有这些免费的嵌入变得毫无意义；要么它们不会，这意味着通用人工智能并不迫在眉睫。

Now, either these models will soon learn on the job in a self directed way, which will make all this free baking pointless, or they won't, which means that AGI is not imminent.

人类不需要经历专门的培训阶段，去反复练习未来工作中可能用到的每一种软件。

Humans don't have to go through the special training phase where they need to rehearse every single piece of software that they might ever need to use on the job.

巴伦·米利奇最近在一篇博客文章中对此提出了一个有趣的观点。

Baron Milledge made an interesting point about this in a recent blog post he wrote.

他写道：'当我们看到前沿模型在各种基准测试中取得进步时，我们不仅应该考虑规模的扩大和巧妙的机器学习研究思路，还应该想到数十亿美元被用于支付博士、医学博士及其他专家，让他们编写问题并提供针对这些特定能力的示例答案和推理。'

He writes, quote, when we see frontier models improving at various benchmarks, we should think not just about the increased scale and the clever ML research ideas, but the billions of dollars that are paid to PhDs, MDs, and other experts to write questions and provide example answers and reasoning targeting these precise capabilities.

你可以在机器人领域最明显地看到这种矛盾。

You can see this tension most vividly in robotics.

从某种根本意义上说，机器人技术是一个算法问题，而不是硬件或数据问题。

In some fundamental sense, robotics is an algorithms problem, not a hardware or a data problem.

人类只需极少的训练，就能学会操作当前的硬件来完成有用的工作。

With very little training, a human can learn how to teleoperate current hardware to do useful work.

因此，如果我们真的拥有类似人类的学习者，机器人技术在很大程度上就已经解决了。

So if we actually had a human like learner, robotics would be, in large part, a solved problem.

但因为我们没有这样的学习者，就必须走进上千个家庭，反复练习上百万次，学习如何捡盘子或叠衣服。

But the fact that we don't have such a learner makes it necessary to go out into a thousand different homes and practice a million times on how to pick up dishes or fold laundry.

现在，我听到一些认为未来五年内将迎来爆发的人提出的反驳意见是，我们进行这些繁琐的强化学习，是为了构建一个超人的AI研究员。

Now one counter argument I've heard from the people who think we're gonna have a takeoff within the next five years is that we have to do all this kludgy RL in service of building a superhuman AI researcher.

然后，成千上万份这种自动化AI研究员就能自行找出如何从经验中实现稳健高效的learning。

And then the million copies of this automated ILIA can go figure out how to solve robust and efficient learning from experience.

这让我想起了那个老笑话：每笔销售都在亏钱，但我们靠销量来弥补。

This just gives me the vibes of that old joke, we're losing money on every sale, but we'll make it up in volume.

某种意义上，这个自动化研究者将能找出通用人工智能的算法，而这个问题人类已经苦苦钻研了近半个世纪，却连儿童都具备的基本学习能力都没有。

Somehow this automated researcher is gonna figure out the algorithm for AGI, which is a problem that humans have been banging their head against for the better half of a century, while not having the basic learning capabilities that children have.

我觉得这极其不可能。

I find it super implausible.

此外，即使你相信这一点，它也无法解释各大实验室为何采用可验证奖励的强化学习方法。

Besides, even if that's what you believe, it doesn't describe how the labs are approaching reinforcement learning from verifiable reward.

你不需要在自动化ILIA之前，预先内置其制作PowerPoint幻灯片的顾问技能。

You don't need to pre bake in a consultant skill at crafting PowerPoint slides in order to automate Ilia.

因此，显然实验室的行动暗示了一种世界观：这些模型在泛化能力和在职学习方面将继续表现不佳，因此必须提前将我们希望具有经济价值的技能嵌入这些模型中。

So clearly, the lab's actions hint at a worldview where these models will continue to fare poorly at generalization and on the job learning, thus making it necessary to build in the skills that we hope will be economically useful beforehand into these models.

你还可以提出另一个反驳观点：即使模型能在工作中学会这些技能，但在训练阶段一次性内置这些技能，远比为每个用户和每家公司反复学习要高效得多。

Another counterargument you can make is that even if the model could learn these skills on the job, it is just so much more efficient to build in these skills once during trading rather than again and again for each user and each company.

当然，直接内置对浏览器和终端等常用工具的熟练使用能力，是非常合理的。

Look, makes a ton of sense to just bake in fluency with common tools like browsers and terminals.

事实上，AGI的一个关键优势正是这种在多个副本间共享知识的更强能力。

And indeed, one of the key advantages that AGIs will have is this greater capacity to share knowledge across copies.

但人们严重低估了大多数工作所需的企业和情境特定技能。

But people are really underrating how much company and context specific skills are required to do most jobs.

目前还没有一种稳健高效的方式让AI掌握这些技能。

And there just isn't currently a robust, efficient way for AIs to pick up these skills.

最近我参加了一场晚宴，有位AI研究员和一位生物学家在场，结果发现这位生物学家的工作周期非常长。

Was recently at a dinner with an AI researcher and a biologist, and it turned out the biologist had long timelines.

于是我们问她为什么会有这么长的时间周期。

And so we were asking about why she had these long timelines.

然后她说，最近实验室工作的一部分是观察切片，判断切片中的那个点究竟是巨噬细胞，还是只是看起来像巨噬细胞。

And then she said, you know, one part of work recently in the lab has involved looking at slides and deciding if the dot in that slide is actually a macrophage or just looks like a macrophage.

而那位AI研究员，正如你可能预料的那样，回应说：图像分类是深度学习的经典问题。

And the AI researcher, as you might anticipate, responded, look, image classification is a textbook deep learning problem.

这正是我们可以训练这些模型去完成的典型任务。

This is death center in the kind of thing that we could train these models to do.

我觉得这场对话非常有趣，因为它凸显了我和那些预期未来几年内将出现变革性经济影响的人之间的一个关键分歧。

And I thought this is a very interesting exchange because it illustrated a key crux between me and the people who expect transformative economic impact within the next few years.

人类工作者之所以有价值，正是因为我们不需要为他们工作的每一个细微部分都构建繁琐的训练流程。

Human workers are valuable precisely because we don't need to build in the schleppy training bloops for every single small part of their job.

考虑到这个实验室制备载玻片的独特方式，再为下一个实验室特定的微任务构建另一个训练循环，这种做法并不具有净生产力。

It's not net productive to build a custom training pipeline to identify what macrophages look like, given the specific way that this lab prepares slides, and then another training loop for the next lab specific microtask, and so on.

你真正需要的是一个能够通过语义反馈或自我导向的经验学习，并像人类一样进行泛化的AI。

What you actually need is an AI that can learn from semantic feedback or from self directed experience, and then generalize the way a human does.

每天，你都必须完成100件需要判断力、情境意识以及在工作中习得的技能和背景知识的事情。

Every day, you have to do 100 things that require judgment, situational awareness, and skills and context that are learned on the job.

这些任务不仅在不同人之间不同，甚至对同一个人而言，每天也都不一样。

These tasks differ not just across different people, but even from one day to the next for the same person.

仅通过内置一组预定义技能，就不可能自动化哪怕一个工作岗位，更不用说所有岗位了。

It is not possible to automate even a single job by just baking in a predefined set of skills, let alone all the jobs.

事实上，我认为人们严重低估了真正的人工通用智能（AGI）将带来的巨大影响，因为他们只是在想象当前模式的更多延伸。

In fact, I think people are really underestimating how big a deal actual AGI will be because they are just imagining more of this current regime.

他们没有考虑到服务器上可能出现数十亿个类人智能体，这些智能体可以复制并融合所有学到的经验。

They're not thinking about billions of human like intelligences on a server, which can copy and merge all the learnings.

而且明确地说，我预计在未来十年或二十年内会出现真正类似大脑的智能，这简直太疯狂了。

And to be clear, I expect this, which is to say I expect actual brain like intelligences within the next decade or two, which is pretty fucking crazy.

有时人们会说，AI目前在各公司广泛部署并已在编码之外创造大量价值的原因是，技术需要很长时间才能普及。

Sometimes people will say that the reason that AIs are more widely deployed right now across firms and already providing lots of value outside of coding is that technology takes a long time to diffuse.

我认为这是在逃避现实。

And I think this is COPE.

我认为人们正用这种逃避现实的说法来掩盖一个事实：这些模型根本缺乏实现广泛经济价值所需的能力建。

I think people are using this COPE to gloss over the fact that these models just lack the capabilities that are necessary for broad economic value.

如果这些模型真的像服务器上的真人一样，它们的普及速度会快得惊人。

If these models actually were like humans on a server, they'd diffuse incredibly quickly.

事实上，它们比普通人类员工更容易集成和上手。

In fact, they'd be so much easier to integrate and onboard than a normal human employee is.

它们可以在几分钟内读完你所有的Slack消息并立即上手，还能立刻吸收其他AI员工的所有技能。

They could read your entire Slack and drive within minutes, and they could immediately distill all the skills that your other AI employees have.

此外，人类的招聘市场非常像‘柠檬市场’，很难在事前分辨谁是优秀人才，而雇佣到一个糟糕的人代价极高。

Plus, the hiring market for humans is very much like a lemons market, where it's hard to tell who the good people are beforehand, and then hiring somebody who turns out to be bad is very costly.

如果你只是启动另一个经过验证的HAI模型实例，你就不会面临或担心这种动态。

This is just not a dynamic that you would have to face or worry about if you're just spinning up another instance of a vetted HAI model.

因此，出于这些原因，我认为将AI劳动力融入企业，比雇佣人类要容易得多。

So for these reasons, I expect it's going to be much easier to diffuse AI labor into firms than it is to hire a person.

公司一直在不断招聘员工。

And companies hire people all the time.

如果这些模型的能力真的达到HEI水平，人们会愿意每年花费数万亿美元购买这些模型生成的令牌。

If the capabilities were actually at HEI level, people would be willing to spend trillions of dollars a year buying tokens that these models produce.

全球的知识工作者每年累计获得数万亿美元的工资。

Knowledge workers across the world cumulatively earn tens of trillions of dollars a year in wages.

而目前实验室的收入与这一数字相差几个数量级，原因在于这些模型的能力远未达到人类知识工作者的水平。

And the reason that labs are orders of magnitude off this figure right now is that the models are nowhere near as capable as human knowledge workers.

你可能会想：为什么标准突然变成了实验室，而每年的收入却达到了数万亿美元？

Now you might be like, look, how can the standard have suddenly become labs after tens of trillions of dollars of revenue a year?

对吧？

Right?

直到最近，人们还在说，这些模型会推理吗？

Like until recently, people were saying, these models reason?

这些模型有常识吗？

Do these models have common sense?

它们只是在进行模式识别吗？

Are they just doing pattern recognition?

显然，AI乐观派有理由批评AI悲观派不断移动这些目标。

And obviously, AI bulls are right to criticize AI bears for repeatedly moving these goalposts.

这通常是非常公平的。

And this is very often fair.

很容易低估过去十年AI所取得的进步。

It's easy to underestimate the progress that AI has made over the last decade.

但一定程度的目标转移实际上是合理的。

But some amount of GoPro shifting is actually justified.

如果你在2020年给我看Gemini 3，我会确信它能自动化一半的知识工作。

If you showed me Gemini three in 2020, I would have been certain that it could automate half of knowledge work.

因此，我们不断解决那些曾被认为是通向AGI的充分瓶颈。

And so we keep solving what we thought were the sufficient bottlenecks to AGI.

我们拥有了具有通用理解能力的模型。

We have models that have general understanding.

它们具备少样本学习能力。

They have few shot learning.

它们具备推理能力。

They have reasoning.

然而，我们仍然没有实现AGI。

And yet we still don't have AGI.

那么，面对这种现象，理性的反应是什么？

So what is a rational response to observing this?

我认为，完全有理由观察到这一点并说：实际上，智能和劳动所包含的内容比我之前意识到的要多得多。

I think it's totally reasonable to look at this and say, oh, actually, there's much more to intelligence and labor than I previously realized.

尽管我们已经非常接近AGI，并且在许多方面已经超越了我过去对AGI的定义，但模型公司尚未实现AGI本应带来的数万亿美元收入，这清楚地表明，我过去对AGI的定义过于狭窄。

And while we're really close and in many ways have surpassed what I would have previously defined as AGI in the past, the fact that model companies are not making the trillions of dollars in revenue that would be implied by AGI clearly reveals that my previous definition of AGI was too narrow.

我预计这种情况在未来还会继续发生。

And I expect this to keep happening into the future.

我预计到2030年，实验室将在持续学习这一我关注的问题上取得重大进展，模型每年将产生数百亿美元的收入。

I expect that by 2030, the labs will have made significant progress on my hobby horse of continual learning, and the models will be earning hundreds of billions of dollars in revenue a year.

但它们还不会自动化所有知识工作。

But they won't have automated all knowledge work.

我会说：看吧。

And I'll be like, look.

我们取得了很大进展，但还没有实现通用人工智能。

We made a lot of progress, but we haven't hit AGI yet.

我们还需要这些其他能力。

We also need these other capabilities.

我们需要这些模型具备x、y和z能力。

We need x, y, and z capabilities in these models.

模型的进步速度符合短期预测者的预期，但实用性提升的速度却符合长期预测者的预期。

Models keep getting more impressive at the rate that the short timelines people predict, but more useful at the rate that the long timelines predict.

值得思考的是，我们到底在扩展什么？

It's worth asking, what are we scaling?

在预训练阶段，我们在多个数量级的计算量增长中，观察到了损失函数极其清晰且普遍的改善趋势。

With pre training, we had this extremely clean and general trend in improvement in loss across multiple orders of magnitude in compute.

尽管这种改善遵循幂律，其强度远不如指数增长。

Albeit this was on a power law, which is as weak as exponential growth is strong.

但人们正试图将预训练规模扩展所带来的声望——这种声望几乎像宇宙物理定律一样可预测——用来为强化学习从可验证奖励中获得的乐观预测辩护，而我们目前对后者根本没有公开的明确趋势。

But people are trying to launder the prestige that three training scaling has, which is almost as predictable as the physical law of the universe, to justify bullish predictions about reinforcement learning from verifiable reward, for which we have no well but publicly known trend.

当有胆识的研究者试图从有限的公开数据点中推断出潜在影响时，他们得出的结果往往相当悲观。

And when intrepid researchers do try to piece together the implications from scarce public data points, they get pretty bearish results.

例如，托比·博格发表了一篇精彩的文章，巧妙地将不同O系列基准测试的结果联系起来。

For example, Toby Borg has a great post where he cleverly connects the dots between the different O series benchmarks.

这让他得出结论：‘我们需要大约百万倍的强化学习计算量提升，才能获得类似单个GPT级别的性能提升。’

And this suggested to him that, quote, we need something like a million x scale up in total RL compute to give a boost similar to a single GPT level.

结束引述。

End quote.

因此，人们花了大量时间讨论软件奇点的可能性，即AI模型将编写代码来生成更智能的后续系统。

So people have spent a lot of time talking about the possibility of a software in the singularity, where AI models will write the code that generates a smarter successor system.

或者软件加硬件奇点，即AI还会改进其后续系统的计算硬件。

Or a software plus hardware singularity, where AIs also improve their successor's computing hardware.

然而，所有这些情景都忽略了我认为在持续学习基础上进一步提升的主要驱动力。

However, all these scenarios neglect what I think will be the main driver of further improvements atop continual learning.

再想想人类是如何在任何事情上变得更有能力的。

Again, think about how humans become more capable at anything.

这主要来自于在相关领域的经验。

It's mostly from experience in the relevant domain.

在对话中，巴伦·米德吉提出了一个有趣的建议，认为未来可能会是持续学习代理各自外出从事不同工作并创造价值。

Over conversation, Baron Milledge made this interesting suggestion that the future might look like continual learning agents who are all going out and they're doing different jobs and they're generating value.

然后它们将所有学到的知识带回集体智慧模型，该模型会对这些代理进行某种形式的蒸馏。

And then they're bringing back all their learnings to the hive mind model, which does some kind of bash distillation on all of these agents.

这些代理本身可以非常专业化，包含卡帕西所称的认知核心，以及与它们被部署执行的任务相关的知识和技能。

The agents themselves could be quite specialized, containing what Carpathi called the cognitive core plus knowledge and skills relevant to the job they're being deployed to do.

解决持续学习不会是一次性完成的成就。

Solving continual learning won't be a singular one and done achievement.

相反，它会像解决上下文学习一样。

Instead, it will feel like solving in context learning.

早在2020年，GPT-3就已经展示了上下文学习的强大能力。

Now GPT-three already demonstrated in context learning could be very powerful in 2020.

它的上下文学习能力如此显著，以至于GPT-3论文的标题是《语言模型是少样本学习者》。

Its in context learning capabilities were so remarkable the title of the GPT-three paper was Language Models Are Few Shot Learners.

但当然，当GPT-3问世时，我们并没有真正解决上下文学习的问题。

But of course, we didn't solve in context learning when GPT-three came out.

事实上，从理解能力到上下文长度，仍然有大量进展有待实现。

And indeed, there's still plenty of progress that still has to be made, from comprehension to context length.

我预计持续学习也会经历类似的演进过程。

I expect a similar progression with continual learning.

明年，实验室可能会发布一种他们称之为持续学习的技术，而这实际上将是迈向持续学习的一步进展。

Labs will probably release something next year, which they call continual learning, and which will in fact count as progress towards continual learning.

但要在工作中达到人类水平的学习能力，可能还需要五到十年才能完善。

But human level on the job learning may take another five to ten years to iron out.

这就是为什么我不认为，第一个破解持续学习的模型在广泛部署和能力提升后，会带来爆发式的进步。

This is why I don't expect some kind of runaway gains from the first model that cracks continual learning, that's getting more and more widely deployed and capable.

如果你能突然完全解决持续学习问题，那确实可能就像萨提亚在播客中谈到这个可能性时所说的那样，‘比赛结束，胜负已定’。

If you had fully solved continual learning drop out of nowhere, then sure, it might be game set match, as Satya put it on the podcast when I asked him about this possibility.

但那很可能不会发生。

But that's probably not what's gonna happen.

相反，某个实验室将率先找到解决这个问题的初步突破口，随后通过探索这一功能，人们会逐渐明白其具体实现方式，其他实验室很快就会复制这一突破并稍作改进。

Instead, some lab is gonna figure out how to get some initial traction on this problem, and then playing around with this feature will make it clear how it was implemented, and then other labs will soon replicate the breakthrough and improve it slightly.

此外，我本来就认为，这些模型公司之间的竞争会一直非常激烈。

Besides, I just have some prior that the competition will stay pretty fierce between all these model companies.

而且，从以往的经验来看，所有那些所谓的飞轮效应——无论是用户在聊天中的参与度、合成数据，还是其他什么——都几乎没有减弱模型公司之间日益加剧的竞争。

And informed by the observation that all these previous supposed flywheel, that's user engagement on chat or synthetic data or whatever, have done very little to diminish the greater and greater competition between model companies.

每隔一个月左右，三大模型公司就会轮流登上领奖台，而其他竞争者也并没有落后太多。

Every month or so, the big three model companies will rotate around the podium, and the other competitors are not that far behind.

似乎存在某种力量——可能是人才挖角，可能是旧金山的谣言传播，或者是正常的逆向工程——到目前为止，这些因素已经中和了任何一个实验室可能拥有的任何巨大优势。

There seems to be some force, and this is potentially talent poaching, it's potentially the rumor mill in SF, or just normal reverse engineering, which has so far neutralized any runaway advantage that a single lab might have had.

这是我对最初发布在我的博客 dwarcash.com 上的一篇随笔的朗读。

This was a narration of an essay that I originally released on my blog at dwarcash.com.

我将会发布更多随笔。

I'm gonna be publishing a lot more essays.

我发现，在接受采访之前，写随笔确实有助于理清我的思路。

I found it's actually quite helpful in ironing out my thoughts before interviews.

如果你想及时了解这些内容，可以订阅 dwarcash.com。

If want you to stay up to date with those, can subscribe at dwarcash.com.

否则，我们下次播客再见。

Otherwise, I'll see you for the next podcast.

干杯。

Cheers.