ARTICLE · 1122539
封面文章:警惕AI利他主义
封面文章:警惕AI利他主义THOUGHT PRECEDES deed as lightning precedes thunder, wrote the poet Heinrich Heine. So it has proved with effective altruism (EA), an ethical doctrine cooked up by philosophers in Oxford that has gone from the ivory tower to the corridors of power with remarkable speed. Effective altruists fill Silicon Valley, staff the world’s leading artificial-intelligence companies—and alarm the Trump administration. “The United States will NEVER be an effective-altruist country,” pledges the Pentagon. The secretary of war brands the philosophy “defective altruism”; others in the administration call it “dystopian”. Take those attacks as a sign of its influence. 诗人海因里希·海涅曾写道：“正如闪电先于雷鸣，思想先于行动。” 这句名言被有效利他主义（EA）证实。有效利他主义是牛津大学哲学家提出的伦理学说，它以惊人的速度从大学象牙塔走向权力竞技场。目前有效利他主义者遍布硅谷，他们任职于全球领先的人工智能公司，并引起了特朗普政府的警觉。五角大楼声称：“美国永远不会成为一个有效利他主义国家。国防部长将这一哲学称为“有缺陷的利他主义”；政府中的其他人则称之这种哲学为“反乌托邦”。这些攻击可以视为有效利他主义的影响力标志。 Some complain that the 21st century has seen little fresh thinking. Into the void have stepped MAGA and Co, making nostalgic populism the century’s big political project. But the century’s big idea is effective altruism. As we report, what began in the 2000s as a geeky way to make charitable giving more cost-effective has changed into a movement that subjects the world to complex, even bizarre, moral calculations. For many of its adherents, the urgent ethical requirement for people who want to do good is to stop artificial intelligence from wiping out humanity. 有人抱怨道21世纪鲜有新思想。它让MAGA们及其支持者——旧民粹主义变成了本世纪重大的政治工程的势力，变得毫无用处。但本世纪最重要的理念是“有效利他主义”。正如我们所报道的那样，这一理念始于21世纪初，最初是一种旨在提高慈善捐赠的效率的极客式慈善捐赠方式，如今已演变成一场将世界置于复杂甚至怪异的道德计算之中的运动。对于其许多追随者来说，那些想要行善的人所面临的紧迫伦理要求是：阻止人工智能毁灭人类。 EA is sociologically important, too. EAs are not numerous, but the idea has swept up a generation of gifted youngsters who have made fortunes in cryptocurrency and AI. As the technology mints more billionaires, effective altruists will channel billions of dollars to causes that will shape society. EAs helped start OpenAI and Anthropic, the leading labs, in the belief that those who understood the dangers of AI should control its development. To a degree, they now do—though, ironically, by creating more labs, and thus competition, they have probably accelerated AI’s progress. 有效利他主义（EA）具有重要社会学意义。虽然有效利他主义者人数不多，但这一理念已经吸引了一代才华横溢的、加密货币和人工智能领域积累了巨额财富的年轻人。随着技术不断催生新贵，有效利他主义者将把数十亿美元投入到改造社会的事业中。他们资助创立了OpenAI和Anthropic等领先实验室，因为他们相信那些理解人工智能危险性的人应该掌控人工智能的发展。在一定程度上，他们确实做到了——尽管讽刺的是他们通过创建更多实验室而加剧竞争，最后可能加速了人工智能的进步。 And all this has now made EA politically important. The Trump administration fears the movement will succeed in throttling AI. It has feuded with Anthropic over the firm’s refusal to let its models be used for some military purposes. Leading EAs acknowledge that AI may prove hugely benign, but nonetheless want to slow its development, to reduce the chance that an unstoppable machine-brainruns amokin pursuit of goals that are “misaligned” with those of humanity. 如今这一切已使有效利他主义者（EA）有了政治上的重要性。特朗普政府担心EA运动将成功扼杀人工智能的发展。特朗普政府曾因Anthropic公司拒绝让其模型用于某些军事目的而与该公司发生争执。处于EA潮头的人承认，人工智能可能被证明极其有益，但他们仍然希望放缓其发展，以降低无法节制的机器大脑在追逐与人类“不一致”的目标中而肆意妄为的概率。 Some critics in America would rather accelerate the technology, to prevent China from winning the race. But when attacking AI doomerism they find it easier toplay the nerd than the ball.They say thateffective altruism is a movement of fanatics that is variously undemocratic, dangerous and cultish. 在美国对EA的批评者认为为了防止中国在AI竞赛中获胜，他们宁愿加速其技术发展。但在攻击人工智能末日论时，批评者发现对付AI怪比对付AI本身容易。批评者认为有效利他主义是一场狂热运动，具有不同程度的不民主、危险和狂热性质。 That is too crude. The movement’s charitable roots are obviously praiseworthy. LikeThe Economist, EAs are committed to empiricism and, especially since they blended with the online “rationalist” community, the application of logical thinking. They hold reason in higher esteem than tradition or norms and will enthusiastically debate any topic, however controversial, in pursuit of the truth. 这种批评太武断了。EA运动的慈善初衷显然值得称赞。与本刊一样，有效利他主义者（EAs）致力于宣扬经验主义，尤其是自从他们与在线“理性主义者”社区融合以来，更加注重逻辑思维的应用。他们将理性置于传统或规范之上，只为追求真相，无论多么有争议，会热情地辩论任何话题。 EAs’ repeated success in predicting the rapid progress of AI has been impressive, too. They have a point about the technology’s risks to humanity. On September 28th OpenAI scrapped plans to release its latest model after it appeared too dangerous in testing. Many recent safety incidents resemble scary scenarios set out on effective-altruist blogs years ago. 有效利他主义者在预测人工智能快速进展方面的屡试不爽也令人印象深刻。他们关于这项技术对人类构成的风险的观点可圈可点。9月28日，OpenAI在测试中发现其最新模型过于危险后，取消了发布计划。许多最近的安全事件与多年前在有效利他主义在博客上描述的可怕情景如出一辙。 Yet the critics are right to warn that effective altruists reach some odd conclusions. Like the utilitarian philosophers before them, EAs boil ethical behaviour down to an optimisation problem. How can people bring about the best outcomes for humanity as a whole? Their calculus downplays local imperatives such as “keep promises” or “look after your family”. What matters is reducing suffering, impartially and everywhere. 然而，EA批评者警示有效利他主义者得出了一些奇怪的结论无疑正确。和之前的功利主义哲学家一样，有效利他主义者将道德行为简化为最优解问题：人们如何为全人类带来最好的结果？他们的计算弱化了诸如“遵守承诺”或“照顾家人”等本地化的道德要求。对他们来说重要的是无差异地减少痛苦。 That leads them to factor a very wide circle of concerns into their moral calculations. It is easy to see how being ethical means helping your neighbour. EAs ask: why not work as hard to help those far away? Early on, the movement directed its army of overachievers to take high-paying jobs in finance and consulting. Their wealth could buy good works, achieving more than if they became charity workers themselves. 这促使他们将广泛的关切纳入道德考量之中。做一个有道德的人意味着要帮助自己的邻居是容易让人理解的。有效利他主义者（EAs）会问：为什么不努力帮助那些遥远的人呢？早期，这一运动将其众多高成就者引导至金融和咨询行业的高薪职位。他们的财富可以用于购买善行，比他们亲自成为慈善工作者能实现更多善举。 But as it has become stronger, EA has become stranger. Some EAs see the moral distinction between humans and other animals as weak, because animals experience suffering. Stopping factory farming and helping shrimp are major concerns. Likewise, they have a counterintuitive approach to time. Those not yet born should be weighed in the balance—and there could be trillions of them. Many EAs are “longtermists” who feel bound to forestall risks in the distant future, even if that starves today’s good causes, such as helping the desperately poor. 但随着运动变得越来越强大，有效利他主义也变得越来越奇怪。一些有效利他主义者认为人类与其他动物之间的道德区别差异很小，因为动物也会经历痛苦。停止工厂化养殖和帮助虾类成为他们的主要关注点。同样，他们对时间持有反直觉的看法。尚未出生的人也应该被纳入道德的考量范围——而且可能达到数万亿计。许多有效利他主义者是“长期主义者”，他们认为必须预防遥远的未来风险，即使它们会削弱今日的善举，比如帮助极度贫困的人群。 Pushed to an extreme, this calculus can lead to a morality that is not just counterintuitive, but repugnant. Some EAs think that insect life is, in aggregate, more valuable than human life, because there are so many insects. Some say it would be better to bring about the existence of trillions of humans whose lives are barely worth living than to have mere billions with genuinely good lives. Others even think the welfare of AIs themselves will one day be of vital importance. They would redirect resources towards future humans or nonhumans, and away from people suffering today. 推到极致，这种计算会导致不仅违反直觉而且令人厌恶的道德观。 一些有效利他主义者认为，从总体上看，昆虫的生命比人类的生命更有价值，因为昆虫的数量多。 有些人认为，创造数万亿个生活勉强值得过的人比创造仅仅数十亿过着真正美好生活的人更好。甚至有人认为，人工智能本身的福祉有一天会变得至关重要。AI会将让资源远离今天正在受苦的人们，而分配给未来的人类或非人类。 EA+AI=p(doom) It is right to revere reason. But moral philosophers from Aristotle to Rawls knew that if an argument leads to intolerable places, it must have a hidden faulty step. EAs’ instinct, though, is to enjoy counterintuitive conclusions, even if they seem perverse. A problem is the margins of error that inevitably accompany effective-altruist arguments. Nobody is truly capable of weighing up moral quandaries involving billions today, let alone gigantic sums stretching into the future, involving uncertain numbers of people, discount rates, the progress of technology, and countless other variables. Another is that extinction is so dire that in a crude moral arithmetic even a small chance of itoverpowerstoday’s flesh-and-blood concerns. 尊重理性无可厚非。但自亚里士多德到罗尔斯的道德哲学家们都知道，如果一个论证导向了无法容忍的结论，那其中必定隐藏着错误步骤。然而，有效利他主义者的本能却是享受那些反直觉的结论，即使它们看起来荒谬。问题之一是有效利他主义论证中不可避免地存在误差。如今，没有人真正能够权衡涉及数十亿人的道德困境，更不用说那些延伸至未来、涉及不确定人数、贴现率、技术进步以及无数其他变量的问题。另一个问题是，灭绝的后果如此严重，以至于在粗略的道德算术中，哪怕只是极小的灭绝风险的重要性也会碾压当下活生生的人类关切。 A third is that in weighing up the interests of billions, you lose sight of the inherent dignity of the individual. Anyone wielding a spreadsheet and grandiose claims about the future of humanity is one step away from anointing themself as a central planner. Moral authority in hand, the rights, welfare and opinions of a neighbour can come to seem unimportant. 第三，在权衡数十亿人的利益时会忽视个体固有的尊严。任何手持电子表格并对人类未来做出宏大叙事的人，离自封为中央计划者仅一步之遥。一旦掌握了道德权威，邻居的权利、福祉和意见就可能显得无关紧要。 The French revolution was supposedly founded on reason; it led to the execution of nuns.Lenin, too, thought that intellectuals who had mastered Marxism understood workers’ interests better than the workers themselves.If you truly believe that you act on behalf of all people, let alone trillions of future ones, you may end up justifying all sorts of atrocities. 法国大革命据称建立在理性之上却给修女带来灭顶之灾。列宁也曾认为，掌握了马克思主义的知识分子比工人自己更了解工人的利益。如果你真的相信你代表所有人，甚至还代表数万亿未来的生命，那么你最终可能会为各种暴行辩护。 Fortunately, effective altruists are hardly about to bring back the guillotine. The fear instead is that controlling AI will eventually bring them great power, and that plenty of EAs in Silicon Valley think that only they can be trusted with the technology. That is an alarming idea.Effective altruists’ argument for worrying about really powerful AIs is that the machines could optimise for a single and fatally wrong objective. Everyone should worry, too, about what happens when a small group of people fall into the same trap. ■ 幸运的是，有效利他主义者几乎不会重新启用断头台。真正的担忧在于，控制人工智能最终将赋予他们巨大的权力，而且硅谷中许多有效利他主义者认为只有他们自己才值得信赖来掌握这项技术。这是令人不安的想法。有效利他主义者之所以担忧超级强大的人工智能，是因为这些机器可能会为一个单一且致命错误的目标进行优化。每个人都应该同样担忧，当一小群人不断对一个单一且致命的错误目标进行优化时时会发生什么。■
有效利他主义是本世纪最大的理念
这个运动值得尊敬，同时非常危险
Effective altruism is this century’s biggest idea
The movement is admirable, but dangerous