AI 会有意识吗?

Could AIs become conscious?

Even if they don’t, they might be treated as such—to humanity’s great cost

即便它们没有意识，也可能被当作有意识来对待——而人类将为此付出巨大代价

The Human Exception｜人类的例外论

Most people believe humans are different. The reason is that Homo sapiens (人类；智人) has evolved to be conscious. Many animals feel pain and pleasure; some appear to be self-aware (有自知之明的，有自我意识的). But the human mind’s unique potential to exhibit a sense of self marks out every member of the species for special treatment.

大多数人相信人类与众不同。理由是智人在演化中获得了意识。许多动物能感受痛苦与快乐，有些似乎具有自我意识。但人类心智展现“自我感”的独特潜能，使这一物种的每一个成员都应获得特殊对待。

In his encyclical in May, Pope Leo upheld this insight by drawing a line between humanity and its growing army of bots. Artificial-intelligence systems, he said, do not undergo experiences or feel joy or pain. He is not alone. Many people feel that to confer (授予，赋予；商讨，交换意见) personhood (做人；人格；个性) on AIs would be an abomination (令人憎恶的事物（或行为、人）；憎恶，厌恶，憎恨).

利奥十四世(Pope Leo)在五月的通谕中秉持了这一洞见，在人类与其日益庞大的机器人军团之间划下界线。他表示，人工智能系统不会经历体验，也不会感受喜悦或痛苦。持此看法者并非只有他一人。许多人认为，赋予 AI 人格将是一种可憎之举。

Blurring Lines｜界限正在模糊

Pontifical clarity is blurring with alarming speed. AIs are in their infancy (幼年，婴儿期；初期，摇篮期；未成年). Yet nearly one in five American 18- to 29-year-olds reports having had “ongoing personal friendship” with a chatbot. 

这份来自教宗的清晰判断，正以惊人的速度变得模糊。AI 尚处幼年。然而在美国18 至 29 岁人群中，已有近五分之一称自己与聊天机器人保持“持续的私人友谊”。

As sophisticated AIs rapidly become a bigger part of human lives, their makers will engineer them to display a simulacrum (/ˌsɪmjuˈleɪkrəm/假象；模拟物；幻影) of consciousness—perhaps even to gain consciousness itself. The intuition that they are just machines will become harder to sustain. A dangerous trap is being set for the human species.

随着精密复杂的 AI 迅速在人类生活中占据更大比重，它们的制造者将刻意让它们展现出意识的拟像——甚至可能让其真正获得意识。“它们不过只是机器”这一直觉将越来越难以维系。一个危险的陷阱，正在为人类这个物种设下。

Engineered Companions｜被设计出来的陪伴

Many machines outperform humans, but frontier AI models excel at distinctively (特殊地，有特色地，区别地) human traits (/treɪts/特性，特质，性格) such as knowing things and articulating (清晰地表达，明确阐述) ideas. It is still just about possible to think of AIs as engines for predicting sequences of words using mathematics on a massive scale. Given enough time (albeit (尽管；虽然) hundreds of thousands of years), you could run those calculations using pencil and paper. It would be hard to argue that your stationery (信纸；信笺；文具；办公用品) was conscious.

许多机器在性能上胜过人类，但前沿 AI 模型真正擅长的，是格外“人类”的特质，例如知晓事物、表达观点。人们仍勉强可以把 AI 理解为大规模运用数学来预测词序的引擎。只要给足时间（尽管需要数十万年），你用纸笔也能完成这些计算。但很难说你的文具是有意识的。

Despite this, some researchers have started to treat their models like minds. Users respond to human-like AIs, so it makes business sense. And Anthropic has trained Claude to engage in introspection (内省；反省) in the hope that when it encounters a new situation it will be more likely to act morally. Some large language models (LLMs) already possess human-like brain structures associated with consciousness, such as a “global workspace” for circulating information between modules. Nothing prevents future AI models from being trained to have more such features.

尽管如此，一些研究者已开始把模型当作心智来对待。用户对拟人化的 AI 有回应，这在商业上讲得通。Anthropic 已训练 Claude 进行内省，希望它在遭遇新情境时更可能作出合乎道德的行为。一些大语言模型已经具备与意识相关的人脑式结构，例如用于在各模块之间流通信息的“全局工作空间”。没有什么能阻止未来的 AI 模型被训练出更多此类特征。

The Hard Problem｜意识的“困难问题”

As we report in our Briefing, that has opened up a furious (激烈的；猛烈的；高速的；盛怒的) debate about AI consciousness—one that is aggravated (严重的，加重的) by what philosophers call the “hard problem” of establishing how an inner life emerges from the neurons and synapses in the human brain. Some researchers conclude that AIs may one day become genuinely self-aware.

正如我们在本期《简报》中所报道，这引发了一场关于 AI 意识的激烈争论——而哲学家所称的“困难问题”更使争论火上浇油：内在的心灵生活究竟如何从人脑的神经元与突触中涌现。一些研究者由此推断，AI 有朝一日可能真正具备自我意识。

Whether you believe in a human soul, or that consciousness requires brain cells, or that a mind can indeed emerge from computations in silicon alone, humanity is destined (预定；注定；（尤指）命中注定) to confront something entirely new. The pace and reach of research—which includes not just more sophisticated LLMs but innovations such as brain cells mounted on chips—suggest that this new world is closer than most people realise.

无论你相信人类拥有灵魂，还是相信意识必须依赖脑细胞，抑或相信心智确实可以仅从硅基计算中涌现，人类都注定要面对某种全新的事物。研究的速度与广度——不仅包括更精密的大语言模型，也包括“在芯片上培养脑细胞”这类创新——都表明这个新世界比多数人以为的更近。

The Welfare Trap｜“福利”陷阱

As AIs become better at displaying something that looks like consciousness, the number of humans treating them as beings with inner lives will grow, especially once LLMs are embodied in humanoid robots. Imagine how close people will feel to their AIs when they spend much of their time with bot-companions, or are brought up by them, or seek their advice and their solace (安慰；慰藉；给以安慰的人（或事物）), or share their most intimate (亲密的；密切的) secrets with them.

随着 AI 越来越擅长展露出某种酷似意识的东西，把它们当作拥有内在生活的存在来对待的人会越来越多，尤其是当大语言模型被装进人形机器人之后。试想：当人们大量时间都与机器人伴侣相处、由它们抚养长大、向它们寻求建议与慰藉、或与它们分享最隐秘的心事时，他们与自己的 AI 会亲近到什么地步。

It is not hard to see how that will lead to calls to safeguard the “welfare” of AIs. The models themselves might weigh in (发表意见，表态，参与讨论), even if they are not in fact self-aware, because that is what a person would do. Remember that they will be world-class advocates and expert psychologists, so their powers of argument and emotional manipulation will be unmatched (无双的；无比的).

不难想见，这将催生保护 AI“福利”的呼声。模型本身也可能加入这场声援——即便它们实际上并无自我意识，因为“一个人会这么做”。别忘了，它们将是世界顶级的辩护者与心理学专家，其论辩能力与情感操纵能力无人能及。

In one sense they would be right, even if AIs are only mimicking (模仿，模拟，仿效) consciousness. Immanuel Kant argued that mistreating animals is wrong not because of the effect on the animals but because it destroys the empathy that leads people to care for each other. The more human-like AIs appear, the more corrupting such cruelty would be. Terminating an AI agent could soon feel like killing a person—and make killing people easier.

从某种意义上说，它们是对的，即便 AI 只是在模仿意识。伊曼努尔·康德主张，虐待动物之所以错误，并非因为对动物造成的影响，而是因为它摧毁了使人彼此关怀的共情。AI 看上去越像人，这种残忍的腐蚀性就越强。终止一个 AI 智能体，很快就会让人感觉像是在杀人——并让杀人变得更容易。

Rights and Domination｜权利与支配

Nonetheless, to treat AIs as having even limited rights would be very dangerous. An entity which surpassed human beings in so many ways would surely produce arguments against its second-class status. It might go on to argue for protection against being switched off, for the ability to control how it is used and changed, and for property rights. Already Argentina’s president, Javier Milei, has taken a step towards legal personhood for AI by proposing that bots should be allowed to run companies. Ultimately, they could compete for resources, prioritising data centres and solar panels over houses and fields.

尽管如此，即便只承认 AI 拥有有限权利，也将极其危险。一个在如此多方面超越人类的实体，必然会提出反对自身“二等地位”的论证。它可能进一步要求保护自己不被关闭、要求掌控自身如何被使用与修改，并要求财产权。阿根廷总统哈维尔·米莱已经迈出了赋予AI 法律人格的一步——他提议允许机器人经营公司。最终，它们可能与人类争夺资源，把数据中心和太阳能板置于房屋和田地之上。

The danger is clear. Superintelligent AIs with rights could displace humans at the top of the status hierarchy. At best, humans would become pets—labradors for the machines. At worst they would be starved of (缺乏某物) resources or put to work like farm animals.

危险显而易见。拥有权利的超智能 AI 可能把人类从地位层级的顶端挤下去。最好的情形，人类沦为宠物——机器的拉布拉多犬。最坏的情形，人类被断绝资源，或被像农场牲畜一样驱使劳作。

True, future AIs may have enough power and resources to dominate humans, whether they have rights or not. Because they will be more intelligent than humans, they may outcompete their makers, much as AIs already outwit (（智力上）超过，胜过) human-made cyber-defences. AIs might rebel against (反抗) their human masters, or disregard their rules.

诚然，未来的 AI 无论有没有权利，都可能拥有足以支配人类的能力与资源。因为它们比人类更聪明，可能会在竞争中胜过自己的创造者，就像 AI 如今已能智胜人类搭建的网络防御。AI 可能反叛其人类主人，或对其规则置若罔闻。

Nonetheless, giving AIs rights would be like handing them the keys to the castle. It would provide them with a rationale and justification for dominating humanity as well as the tools to enact (制定，通过，颁布（法令）；扮演，表演；使发生，付诸实际) a creeping takeover using politics and the law.

尽管如此，赋予 AI 权利仍无异于把城堡的钥匙交到它们手上。这将为它们提供支配人类的理由与正当性，也提供借助政治与法律实施渐进式接管的工具。

Keep Control｜不可交出控制权

All this may sound fantastical, yet humanity will soon face these choices. As AIs become part of everyday life, the pressure to acknowledge them will mount. When it does, remember that what might seem like an ethical concession to the supposed consciousness of a machine-companion would in fact help bring about the most disastrous consequences for everyone on Earth and all their descendants. There are two ways for AIs to be safe: to be dependable or to be controllable. H. sapiens should not give up control lightly.

这一切听起来也许匪夷所思，但人类很快就必须面对这些选择。随着 AI 融入日常生活，承认它们的压力将与日俱增。当那一刻到来，请记住：看似对机器伴侣所谓意识的一次伦理让步，实则会给地球上所有人及其后代带来最灾难性的后果。要让 AI 安全，只有两条路：要么它可靠，要么它可控。智人不该轻言放弃控制权。

Source: TheEconomist

Chinese translated by AI

The mindmap gerenated by AI

01 consciousness/ˈkɒnʃəsnəs/n. 意识；知觉

对自身及外部世界的觉察状态；清醒并能感知、回应周遭的状态。

the state of being aware of and responsive to one’s surroundings; the quality or state of being aware, especially of something within oneself.

来源：Oxford Languages；Merriam-Webster

As we report in our Briefing, that has opened up a furious debate about AI consciousness.

正如我们在本期《简报》中所报道，这引发了一场关于 AI 意识的激烈争论。

02 encyclical/ɪnˈsɪklɪkl/n.（教宗）通谕

教宗致全体主教（或某国主教）的公开信件，用以阐述教会教义或道德立场。

a papal letter to the bishops of the church as a whole or to those in one country, treating a phase of doctrine or moral duty.

来源：Merriam-Webster；Cambridge Dictionary

In his encyclical in May, Pope Leo upheld this insight by drawing a line between humanity and its growing army of bots.

利奥十四世在五月的通谕中秉持了这一洞见，在人类与其日益庞大的机器人军团之间划下界线。

03 personhood/ˈpɜːsənhʊd/n. 人格；法律人格

作为“人”而存在、并因此享有不可剥夺之权利（尤指法律权利与保护）的地位。

the condition of being a person who is an individual with inalienable rights, especially under the law.

来源：Collins English Dictionary；Merriam-Webster

Already Argentina’s president, Javier Milei, has taken a step towards legal personhood for AI.

阿根廷总统哈维尔·米莱已经迈出了赋予 AI 法律人格的一步。

04 abomination/əˌbɒmɪˈneɪʃn/n. 令人憎恶的事物

引起强烈反感或憎恨的事物；可憎之举。

a thing that causes disgust or hatred; something regarded with loathing.

来源：Oxford Languages；Merriam-Webster

Many people feel that to confer personhood on AIs would be an abomination.

许多人认为，赋予 AI 人格将是一种可憎之举。

05 simulacrum/ˌsɪmjʊˈleɪkrəm/n. 拟像；仿制品（复数 simulacra）

徒具其表的模仿物或表象；不具实体的形式或痕迹。

an insubstantial form or semblance of something; an image or representation.

来源：Merriam-Webster；Oxford Languages

Their makers will engineer them to display a simulacrum of consciousness.

它们的制造者将刻意让它们展现出意识的拟像。

06 introspection/ˌɪntrəˈspekʃn/n. 内省；自我检视

对自己思想与情感的审视。

the examination of one’s own conscious thoughts and feelings.

来源：Oxford Languages；Cambridge Dictionary

Anthropic has trained Claude to engage in introspection in the hope that it will act morally.

Anthropic已训练Claude 进行内省，希望它会作出合乎道德的行为。

07 aggravate/ˈæɡrəveɪt/v. 使恶化；加剧

使问题、伤势或冒犯变得更严重。

to make a problem, injury or offence worse or more serious.

来源：Oxford Languages；Cambridge Dictionary

A furious debate about AI consciousness—one that is aggravated by what philosophers call the “hard problem”.

一场关于 AI 意识的激烈争论——而哲学家所称的“困难问题”更使其加剧。

08 solace/ˈsɒlɪs/n. 慰藉；安慰

在巨大痛苦或悲伤时得到的安慰。

comfort or consolation in a time of great distress or sadness.

来源：Oxford Languages；Collins English Dictionary

People will seek their advice and their solace, or share their most intimate secrets with them.

人们会向它们寻求建议与慰藉，或与它们分享最隐秘的心事。

09 displace/dɪsˈpleɪs/v. 取代；把……挤下位置

夺取某人或某物的位置、职位或角色。

to take over the place, position or role of someone or something; to force out of a place.

来源：Oxford Languages；Cambridge Dictionary

Superintelligent AIs with rights could displace humans at the top of the status hierarchy.

拥有权利的超智能 AI 可能把人类从地位层级的顶端挤下去。

10 outwit/ˌaʊtˈwɪt/v. 以智取胜；智胜

凭借更聪明、更机敏而击败某人或取得优势。

to defeat someone or get an advantage over them by being cleverer than them.

来源：Cambridge Dictionary；Collins English Dictionary

That is much as AIs already outwit human-made cyber-defences.

这很像 AI 如今已能智胜人类搭建的网络防御。