TED:How I created OpenClaw, the breakthrough AI agent[为什么人工智能不会变得有意识]

讲演人：Anil Seth     翻译：鹿之

So for centuries, people have fantasized about playing God by creating artificial versions of ourselves. From Mary Shelley’s “Frankenstein” to Hal in Stanley Kubrick’s “2001” and Ava in Alex Garland’s “Ex Machina,” this is a dream reinvented with every breaking wave of technology, and with AI, the wave is a big one.

几个世纪以来，人们一直做着通过创造自己的复制品来扮演上帝的白日梦。从Mary Shelley的《弗兰肯斯坦》到Stanley Kubrick的《2001太空漫游》中的哈尔，再到Alex Garland的《机械姬》中的艾娃，每来一波新技术，这个梦就被重新包装一次。而到了AI这儿，这一浪显得尤为汹涌。

The AI we have is already smart, at least in some ways. But could it ever be conscious? Will a robot ever gaze at a sunset and experience the beautiful colors, the reds and the oranges? Will it feel a sense of beauty or a rush of joy? Or will computers, however smart they get, always remain dark on the inside, always an object and never a subject?

我们现有的AI已经很聪明了，至少在某些方面是这样的。但它能有意识吗？机器人会凝视日落并感受那抹红色与橙色相间的美丽晚霞吗？它会体会到美或一丝喜悦吗？还是说，无论计算机变得多么聪明，其内在始终是黑暗的，永远是一个客观存在的东西而绝非有主体性的存在？

Whether AI can be conscious is one of the most consequential questions we face in our time. If computers can be conscious or sentient or aware, we’d be entering a new era in human history. We’d have new entities that have their own inner lives, new inventions that matter for their own sakes and not only for their effects on us. Conscious AI might suffer the click of a mouse, perhaps in ways we wouldn’t even recognize. And if silicon can be sentient, then maybe our messy flesh-and-blood bodies will soon be superseded by machines that never age and cannot die.

AI能否拥有意识，是我们这个时代所面临的最要命的问题之一。如果计算机能有意识、有感知力、有知觉，我们将进入人类历史的新纪元。我们将有具备自己内心生活的全新实体，拥有为了他们自身利益而非仅仅对我们产生影响的重要的新发明。有意识的AI可能会因为点击一下鼠标或者其他我们无法识别的方式而遭受痛苦。如果硅基材料能有感知力，那么我们这些乱七八糟的血肉之躯将很快就被永不衰老、不会死亡的机器所取代。

Now over the last few years, progress in AI has been simply astonishing. And who knows what’s around the corner. Many experts think that conscious AI is possible. Quite a few think it’s inevitable, and some, some think it’s here already. I think they’re wrong. I want to tell you why and why this matters so much.

在过去的几年里，AI的进步简直令人惊叹，谁也不知道未来会发生什么。许多专家认为有意识的AI是有可能的，很多人认为这是不可避免的，并且有一些人觉得它已经存在了。我认为他们错了，我这就想告诉你们原因，以及为什么这事儿如此重要。

So I’ve been studying brains, minds and consciousness for nearly 30 years, and one thing I’ve learned is that to answer the question, “can AI be conscious,” we need to start by looking within ourselves at the makeup of our own human minds. Now we humans, we tend to see the world in our own terms. We know we’re conscious, and we like to think that we’re intelligent. So we think the two go together. And this is why some people think that consciousness might just glimmer into existence as AI gets smarter and smarter. But consciousness and intelligence are different things. Intelligence is all about doing. It’s solving a crossword puzzle, assembling some furniture, navigating a tricky family situation.

我研究大脑、心智和意识已经将近30年了，我从中学到的一件事是，要回答：“AI能不能有意识？”这个问题的话，我们需要从自身出发找答案，从审视我们人类心智的构造开始。现在我们人类吧，倾向于用自己的方式来理解世界，我们知道自己有意识，并且我们喜欢认为自己是聪明的，所以我认为这两者是相伴相生的。这就是为什么有些人认为意识可能会随着AI越来越聪明而初现端倪。但意识和智力是不同的概念，智力讲的是做事 – 比如解个填字游戏、拼个家具以及怎样处理一个棘手的家庭状况。

Consciousness, on the other hand, it’s all about feeling and being. It’s the difference between normal, wakeful awareness and the oblivion of general anesthesia. It’s the bitter tang of coffee, it’s the warmth of a log fire. The joy of seeing a loved one. Just because consciousness and intelligence go together in us does not mean that they go together in general. The assumption that they do, well, that’s a reflection of our own psychology, not an insight into the nature of reality.

另一方面，意识讲的是“感受”和“存在”。这就是清醒状态下正常意识与全身麻醉后失去意识之间的区别。这便是咖啡的苦涩，是炉火的温暖，是见到爱人的喜悦。不能因为我们的意识和智力是相伴而生的，就觉得它们在哪儿都是绑在一起的。认为它们必然共存的假设，反映的是我们自身的心理，而不是对现实本质的深刻洞察。

Take language models like Claude or GPT, trained on vast quantities of written texts. They reflect back to us an image of ourselves, of our collective, digitized past. We talk about ourselves endlessly, and so do they. We wonder about consciousness and the meaning of it all. And so, it seems, do they. But language models are not conscious. They simulate consciousness. We project consciousness into them in the same way we might project faces into clouds, or even the image of Mother Teresa in a cinnamon bun. I don’t know if you can see that. There she is, Mother Teresa in a cinnamon bun.

以Claude或GPT这样在海量文本的基础上训练而成的语言模型为例，它们映射出来的，其实就是我们自己的样子 – 咱们那些被搬上网的、大家共同的过去。我们无休止地谈论自己，它们也是如此。我们思考关于意识的全部意义，看起来，它们也在做同样的事。但语言模型不具备意识，它们只是在模拟意识。我们把自己的意识感强加给了它们，就像我们会在云朵里看出人脸，甚至在肉桂卷上看出特蕾莎修女的轮廓一样。我不知道你能不能看出来，她就在那儿，肉桂卷上的特蕾莎修女。

We are built to be seduced, like Narcissus, by our own reflections. And so we see ourselves in our algorithms. I’m always struck that nobody really worries whether DeepMind’s AlphaFold is conscious. AlphaFold predicts the structure of proteins rather than words and sentences, but under the hood, it’s not much different from Claude or GPT, algorithms running on silicon and trained on vast reservoirs of data. AlphaFold just doesn’t pull our psychological strings in the same way. So if we think that Claude is conscious, but AlphaFold isn’t, that says more about us than it says about AI.

我们生来就像那喀索斯一样，被自己的倒影所诱惑。因此我们在自己的算法中也看到了自己，我总是感到奇怪的是，似乎没有人真正担心DeepMind的AlphaFold是否有意识。AlphaFold预测的是蛋白质的结构，而不是单词和语句，但要看底层代码，它与Claude或者GPT没有太大区别，都是在硅基上运行算法并是在海量数据储备上训练而成。所以只是AlphaFold没有以同样的方式波动我们的心理之弦罢了。如果我们认为Claude有意识而AlphaFold没有的话，这说明的问题不在 AI，而在我们自己身上。

Narcissus“那喀索斯”（希腊神话里的美少年，迷恋自己在水中的倒影，最后化作水仙花）。

But how can I be so sure those systems, like Claude or GPT, aren’t conscious? Well, nothing is for certain when it comes to consciousness, but the very idea of conscious AI depends on a deeper assumption, a kind of myth, really. And this is the myth that the brain is a computer that just happens to be made of meat rather than metal. Now consciousness in this story is a special algorithm, a collection of computations that just happens to be carried out in the wetware of the brain in us, but which could equally be carried out in silicon, in AI. But the computer is just one in a long line of technological metaphors that we’ve reached for when trying to understand the deep complexity of the brain.

但是，我凭什么如此确信像是Claude或GPT那样的系统并不具备意识呢？这么说吧，但凡涉及意识，就没有什么是确定的。但“有意识的AI”这个观点本身依赖于一个更深层的假设，甚至可以说是一种迷思。这个迷思就是：大脑是一台计算机，只不过恰好由血肉构成，而非金属。照这么说，意识就是一套特殊算法，是一系列的运算过程，而这套过程恰好发生在我们大脑的生物湿件(生物神经系统)中，但同样也可以在硅基材料、AI里实现。但是，“计算机”只是我们为理解大脑的极度复杂性而借用的众多技术隐喻中的其中一种罢了。

wetware“湿件”。相对硬件（hardware）和软件（software）造的词，指生物大脑的物质基础，包括神经、化学物质等（因为大脑是湿乎乎的）即生物神经系统。

One time, the brain was a system of plumbing. Later, it was a telephone exchange. And for the last few decades, it’s been a computer. And this most recent metaphor has been extremely powerful. But it is still a metaphor. And we will always get into trouble when we confuse a metaphor with the thing itself. The map with the territory.

曾经有段时间，大脑被比作一套管道系统；后来，它被认为是电话交换机；而过去这几十年里，它又被说成是一台计算机。这一最近的隐喻影响力极大，但它终究只是个隐喻而已。当我们总是把隐喻与事物本身划等号时准会出问题——就像把地图当成了真实的地形一样。

The map with the territory “地图并非疆域”。源于波兰裔美国哲学家阿尔弗雷德·柯日布斯基（Alfred Korzybski），这句话的核心含义是：我们用于描述、理解或代表现实的事物（地图/概念/语言），绝不等同于现实本身（领土/客体/物自体）。

For one thing, in a real brain, there’s no sharp separation between the mindware and the wetware. Unlike the separation that you get between software and hardware in a computer. And this really matters because in a computer, you can describe and understand everything about an algorithm, whether it’s a language model or a word processor, without worrying about all the silicon shenanigans going on underneath. The computation, the algorithm is all that matters. But for brains, you just cannot separate what they do from what they are. And this means that what they do is unlikely to be a matter of computation of algorithm alone.

首先，在真实的大脑里，“心智软件”与“生物湿件”之间没有清晰的界限。不像计算机软件和硬件分得那么清楚，这一点非常重要，因为在计算机中，无论研究的是语言模型还是文字处理器，都可以把它的算法给描述清楚、弄明白，完全不用担心底层硅芯片的那些小动作，计算、算法才是最重要的。但对于大脑而言，你根本无法将它们的功能与本质割裂开，这也意味着大脑的运作方式不太可能是计算或算法的问题。

Look closely at a brain, at any brain, and it becomes less and less plausible that all that’s going on is just turning some numbers into other numbers in this endless dance of zeros and ones. Yes, there are neural circuits which exchange signals and may do computation, or at least something like it. But there’s so much else that escapes the confines of the digital. Neurotransmitter chemicals course through the brain circuitry, electromagnetic fields sweep through the cortex like weather systems. Even a single neuron is such a beautiful biological machine. A far cry from the simplified, cartoon-like neurons that power today’s AI. The brain is not, or at least not just, a computer made of meat. And so consciousness is very unlikely to be a matter of computation alone. And if this is true, then conscious AI is off the table, at least for AI as we know it today.

仔细观察一个大脑，随便哪个大脑都行，你会发现大脑里发生的一切仅仅是将一些数字转换成另一些数字，在0和1之间无尽循环打转的这种说法越来越说不通。是的，神经回路会传递信号，或许也在进行某些计算，至少是类似计算的东西。但除此之外还有太多东西超出了数字领域的框架。神经递质在大脑回路中流动，电磁场如同天气系统一般掠过大脑皮层。就连一个神经元，都是一台精妙绝伦的生物机器，与驱动当今AI的那些简化、卡通一样的神经元相去甚远。大脑并不是，至少不只是一台用肉构成的计算机。因此，意识极不可能仅仅是计算的产物。如果这点成立，那么有意识的AI就无从谈起，至少对我们今天所知的AI而言是这样。

Let me put it another way. What if we simulated every last detail about the brain in some massive supercomputer? Now if the fine details of the brain do matter for consciousness, well, wouldn’t this be enough for consciousness to happen inside a machine? Well, a computer simulation of a hurricane does not create real wind. A computer simulation of a black hole doesn’t suck the Earth into its algorithmic singularity. Making these simulations more detailed can make them more useful, but it does not make them any more real. We can have a simulation of the brain, and you can make it as detailed as you want. This might make it more useful, but it’s not going to make it any more conscious.

让我换一种说法，如果我们用一台巨大的超级计算机模拟出大脑的每一个细微细节会怎样呢？如果大脑的精细结构确实对意识重要的话，那这么做难道不足以让机器内部产生意识吗？可你想想看，计算机模拟一场飓风并不会真的刮起风来；计算机模拟的黑洞也不会把地球都吸入进它算法里的“奇点”；把模拟做得更细节能让它们变得更有用，却不会让它们变得更真实。我们完全可以模拟大脑，你想做多细都行，这可能会让它更有用，但不会让它变得更有意识。

Now consciousness, has to be said does remain a bit mysterious, but perhaps one reason for this is that we’ve been so constrained by our metaphors, by the idea that it just has to be some kind of information processing. After all, if you think the brain literally is a computer, then what else could it really be? But once we see the brain more clearly for what it really is, many new possibilities arise. And my own view, developed over many years, is that consciousness is intimately connected to our nature as living creatures. Unlike the abstract universe of computation, life is all about materiality. Unlike algorithms, living systems are deeply embedded in flows of energy and matter, and they continually regenerate their own conditions for existence and for persistence over time. I think we can draw a direct line from the molecular furnaces of metabolism, one billion biochemical reactions in every cell, in every second, all the way to the neural circuits that underlie each and every experience that we have, whether it’s the sight of a blue sky or a pang of envy.

现在必须得承认意识仍然有点神秘，其可能的原因之一是我们一直被各种隐喻所束缚，总觉得它肯定是某种信息处理之类的玩意儿。毕竟，如果你打心眼儿里认为大脑实际是一台计算机，那意识除了是信息处理还能是什么？可一旦我们更清晰地认识到大脑的本质，就会冒出更多新的可能性，并且我个人多年来形成的观点是：意识与我们作为生物的本性有着紧密的联系，与抽象、纯计算的领域不同，生命的本质在于物质实在；与算法不同，生命系统是深深嵌在能量与物质的流动之中的，并且它们会持续不断地调整自身生存和延续所需的条件。我觉得我们可以画一条直线 – 从新陈代谢那个分子熔炉（每个细胞里每秒十亿次的生化反应），一路连到那些支撑着我们每一次体验的神经回路上，不管是看见一片蓝天，还是心里头那一下子嫉妒。

Every conscious experience is imbued, however subtly, with a tinge of aliveness, with some core relevance for our future survival prospects. And at the heart of every experience, beneath even emotion, is this simple, shapeless and formless but fundamental feeling of being alive. And in this story, it’s life, not computation, that breathes the fire into the equations of experience. And if this is right, then conscious AI will need to be living AI.

每一次有意识的体验，无论多么微妙，都充满了生命的气息，都与我们未来的生存前景有着某种核心关联。并且每一段体验的核心，甚至在情感之下，都存在着一种简单、无形无状却又深刻的感受 – 活着的感觉。按照我这个说法，是生命，而非计算，赋予了体验真正的生命力和能量。如果这个说法成立，那么拥有意识的AI必将是拥有生命的AI.

breathes the fire into the equations“为…… 注入火种 / 灵魂”。出自霍金的超级畅销书《时间简史》（A Brief History of Time）。

Let me bring things together. First, we’re built to see consciousness where it isn’t, thanks to deep-seated psychological biases that bundle language, intelligence and consciousness together.

我来总结一下。首先，我们天生就容易在不存在意识的地方“看出”意识，这源于我们根深蒂固的心理偏见，我们总是把语言、智力还有意识混为一谈。

Second, the brain is not, or not just, a computer. So consciousness is unlikely to be a matter of computation of algorithm alone. Brains are the kinds of things for which you can’t separate what they do from what they are. And silicon is not up to the job.

其次，大脑并不是，或者说不只是一台计算机。因此意识也不太可能仅仅是光是计算或者算法就能解决的事儿。大脑这类事物的运作方式与自身本质密不可分，而硅基材料根本胜任不了这项工作。

And third, many other things about our biological brains and bodies might matter for consciousness, including a deep connection between consciousness and life.Artificial intelligence is computer software. It is not a living mind. It might give the impression of being conscious, but it is vanishingly unlikely that it actually is.

第三，我们生物性的大脑和身体的许多其他方面或许对意识同样至关重要，包括意识与生命之间的深层联结。人工智能是计算机软件，不是有生命的心智。它可以表现出拥有意识的样子，但真正具备意识的可能性微乎其微。

I want to close by returning to why this matters so much. Take the idea of AI welfare. Now there are already influential groups advocating that AI systems should have their own rights based mainly on the idea that they might be or become conscious. Now if real artificial consciousness is on the way, maybe through some other technology or other pathway, then this is entirely justified. We humans have a terrible track record in our ethical treatment of non-human animals and of other humans, and we don’t want to make the same mistake again. This is one reason why even trying to build conscious AI is a very bad idea.

最后，我想回到这件事为什么如此重要的这个问题上来。就拿“AI福利”这个概念来说，现在已经有一些颇具影响力的组织在主张AI系统应当拥有它们自身的权利，其主要依据就是AI可能已经具备意识，或终将会产生意识。现在如果真正的人工意识出现，也许是通过其他的技术或途径实现，那么这种主张完全合理。我们人类在对待非人类动物乃至对待同类的问题上劣迹斑斑，我们不会想重蹈覆辙。这也是为什么即便只是尝试创造有意识的AI已经是极其糟糕的想法了。

But if conscious AI is just an illusion created by design, as I think it is, then by extending rights to these systems, we’d be sacrificing our ability to control, to regulate them and perhaps even to turn them off and for no good reason at all. And this is one reason why even AI that merely seems to be conscious is very dangerous for our society, too. And unlike real artificial consciousness, conscious-seeming AI is either already here or coming very, very soon.

但如果有意识的AI只是人为设计的假象，反正我是这么看的，那么一旦我们赋予这些系统权利，我们就会丧失对它们的控制能力、监管能力，甚至可能无法关闭它们，而且这一切都是毫无正当理由的。这也是为什么即便是那些看似具有意识的AI，对我们社会来说也极具危险性的一个原因。而且与真正的人工意识不同，看似有意识的AI要么已经出现，要么马上就来了，很快很快。

There are other reasons we should avoid creating AI that seems to be conscious. It makes us more psychologically vulnerable. We might be more likely to do what an AI says if we believe that it really feels for us, that it really understands us, even if what it’s telling us to do is very bad for us.

我们应当避免制造看似有意识的AI是有很多其他原因的，它会让我们在心理上变得更脆弱，如果我们相信AI真的为我们着想，真的理解我们，即使它告诉我们要做的事情对我们非常不利，我们也更有可能按照他说的去做。

And finally, the very idea of conscious AI undermines our human nature. The mirror of AI goes both ways. We see ourselves in our algorithms, but we also see our algorithms in ourselves. And when we do, when we think of the mind as a collection of computations floating free from their bases in biology, we reduce and we diminish what it is to be a living, breathing human being in a real world.

最后，“AI拥有意识”这一概念本身就削弱了我们的人性。AI这边镜子是双向映射的，我们在我的算法中看见自己，但反过来，我们也会在自己身上看到算法的影子。一旦我们这么做了，把心智当成脱离了生物根基，自由飘着的运算合集时，我们就是在贬低和削弱一个在真实世界里活生生的、会呼吸的人的意义。

“Frankenstein,” which Mary Shelley wrote when she was just 19, is often taken as a cautionary tale, a warning against the hubris of bringing something to life, a Promethean sin like stealing fire from the gods. The idea of conscious AI is a new Promethean dream wrapped up in a silicon rapture. And if conscious AI is possible, then so is the prospect of uploading our own conscious minds and floating off to eternity in a silicon cloud of living, or at least existing forever in the pristine circuits of some future supercomputer.

Mary Shelley在她年仅19岁时写了《弗兰肯斯坦》，这本书常被视作一则警示寓言 – 告诫人们不要妄自尊大地赋予某个东西生命，这是一种如同普罗米修斯式盗取天火般僭越之罪。有意识的AI的构想正是一场裹在硅基狂热里的全新的普罗米修斯式的幻梦。如果AI真能拥有意识，那么上传我们自身的意识、在硅基云端永生，或是在未来超级计算机的原始纯净电路中永恒存在，也将成为可能。

Promethean sin“普罗米修斯式的罪行”。指的是一种为了人类文明的福祉或个体对知识的渴求，敢于僭越神（或自然、权威）的领域，并甘愿承受随之而来的惩罚与诅咒的行为。

Now the seductive power of this vision — of being at such a pivotal point in the history of life on Earth — I suppose it’s understandable. And back in the real world, talk of conscious AI does other things too — it conjures this sense of technological wonder and magic which might keep share prices aloft and regulators at bay.

现在说什么“身处地球生命史上如此关键的转折点”这种愿景的诱惑力我觉得也能理解吧。但回到现实世界，谈论有意识的AI还有别的用处：它会唤起一种技术上的神奇感和魔法感，既能让股价一直飘在高处，又能让监管机构无从下手。

But we should resist. The sacrament of the algorithm is most likely an empty dream, delivering not post-human Paradise, but silicon oblivion. We need a different story. One in which we’re more part of nature, not apart from it, with consciousness more closely tied to living flesh and blood, not to the dead sand of silicon.

但是我们应该抵制这种幻想。算法的圣餐极有可能只是一场虚妄的幻想，它不会带来什么后人类天堂，只会带来硅基的虚无。我们需要些不同的版本，一个我们更多地融入自然，而不是脱离它，意识与鲜活的血肉之躯紧密相连，而非依附于冰冷死寂的硅基材料。

AI might claim the prize of intelligence, at least in some ways, but consciousness, consciousness remains ours to celebrate and to share with other living creatures.

AI或许可以在某些领域摘得“智能”这块奖牌，但意识呢，意识这东西，还是属于咱们的，值得咱们庆祝，也值得咱们跟其他生灵一起去分享。

So let’s not sell our minds so easily to our machine creations. If we do, we not only overestimate them, we underestimate ourselves.

所以别轻易把我们的心智拱手“卖”给自己创造的机器。如果我们真这么做，我们不只是高看了它们，更是小瞧了我们自己。

Thank you very much.

非常感谢。