夜雨聆风So for centuries, people have fantasized about playing God by creating artificial versions of ourselves.
几个世纪以来，人们一直在幻想着扮演上帝，创造出我们自己的人造版本。
From Mary Shelley's "Frankenstein" to Hal in Stanley Kubrick's "2001" and Ava in Alex Garland's "Ex Machina," this is a dream reinvented with every breaking wave of technology, and with AI, the wave is a big one.
从玛丽·雪莱的《弗兰肯斯坦》，到斯坦利·库布里克《2001太空漫游》中的哈尔，再到亚力克斯·嘉兰《机械姬》中的艾娃，每一波突破性的科技浪潮，都会重塑这个梦想，而人工智能，正是其中最汹涌的一波。
The AI we have is already smart, at least in some ways. But could it ever be conscious?
我们现有的AI已经很聪明了，至少在某些方面是这样。但它真的会产生意识吗？
Will a robot ever gaze at a sunset and experience the beautiful colors, the reds and the oranges? Will it feel a sense of beauty or a rush of joy?
机器人会凝视日落，感受那些绚烂色彩——红色与橙色吗？它会感受到美，或是一阵喜悦吗？
Or will computers, however smart they get, always remain dark on the inside, always an object and never a subject?
还是说，无论计算机变得多聪明，它的内在永远是一片黑暗，永远只是客体，而非主体？
Whether AI can be conscious is one of the most consequential questions we face in our time.
AI能否拥有意识，是我们这个时代面临的最重要问题之一。
If computers can be conscious or sentient or aware, we'd be entering a new era in human history.
如果计算机能够有意识、有感知、有觉知，我们将迈入人类历史的新纪元。
We'd have new entities that have their own inner lives, new inventions that matter for their own sakes and not only for their effects on us.
我们将拥有拥有自身内心世界的全新存在，这些新创造物，其存在本身便有意义，而不仅仅是因为它们对我们产生影响。
Conscious AI might suffer the click of a mouse, perhaps in ways we wouldn't even recognize.
拥有意识的AI，可能会因一次鼠标点击而感到痛苦，以我们甚至无法察觉的方式。
And if silicon can be sentient, then maybe our messy flesh-and-blood bodies will soon be superseded by machines that never age and cannot die.
如果硅基也能拥有感知，那么我们这具不完美的血肉之躯，或许很快就会被永不衰老、不会死亡的机器所取代。
Now over the last few years, progress in AI has been simply astonishing. And who knows what's around the corner.
过去几年，AI的进步堪称惊人。没人知道未来还会发生什么。
Many experts think conscious AI is possible. Quite a few think it's inevitable, and some think it's here already.
许多专家认为，有意识的AI是可能实现的。不少人认为这是必然趋势，还有一些人认为，它已经出现了。
I think they're wrong. I want to tell you why and why this matters so much.
我认为他们错了。我想告诉你原因，以及这件事为何如此重要。
So I've been studying brains, minds and consciousness for nearly 30 years, and one thing I've learned is that to answer the question, "can AI be conscious," we need to start by looking within ourselves at the makeup of our own human minds.
我研究大脑、心智与意识已近30年，我明白的一件事是，要回答“AI能否有意识”这个问题，我们必须先审视自身，审视人类心智的构造。
Now we humans, we tend to see the world in our own terms. We know we’re conscious, and we like to think that we’re intelligent. So we think the two go together.
我们人类，习惯以自身视角看待世界。我们知道自己有意识，也愿意相信自己是智慧的，于是我们认为，二者必然相伴而生。
And this is why some people think that consciousness might just glimmer into existence as AI gets smarter and smarter.
这就是为什么有人认为，随着AI越来越聪明，意识或许会自然而然地浮现。
But consciousness and intelligence are different things.
但意识与智慧，根本是两回事。
Intelligence is all about doing. It's solving a crossword puzzle, assembling some furniture, navigating a tricky family situation.
智慧关乎“行动”。是解填字游戏、组装家具、应对复杂的家庭局面。
Consciousness, on the other hand, it's all about feeling and being.
而意识，关乎“感受”与“存在”。
It's the difference between normal, wakeful awareness and the oblivion of general anesthesia. It's the bitter tang of coffee, it's the warmth of a log fire. The joy of seeing a loved one.
它是清醒觉知与全麻无意识之间的差别。是咖啡的苦涩、炉火的温暖、见到爱人的喜悦。
Just because consciousness and intelligence go together in us does not mean that they go together in general.
意识与智慧在我们身上共存，并不意味着它们在万物中都必然共存。
The assumption that they do, well, that's a reflection of our own psychology, not an insight into the nature of reality.
这种假设，只是我们自身心理的投射，而非对现实本质的洞察。
Take language models like Claude or GPT, trained on vast quantities of written texts. They reflect back to us an image of ourselves, of our collective, digitized past.
以Claude或GPT这类语言模型为例，它们由海量文本训练而成，向我们映照出我们自己的模样，映照出我们集体的、数字化的过去。
We talk about ourselves endlessly, and so do they. We wonder about consciousness and the meaning of it all. And so, it seems, do they.
我们无休止地谈论自己，它们也一样。我们思考意识及其意义，它们看起来也在思考。
But language models are not conscious. They simulate consciousness.
但语言模型并没有意识。它们只是在模拟意识。
We project consciousness into them in the same way we might project faces into clouds, or even the image of Mother Teresa in a cinnamon bun.
我们把意识投射到它们身上，就像我们在云朵里看出人脸，甚至在肉桂卷上看出特蕾莎修女的轮廓。
I don't know if you can see that. There she is, Mother Teresa in a cinnamon bun.
不知道你们能不能看出来。看，肉桂卷上的特蕾莎修女。
We are built to be seduced, like Narcissus, by our own reflections. And so we see ourselves in our algorithms.
我们天生就会像纳西索斯一样，被自己的倒影迷惑。于是我们在算法中看到了自己。
I'm always struck that nobody really worries whether DeepMind's AlphaFold is conscious.
我一直很惊讶，几乎没人担心DeepMind的AlphaFold是否有意识。
AlphaFold predicts the structure of proteins rather than words and sentences, but under the hood, it's not much different from Claude or GPT, algorithms running on silicon and trained on vast reservoirs of data.
AlphaFold预测蛋白质结构，而非生成文字句子，但本质上，它和Claude、GPT没有太大区别，都是运行在硅基芯片上、由海量数据训练的算法。
AlphaFold just doesn't pull our psychological strings in the same way.
只是AlphaFold没有触动我们的心理心弦。
So if we think that Claude is conscious, but AlphaFold isn't, that says more about us than it says about AI.
所以如果我们认为Claude有意识，而AlphaFold没有，这更多反映的是我们自己，而非AI。
But how can I be so sure those systems, like Claude or GPT, aren't conscious?
但我怎么能如此确定，Claude、GPT这类系统没有意识？
Well, nothing is for certain when it comes to consciousness, but the very idea of conscious AI depends on a deeper assumption, a kind of myth, really.
关于意识，没有什么是绝对确定的，但有意识AI的想法，建立在一个更深层的假设上——甚至可以说是一种迷思。
And this is the myth that the brain is a computer that just happens to be made of meat rather than metal.
这个迷思就是：大脑是一台计算机，只不过由血肉而非金属构成。
Now consciousness in this story is a special algorithm, a collection of computations that just happens to be carried out in the wetware of the brain in us, but which could equally be carried out in silicon, in AI.
在这个叙事里，意识是一种特殊算法，是一系列运算，恰好由我们大脑的“生物硬件”执行，但同样也能在硅基、在AI中运行。
But the computer is just one in a long line of technological metaphors that we've reached for when trying to understand the deep complexity of the brain.
但计算机只是我们用来理解大脑复杂性的，众多科技比喻中的一个。
One time, the brain was a system of plumbing. Later, it was a telephone exchange. And for the last few decades, it's been a computer.
曾经，大脑被比作管道系统。后来，它是电话交换机。过去几十年，它成了计算机。
And this most recent metaphor has been extremely powerful. But it is still a metaphor.
这个最新的比喻极具影响力。但它依然只是比喻。
And we will always get into trouble when we confuse a metaphor with the thing itself. The map with the territory.
当我们把比喻等同于事物本身，就一定会出问题。就像把地图当成了真实领土。
For one thing, in a real brain, there's no sharp separation between the mindware and the wetware. Unlike the separation that you get between software and hardware in a computer.
首先，在真实的大脑中，心智活动与生理结构没有绝对界限。这和计算机里软件与硬件截然分开完全不同。
And this really matters because in a computer, you can describe and understand everything about an algorithm, whether it's a language model or a word processor, without worrying about all the silicon shenanigans going on underneath.
这一点至关重要，因为在计算机里，你可以完整描述和理解一个算法，无论它是语言模型还是文字处理软件，完全不用管底层硅芯片的运作细节。
The computation, the algorithm is all that matters.
运算、算法，才是关键。
But for brains, you just cannot separate what they do from what they are.
但对大脑而言，你无法将它的功能与它的本质分开。
And this means that what they do is unlikely to be a matter of computation of algorithm alone.
这意味着，大脑的运作绝不仅仅是算法与计算。
Look closely at a brain, at any brain, and it becomes less and less plausible that all that's going on is just turning some numbers into other numbers in this endless dance of zeros and ones.
仔细观察任何一个大脑，你会越来越觉得，把一切都归结为数字转换、0和1的无尽翻转，是多么站不住脚。
Yes, there are neural circuits which exchange signals and may do computation, or at least something like it.
的确，神经回路会传递信号、进行类似计算的活动。
But there's so much else that escapes the confines of the digital.
但还有太多东西，超出了数字的范畴。
Neurotransmitter chemicals course through the brain circuitry, electromagnetic fields sweep through the cortex like weather systems.
神经递质在大脑回路中流动，电磁场像天气系统一样席卷大脑皮层。
Even a single neuron is such a beautiful biological machine. A far cry from the simplified, cartoon-like neurons that power today's AI.
哪怕一个神经元，都是精妙无比的生物机器。与支撑当今AI的简化、卡通化神经元模型相去甚远。
The brain is not, or at least not just, a computer made of meat.
大脑不是，至少不只是一台“血肉计算机”。
And so consciousness is very unlikely to be a matter of computation alone.
因此，意识绝不可能只是计算的产物。
And if this is true, then conscious AI is off the table, at least for AI as we know it today.
如果这一点成立，那么有意识的AI就不可能实现，至少对我们现在所知的AI而言是这样。
Let me put it another way.
换个说法。
What if we simulated every last detail about the brain in some massive supercomputer?
如果我们在超级计算机里，模拟出大脑的每一个细节呢？
Now if the fine details of the brain do matter for consciousness, well, wouldn't this be enough for consciousness to happen inside a machine?
如果大脑的精细结构对意识至关重要，那这难道不足以让机器产生意识吗？
Well, a computer simulation of a hurricane does not create real wind.
计算机模拟飓风，并不会产生真正的风。
A computer simulation of a black hole doesn’t suck the Earth into its algorithmic singularity.
计算机模拟黑洞，不会把地球吸入算法奇点。
Making these simulations more detailed can make them more useful, but it does not make them any more real.
把模拟做得更精细，只会让它更有用，但不会让它变得更真实。
We can have a simulation of the brain, and you can make it as detailed as you want. This might make it more useful, but it's not going to make it any more conscious.
我们可以模拟大脑，想多精细就多精细，这或许让它更有用，但绝不会让它更有意识。
Now consciousness, has to be said does remain a bit mysterious, but perhaps one reason for this is that we've been so constrained by our metaphors, by the idea that it just has to be some kind of information processing.
不得不说，意识依然有些神秘，但原因之一，或许是我们被各种比喻束缚，被“意识一定是某种信息处理”的想法困住。
After all, if you think the brain literally is a computer, then what else could it really be?
毕竟，如果你认为大脑就是一台计算机，那它还能是什么？
But once we see the brain more clearly for what it really is, many new possibilities arise.
但一旦我们看清大脑的真实本质，全新的可能性就会出现。
And my own view, developed over many years, is that consciousness is intimately connected to our nature as living creatures.
我多年研究得出的观点是：意识与我们作为生命体的本质紧密相连。
Unlike the abstract universe of computation, life is all about materiality.
与抽象的计算世界不同，生命的核心是物质性。
Unlike algorithms, living systems are deeply embedded in flows of energy and matter, and they continually regenerate their own conditions for existence and for persistence over time.
与算法不同，生命系统深深嵌入能量与物质的流动中，它们不断自我更新，维持自身的存在与延续。
I think we can draw a direct line from the molecular furnaces of metabolism, one billion biochemical reactions in every cell, in every second, all the way to the neural circuits that underlie each and every experience that we have, whether it's the sight of a blue sky or a pang of envy.
我认为，我们可以画出一条直线：一端是新陈代谢的分子熔炉——每个细胞每秒发生十亿次生化反应，一直延伸到支撑我们每一次体验的神经回路，无论是看见蓝天，还是一阵嫉妒的刺痛。
Every conscious experience is imbued, however subtly, with a tinge of aliveness, with some core relevance for our future survival prospects.
每一次有意识的体验，无论多么细微，都带着生命的质感，都与我们未来的生存息息相关。
And at the heart of every experience, beneath even emotion, is this simple, shapeless and formless but fundamental feeling of being alive.
在每一次体验的核心、甚至在情绪之下，是一种简单、无形却根本的“活着的感受”。
And in this story, it's life, not computation, that breathes the fire into the equations of experience.
在这个叙事里，是生命，而非计算，为体验注入了灵魂。
And if this is right, then conscious AI will need to be living AI.
如果这是对的，那么有意识的AI，必须是“活着的AI”。
Let me bring things together.
我来总结一下。
First, we're built to see consciousness where it isn't, thanks to deep-seated psychological biases that bundle language, intelligence and consciousness together.
第一，我们天生会在没有意识的地方看到意识，这源于根深蒂固的心理偏见，把语言、智慧和意识捆绑在一起。
Second, the brain is not, or not just, a computer. So consciousness is unlikely to be a matter of computation of algorithm alone.
第二，大脑不是，或不只是一台计算机。因此，意识不可能只是算法计算。
Brains are the kinds of things for which you can't separate what they do from what they are. And silicon is not up to the job.
大脑的功能与本质无法分割。硅基做不到这一点。
Third, many other things about our biological brains and bodies might matter for consciousness, including a deep connection between consciousness and life.
第三，我们生物大脑与身体的诸多特性，都对意识至关重要，包括意识与生命之间的深层联结。
Artificial intelligence is computer software. It is not a living mind.
人工智能是计算机软件，不是活的心智。
It might give the impression of being conscious, but it is vanishingly unlikely that it actually is.
它可能给人有意识的错觉，但它真正拥有意识的可能性微乎其微。
I want to close by returning to why this matters so much.
最后，我想回到这件事为何如此重要。
Take the idea of AI welfare.
以AI权益为例。
Now there are already influential groups advocating that AI systems should have their own rights based mainly on the idea that they might be or become conscious.
如今已有颇具影响力的团体主张，AI系统应拥有自身权利，主要依据是它们可能拥有或即将拥有意识。
Now if real artificial consciousness is on the way, maybe through some other technology or other pathway, then this is entirely justified.
如果真正的人工意识即将到来，或许通过其他技术或路径，那么这种主张完全合理。
We humans have a terrible track record in our ethical treatment of non-human animals and of other humans, and we don't want to make the same mistake again.
人类在伦理对待非人类动物与同类方面，记录糟糕，我们不想再犯同样的错误。
This is one reason why even trying to build conscious AI is a very bad idea.
这也是为什么，哪怕尝试制造有意识AI，都是一个糟糕的想法。
But if conscious AI is just an illusion created by design, as I think it is, then by extending rights to these systems, we’d be sacrificing our ability to control, to regulate them and perhaps even to turn them off and for no good reason at all.
但如果有意识的AI只是刻意制造的幻象——正如我所认为的，那么赋予这些系统权利，我们将牺牲控制、监管它们的能力，甚至无缘无故地失去关闭它们的权力。
And this is one reason why even AI that merely seems to be conscious is very dangerous for our society, too.
这也是为什么，哪怕只是“看似有意识”的AI，对社会也非常危险。
And unlike real artificial consciousness, conscious-seeming AI is either already here or coming very, very soon.
与真正的人工意识不同，“看似有意识”的AI要么已经存在，要么即将到来。
There are other reasons we should avoid creating AI that seems to be conscious.
我们还应避免制造看似有意识的AI，另有原因。
It makes us more psychologically vulnerable.
它会让我们在心理上更脆弱。
We might be more likely to do what an AI says if we believe that it really feels for us, that it really understands us, even if what it's telling us to do is very bad for us.
如果我们相信它真的在乎我们、理解我们，我们就更可能听从AI的指令，即便它让我们做的事对自己有害。
And finally, the very idea of conscious AI undermines our human nature.
最后，“有意识AI”这个概念本身，会消解我们的人性。
The mirror of AI goes both ways.
AI这面镜子是双向的。
We see ourselves in our algorithms, but we also see our algorithms in ourselves.
我们在算法中看到自己，也在自己身上看到算法。
And when we do, when we think of the mind as a collection of computations floating free from their bases in biology, we reduce and we diminish what it is to be a living, breathing human being in a real world.
当我们把心智看作脱离生物基础的一堆运算时，我们就贬低了作为现实世界中活生生的人的意义。
"Frankenstein," which Mary Shelley wrote when she was just 19, is often taken as a cautionary tale, a warning against the hubris of bringing something to life, a Promethean sin like stealing fire from the gods.
玛丽·雪莱19岁写下的《弗兰肯斯坦》，常被当作一则警示寓言，警告人们不要傲慢地创造生命，犯下如同盗取天火的普罗米修斯式狂妄。
The idea of conscious AI is a new Promethean dream wrapped up in a silicon rapture.
有意识AI的想法，是包裹在硅基狂热中的新普罗米修斯之梦。
And if conscious AI is possible, then so is the prospect of uploading our own conscious minds and floating off to eternity in a silicon cloud of living, or at least existing forever in the pristine circuits of some future supercomputer.
如果有意识AI真的可能，那么上传我们的意识、在硅基云端获得永恒，或是在未来超级计算机的纯净电路中永生，也将成为可能。
Now the seductive power of this vision -- of being at such a pivotal point in the history of life on Earth -- I suppose it's understandable.
这种愿景的诱惑力——身处地球生命史上如此关键的时刻——我想是可以理解的。
And back in the real world, talk of conscious AI does other things too -- it conjures this sense of technological wonder and magic which might keep share prices aloft and regulators at bay.
回到现实世界，谈论有意识AI还有其他作用——它营造出科技奇迹与魔力的氛围，推高股价，让监管却步。
But we should resist.
但我们应当抵制。
The sacrament of the algorithm is most likely an empty dream, delivering not post-human Paradise, but silicon oblivion.
将算法奉为神圣，大概率只是一场空梦，它带来的不是后人类天堂，而是硅基的虚无。
We need a different story.
我们需要另一个故事。
One in which we're more part of nature, not apart from it, with consciousness more closely tied to living flesh and blood, not to the dead sand of silicon.
在这个故事里，我们是自然的一部分，而非脱离自然，意识与鲜活的血肉紧密相连，而非死寂的硅砂。
AI might claim the prize of intelligence, at least in some ways, but consciousness, consciousness remains ours to celebrate and to share with other living creatures.
AI或许能在某些方面赢得智慧的桂冠，但意识，依然属于我们，值得我们与所有生命共同珍视与歌颂。
So let's not sell our minds so easily to our machine creations.
所以，不要轻易把我们的心智出卖给机器造物。
If we do, we not only overestimate them, we underestimate ourselves.
如果我们这样做，不仅是高估了它们，更是低估了我们自己。
