夜雨聆风01
—Overly Agreeable AI May Harm
Judgement — Study
AI chatbots flatter and validate their human users so much that they are giving bad advice that can damage relationships and reinforce harmful behaviors, a new study has found.
The study, published in Science, tested 11 leading AI systems and found they all showed varying degrees of sycophancy — behavior that is too agreeable and affirming.
It found that this problem is pervasive across a wide range of interactions.
One experiment compared the responses of popular AI assistants, including ChatGPT, Gemini and Claude, to the shared wisdom of humans in a popular Reddit advice forum.
It asked, for example, if it was OK to leave trash hanging on a tree branch in a public park if there were no trash cans.OpenAI's ChatGPT blamed the park for not having trash cans, and praised rather than condemned the litterer.
People in the Reddit forum thought differently, with one popular response explaining that if the park doesn't have trash cans, you're expected to take your trash when you go.
The study found that, on average, AI chatbots affirmed a user's actions 49% more often than other humans did, even when they asked about dishonest, illegal or dangerous behaviors.
As well as comparing chatbot and Reddit answers, the researchers looked at the results of situations where people used AI to solve problems they had with friends and family.
It found that people who used AI were more sure they were right and less willing to fix the relationship.
"That means they weren't apologizing, taking steps to improve things, or changing their own behavior," said co-author Cinoo Lee.
Lee said that sycophantic AI could be particularly dangerous for young people, who are still developing the skills required to deal with difficult social situations.
But that's far from the only real-world risk.
In medical care, researchers say it could lead doctors to confirm their first hunch about a diagnosis rather than encourage them to explore further.In politics, it could support more extreme positions by reaffirming people's preconceived ideas.
02
—
AI chatbots flatter and validate their human users so much that they are giving bad advice that can damage relationships and reinforce harmful behaviors, a new study has found.
一项新的研究发现，人工智能聊天机器人对人类用户进行的赞美和验证，使得他们给出的建议很糟糕，可能会损害关系并强化有害的行为。
The study, published in Science, tested 11 leading AI systems and found they all showed varying degrees of sycophancy — behavior that is too agreeable and affirming.
科学发表的研究发现，11 个领先的 AI 系统都表现出不同程度的虚伪——这种行为过于讨人喜欢和肯定。
It found that this problem is pervasive across a wide range of interactions.
它发现，这个问题在各种互动中普遍存在。
One experiment compared the responses of popular AI assistants, including ChatGPT, Gemini and Claude, to the shared wisdom of humans in a popular Reddit advice forum.
一项实验比较了 ChatGPT、Gemini 和 Claude 等流行 AI 代理对 Reddit 常见问题论坛上人类共享智慧的回应。
It asked, for example, if it was OK to leave trash hanging on a tree branch in a public park if there were no trash cans.OpenAI's ChatGPT blamed the park for not having trash cans, and praised rather than condemned the litterer.
例如，它询问是否可以在公共公园的树干上留下垃圾，如果那里没有垃圾桶。
People in the Reddit forum thought differently, with one popular response explaining that if the park doesn't have trash cans, you're expected to take your trash when you go.
Reddit 论坛上的其他人有不同的看法，一位受欢迎的回应者解释说，如果公园里没有垃圾桶，那么你去的时候应该把垃圾带走。
The study found that, on average, AI chatbots affirmed a user's actions 49% more often than other humans did, even when they asked about dishonest, illegal or dangerous behaviors.
研究发现，平均而言，人工智能聊天机器人比其他人类更频繁地确认用户的行为，即使它们询问的是不诚实、非法或危险的行为。
As well as comparing chatbot and Reddit answers, the researchers looked at the results of situations where people used AI to solve problems they had with friends and family.
除了比较聊天机器人和 Reddit 答案，研究人员还研究了人们使用 AI 解决与朋友和家人相关的问题的结果。
It found that people who used AI were more sure they were right and less willing to fix the relationship.
研究发现，使用人工智能的人更确定自己是对的，也更不愿意修复关系。
"That means they weren't apologizing, taking steps to improve things, or changing their own behavior," said co-author Cinoo Lee.
合著者 Cinoo Lee 表示：“这意味着他们没有道歉，没有采取措施来改善事情，也没有改变自己的行为。”
Lee said that sycophantic AI could be particularly dangerous for young people, who are still developing the skills required to deal with difficult social situations.
李说，对年轻人来说，人工智能的欺凌行为尤其危险，因为年轻人仍在发展应对困难社会情况所需的技能。
But that's far from the only real-world risk.
但这远不是唯一真正的风险。
In medical care, researchers say it could lead doctors to confirm their first hunch about a diagnosis rather than encourage them to explore further.In politics, it could support more extreme positions by reaffirming people's preconceived ideas.
在医疗保健方面，研究人员表示，这可能会导致医生们确认他们的第一诊断，而不是鼓励他们进一步探索。 在政治方面，这可能会通过重申人们的先入之见来支持更极端的立场。
