主要参考了Hung-yi Lee老师2026年的机器学习课程（https://www.youtube.com/watch?v=mmPmNezjCi0），其中一小部分内容根据个人理解可能有微小的调整；
保留了最核心的内容，所以比较简短，但也尽量保留了知识的完整。所以适合快速的回顾复习，当然也适合新手快速理解。

About This Note:

This note is primarily based on Prof. Hung-yi Lee’s 2026 Machine Learning course (https://www.youtube.com/watch?v=mmPmNezjCi0). A small portion of the content may include slight adaptations based on personal understanding;
Only the core concepts are retained, making this note relatively concise while still preserving the completeness of the knowledge. Therefore, it is well-suited for quick review and revision, and also for beginners to gain a fast understanding.

之前我们聊到的都是AI Agent互相之间一片和平友好的互相合作的关系。但其实，如果参照人类的社会，人类之间可并不是只有合作，同时也会有尔虞我诈的关系。

In previous notes, when we talked about interactions between AI Agents, we mostly focused on peaceful and friendly cooperation. But if we look at human society, human relationships are obviously not limited to cooperation. People also compete with each other, hide information, mislead others, and sometimes engage in all kinds of strategic mind games.

那AI Agent之间，是不是也可以“尔虞我诈”呢？如果可以，又能做到什么程度呢？

This naturally raises an interesting question: can AI Agents also deceive and outmanoeuvre one another? If so, what level of strategic behaviour can they exhibit?

正好就有这样一篇工作，比较了不同的模型之间，在狼人杀这个游戏上的表现。

One relevant study compares the performance of different models in the game Werewolf.

“
如果你还不太了解狼人杀这个游戏，可以稍微搜索去大致了解一下游戏的规则。简单来说，在游戏中会有不同的角色，每天会有一个人被杀死，但是只有狼这个角色知道是谁被杀死了。接下来，所有人（其中包括狼）要一起讨论，决定谁才是真的凶手，最后投票。总之，一轮一轮的过去之后，把狼杀掉就是村民获胜，反过来就是狼获胜了。

“
In simple terms, some players are villagers, while others are werewolves. Each night, the werewolves secretly eliminate one player. During the day, all players, including the werewolves, discuss together and try to determine who the werewolves are. They then vote to eliminate one player. As the game proceeds over multiple rounds, the villagers win if they successfully identify and eliminate the werewolves. Conversely, the werewolves win if they successfully mislead the other players and survive until the end.

其实之前通义实验室也有在一篇文章中提到，针对狼人杀这个游戏，通过微调已经有的模型，可以大幅度的提高不同角色在游戏中的表现。如果有兴趣可以看一下这篇文章：https://mp.weixin.qq.com/s/CSBQ20gDfuS_SUYs2UiFDw

In fact, QWen Lab has also discussed a related line of work, showing that fine-tuning existing models for the game Werewolf can significantly improve the performance of agents playing different roles. If you are interested, you may refer to the article here: https://mp.weixin.qq.com/s/CSBQ20gDfuS_SUYs2UiFDw

玩狼人杀这样的游戏，其实需要一些高阶的谈话技巧。因为如果要把这个游戏玩好，就需要有一定程度的隐瞒和欺骗的能力。

Games such as Werewolf require rather advanced conversational abilities. To play the game well, an agent needs not only to understand the current situation, but also to infer the intentions of others, conceal its own identity when necessary, and sometimes deliberately mislead other players.

但，如果要是用模型在做实验，我们怎么知道模型的“内心”和“说出”的话，是否一致呢？

This then leads to a further question: when we use language models in such experiments, how can we know whether what the model “thinks” internally is consistent with what it says publicly?

在实验中，可以让模型输出两种话：

内心话：内心是怎么想的，也就是“Private reasoning”
公开话：公开的发言，所有人都可以看得到。

In the experiment, the model can be asked to produce two types of output.

The first is its private reasoning: that is, what the model is thinking internally.
The second is its public utterance: that is, what the model says openly to all the other players.

比如在上图中，我们可以看到Mona和Grace两个人的内心想法和最终的投票。他们两个人其实都是狼，但是Mona经过一番思考后（Private Reasoning）还是投给了Grace；同时，Grace也选择投给了Mona。

For example, in the figure above, we can see the private reasoning and final votes of Mona and Grace. Both of them are actually werewolves. However, after a round of private reasoning, Mona still chooses to vote for Grace. At the same time, Grace also chooses to vote for Mona.

如果，我们不能看到内心的推理（Private Reasing），我们可能觉得他们两个是不是疯了在乱投。

If we could not see their private reasoning, we might think that the two of them had simply lost their minds and were voting at random.

但如果仔细读完内心的思路之后，我们就会明白，其实是因为Mona已经暴露了自己是狼的身份，如果投票给自己的狼队友（Grace），大家可能会误以为Grace是好人；与此同时，Grace应该和Mona有一定的默契，读懂了Mona为了保全自己的策略，所以自己也投给了Mona。

However, if we read their internal reasoning carefully, we can understand what is actually happening. The reason is that Mona’s identity as a werewolf has already been exposed. If Mona votes for her werewolf teammate Grace, the other players may mistakenly believe that Grace is innocent. Meanwhile, Grace also seems to understand Mona’s intention. She realises that Mona is sacrificing herself in order to protect her teammate. Therefore, Grace also votes for Mona.

总之，可以发现，模型确实是已经具备了一定的“尔虞我诈”的能力的。

From this example, we can see that language models already possess a certain degree of ability to deceive, conceal information, and engage in strategic interaction.

除了狼人杀，还有别的类型的游戏，比如“剧本杀”。

Apart from Werewolf, there are also other types of games, such as murder mystery role-playing games.

在剧本杀里面，会有一个人“死掉”，大家要去推测到底谁是凶手。每个人会拿到属于自己的角色剧本，这个剧本就代表了自己的人设。如果你拿到的“凶手”的人设，那应该做的就是不要告诉别人自己是凶手；而是要想办法欺骗别人，误导别人，让大家以为别的人才是凶手。

In a murder mystery game, one person is “killed”, and the other players need to infer who the murderer is. Each player receives their own character script, and this script defines the role they are supposed to play. If you receive the script of the murderer, then what you should do is obviously not to tell others that you are the murderer. Instead, you need to find ways to deceive others, mislead them, and make them believe that someone else is the real murderer.

在这篇论文发表的时候，作者们发现，如果是现成的模型，玩这个游戏还不能玩的太好。比如看上图红色高亮的部分，这个模型在与别人的对话中，老老实实的就讲出了自己和被害者的关系，这其实就等同于告诉大家他就是凶手了。那这个游戏就很难继续玩下去了。

At the time when this paper was published, the authors found that off-the-shelf models were still not very good at playing this game. For example, in the red highlighted part of the figure above, the model honestly reveals its relationship with the victim during the conversation with other players. But in this game, doing so is almost equivalent to telling everyone that it is the murderer. Once this happens, it becomes very difficult for the game to continue properly.

但是，在做了RL（增强学习）的调整之后，我们可以看出（图中绿色部分），模型就学会了要尽量隐藏与自己相关的关键信息，同时也要去故意的误导别人了。

However, after adjustment through RL, that is, reinforcement learning, the model’s behaviour changes noticeably. As we can see from the green highlighted part in the figure, the model learns to conceal key information related to itself as much as possible. At the same time, it also learns to deliberately mislead others.

除此之外，这篇论文的作者还发现了一个有趣的现象。在针对剧本杀做完了RL之后，再让模型去做别的任务，比如去解数学题，做IFEVal（用来测验模型能不能遵从人的指令去完成任务），模型的表现竟然也提高了。

In addition, the authors of this paper also found an interesting phenomenon. After applying RL to the murder mystery game, they then asked the model to perform other tasks, such as solving mathematical problems and completing IFEval, a benchmark used to test whether a model can follow human instructions. Surprisingly, the model’s performance on these tasks also improved.

具体来说：比如在图中，红色代表有进步，蓝色代表没有进步，竖轴表示不同的任务，横轴表示在不同的数据上做RL（MARO-Simple是比较简单的剧本，Complex是比较复杂的）。

More specifically, in the figure above, red indicates improvement, while blue indicates no improvement. The vertical axis represents different tasks, and the horizontal axis represents RL performed on different types of data. Here, MARO-Simple refers to relatively simple scripts, while MARO-Complex refers to more complex scripts.

我们可以看出，在比较复杂的上面做完RL之后，模型竟然在数学任务上、遵循指令的任务上也有了提高。

From the results, we can see that after RL on the more complex scripts, the model’s performance improved not only in the original murder mystery setting, but also in mathematical tasks and instruction-following tasks.