夜雨聆风理解 AI Agent 的核心技术：过滤Context和模型自主做Context Engineering
Context Engineering Explained: Context Filtering and Autonomous Context Engineering
Slides:
https://github.com/createmomo/Notes-on-AI-Agents
之前的文章：
从 OpenClaw 看 AI Agent 的运作原理 理解Context Engineering
Previous Articles:
How AI Agents Work: The OpenClaw Case Context Engineering Explained
目录：
那么多 token 都来自哪里？ 过滤：按需加载 System Prompt 中的过滤 让模型自己做 Context Engineering 存在硬盘上的 Context
Contents:
Where Do All Those Tokens Come From? Filtering: Loading Only What Is Needed Filtering in the System Prompt Letting the Model Perform Context Engineering by Itself Context Stored on Disk
关于这篇笔记的说明：
主要参考了Hung-yi Lee老师2026年的机器学习课程（https://www.youtube.com/watch?v=urwDLyNa9FU），其中一小部分内容根据个人理解可能有微小的调整； 保留了最核心的内容，所以比较简短，但也尽量保留了知识的完整。所以适合快速的回顾复习，当然也适合新手快速理解。
About This Note:
This note is primarily based on Prof. Hung-yi Lee’s 2026 Machine Learning course (https://www.youtube.com/watch?v=urwDLyNa9FU). A small portion of the content may include slight adaptations based on personal understanding; Only the core concepts are retained, making this note relatively concise while still preserving the completeness of the knowledge. Therefore, it is well-suited for quick review and revision, and also for beginners to gain a fast understanding.
1. 那么多token都来自哪里呢？Where Do All Those Tokens Come From?
前面的文章我们介绍了一些压缩Context的概念。但是你有没有发现，在我们介绍的时候，都是在Context已经很长的情况下，然后Agent再去压缩Context。这当然是控制Context长度的一种方法。但是，有没有一种可能，从一开始，我们就想办法，让进入Context的内容（也就是tokens）就不要那么多呢？
In the previous articles, we introduced several ways to compress context. However, you may have noticed something: these methods usually come into play only after the context has already become very long.
This is certainly one way to control context length. But we can take the question one step further: is it possible to reduce the amount of information entering the context from the very beginning? In other words, can we avoid putting so many tokens into the context in the first place?
要弄清楚这个问题，肯定是要先搞清楚，在我们的Context中，到底包含了哪些内容、哪些内容的占比最大。（一般来说，我们肯定是要先解决分量最大的那个，这样带来的效果才会更快更明显）
To answer this question, we first need to understand what is actually included in an agent’s context, and which part takes up the largest share. Generally speaking, if we want to optimize context length, we should first focus on the part that takes up the most space. This usually leads to faster and more visible improvements.
这篇论文就分析了类似的事情。从图中我们可以看出，确实有一个比例非常非常的大。
One paper analyzed a similar question. According to its statistics, one type of content clearly takes up a very large proportion of the agent’s context.
We can roughly divide the content in the context into three categories: Action, Reasoning, and Observation.
Action: 表示的是模型去产生执行工具的指令，一般来说这些指令都很短。它的占比也确实不高。
Action refers to the instructions generated by the model in order to call tools. For example, when the model decides to read a file, run a program, or call an external tool, it needs to produce the corresponding action command. In most cases, this type of content is relatively short, so it does not take up a large share of the context.
Reasoning：大体是在指模型自己说出来的话。
Reasoning can be roughly understood as the words produced by the model itself. In other words, when the model analyzes the task, explains its thinking, or plans the next step, the generated content can be placed in this category.
Observation：来自外界的输入，比如载入的文件的内容、一些工具的输出啊等等。
Observation refers to information coming from the outside world. This may include the contents of files that have been loaded, outputs from tools, search results, logs, or error messages produced after running a program.
除此之外，还有一篇论文做了类似的事情，以及也得出了相似的结论。只不过这一篇论文主要是集中在software engineering这个主题上。比如让模型去修改代码、执行程序等。
Another paper conducted a similar analysis and reached a similar conclusion. However, that paper focused mainly on software engineering tasks, such as asking a model to modify code, read code, run programs, or fix bugs.
我们可以看出来，其中大概有12%和执行程序有关、11%和修改相关，大部分是在读代码（Read），占了相当大的比例。
We can see that about 12% is related to executing programs, and 11% is related to modification. Most of the content is related to reading code, namely Read, which accounts for a fairly large proportion.
总之，到底有没有办法治本呢？也就是说，有没有办法从一开始就不要读入这么多的内容进入Context呢？
So, is there any way to address the root cause? In other words, is there any way to avoid reading so much content into the Context from the very beginning?
2. 过滤（按需加载）Filtering: Loading on Demand
本质上来说，就是按需加载。当我不需要一些内容的时候，就先不要让它们进入Context。只有当我需要的时候，才把他们读进来。
In essence, this means loading content on demand. When certain information is not needed, we should avoid putting it into the Context for the time being. Only when it becomes necessary should we read it into the Context.
比如，当我们需要读取一些日志的时候，模型就说，请调用Read的工具吧，把log读进来我看一下。
For example, when we need to read some logs, the model may say: please call the Read tool and load the log so that I can examine it.
然后这个Read工具就会打开存在硬盘上的log，然后把它读进来。
The Read tool then opens the log stored on the hard drive and reads it into the Context.
我们也可以把这个工具做的更加精致一些，比如，模型说想要在log中读一下和“Bug Fixing”相关的，那么这个工具就会只返回和“Bug Fixing”相关的日志内容。
We can also make this tool more refined. For example, if the model says that it wants to read the parts of the log related to “Bug Fixing”, the tool will return only the log content relevant to “Bug Fixing”.
所以，这个工具起到了“过滤”的作用。
Therefore, this tool serves a “filtering” function.
那么，这个所谓的工具，到底是做一些工程上的技巧，还是训练一个小的模型来专门做这件事情，可以根据情况来自行决定。
As for what this tool actually is, it can be implemented either through engineering techniques or by training a smaller model specifically for this task. The choice can be made according to the specific situation.
总之，就是让这个工具想尽办法更智能一些，让它只读取需要的内容。
In short, the idea is to make this tool as intelligent as possible, enabling it to read only the content that is actually needed.
Memory
之前我们也提到过Agent的体系里面一般会有memory_get这个工具。其实也是使用了类似于过滤的概念：到底要读取哪些记忆到Context中，Agent会自己来决定，而不是全部都读入。
As mentioned previously, an Agent system usually includes a tool such as memory_get. This also follows a similar idea of filtering: the Agent decides which memories should be read into the Context, rather than loading all of them at once.
过滤（System Prompt）Filtering in the System Prompt
前面我们也提过，System Prompt是一种特殊的Prompt，在每一轮和模型的对话中，这部分内容是都要一直跟着的。
As mentioned earlier, the System Prompt is a special type of prompt. In every round of interaction with the model, this part of the content always remains attached to the conversation.
在System Prompt中，会放置很多有用的信息。比如模型现在有哪些可以使用的工具，当在对话中万一需要用到哪个工具，模型就可以根据System Prompt中的指引，去找到对应的工具并调用。
A System Prompt contains a great deal of useful information. For example, it may tell the model which tools are currently available. If a particular tool is needed during the conversation, the model can follow the instructions in the System Prompt to find and call the corresponding tool.
但，在System Prompt中，关于各式各样的tools的信息，我们到底要放多详细呢？还是只放一部分呢？还是...
However, how much detail should we include in the System Prompt about all kinds of tools? Should we include only part of the information? Or should we handle it in some other way?
比如，在上图中，现在有一个关于使用github的工具。如果要把这个工具的信息塞到System Prompt里面，就需要多于4600的tokens呀。
For example, in the figure above, there is a tool for using GitHub. If we put the information about this tool into the System Prompt, it would require more than 4,600 tokens.
而你可能运行一整天，也用不了这个工具几次，但是这4600多个token却一直跟在你和模型的对话中，这不就是在白白烧钱吗？
You might run the Agent for an entire day and only use this tool a few times. Yet those 4,600-plus tokens would continue to follow every conversation between you and the model. Would that not simply be wasting money?
所以，还是要想办法，不要载入太多的信息。只有需要的之后，再去查阅就好了。
Therefore, we still need to find ways to avoid loading too much information. Only when it is needed should the model look it up.
一种传统的思路是说：就先不加载可以用的tools。只有当一个task，需要寻找合适的工具时，就使用类似于RAG的概念，去搜索有哪些工具可以用。
One traditional approach is not to load all available tools at the beginning. Only when a task requires a suitable tool do we use an approach similar to RAG to search for the tools that may be available.
但其实这种方法往往表现的不是很好，因为很多时候，用户的需求（也就是对task的描述和定义）是很模糊的，很难直接根据他们的描述来搜索使用哪些工具。
In practice, however, this method often does not work very well. In many cases, the user’s request, namely the description and definition of the task, is quite vague. It is difficult to search directly for the right tools based only on that description.
比如，task是想让模型去修复某个bug。但很可能完成这个任务不是只用一个工具就可以了，有可能需要Read工具、编辑的工具等等。但仅仅根据用户的描述，很难让搜索引擎自己一下子决定使用哪些工具。
For example, the task may be to ask the model to fix a bug. However, completing this task may well require more than one tool. It may require a Read tool, an editing tool, and other tools as well. Based only on the user’s description, it is difficult for the search engine itself to determine immediately which tools should be used.
所以，一个可以考虑的改进思路是：让AI模型自己去决定，它想要什么工具，也就是让他自己讲出来“Description of the required tool”。
Therefore, one possible improvement is to let the AI model decide for itself what kind of tool it wants. In other words, the model can produce a “Description of the required tool” on its own.
讲出来之后，搜索引擎再发挥它的作用。
After that description has been produced, the search engine can then play its role.
Skill
之前我们也提到过Agent中的Skill的概念，当我们用户提出一个任务，如果Agent中有预先装好的Skill并且适合这个任务，那么Agent就会去调用这个Skill来完成任务。
We also mentioned previously the concept of a Skill in an Agent. When the user gives a task, if the Agent has a pre-installed Skill that is suitable for the task, the Agent will call that Skill to complete it.
这里，其实也是采用了按需加载的概念。也就是说，Agent不会把Skill的详细内容，一下子全部载入到Context中。
This also follows the idea of loading on demand. In other words, the Agent does not load all the detailed content of a Skill into the Context at once.
那，我们现在再次返回看一下这个公式。在F这个函数中（根据当前的Context,输入和模型的输出来决定新的Context包含哪些内容），大部分来说其实都是由人类自己来决定的。
Now let us return to this formula. In the function F, which determines what the new Context should contain based on the current Context, the input, and the model’s output, most of the work is in fact decided by humans.
换句话说，是我们人类再想进各式各样的方法（无论是工程上的技巧，还是我们在训练一些小的模型来辅助完成这个任务）来完成F函数的工作。
In other words, it is humans who are trying all kinds of methods, whether engineering techniques or small models trained to assist with this task, to perform the work of the function F.
模型自己来想办法做Context Engineering (Letting the Model Find Ways to Perform Context Engineering by Itself)
那，有没有可能，让这部分变得更加智能一些呢？也就是，我们人类不会想尽办法来给他做Context Engineering，而是模型自己去想办法，它自己随便怎么折腾自己的Context。
Then, is it possible to make this part more intelligent? In other words, instead of humans trying every possible way to perform Context Engineering for the model, could the model itself find ways to manage its own Context, however it chooses?
比如在这篇论文中，当产生Context_t和Input_t和Output_t之后，然后模型自己根据这三者去产生新的Context_{t+1}。
For example, in this paper, after Context_t, Input_t, and Output_t have been produced, the model itself generates a new Context_{t+1} based on these three elements.
然后以此类推，得到Context_{t+2}。
Following the same logic, it then obtains Context_{t+1}.
再然后一直执行下去。
The process then continues in this manner.
其实，你会发现，比较重要的内容（比如System Prompt）是没有参与这个过程的。也就是说，在类似的工作中，我们可以专门留出来Context中的一部分让LLM模型一定不要碰，以及也专门有一部分是给LLM来随便玩的，它愿意在里面放什么内容就放什么。
In fact, you will notice that more important content, such as the System Prompt, does not participate in this process. In other words, in similar work, we can deliberately reserve one part of the Context that the LLM must not touch, while also reserving another part for the LLM to freely manipulate. The model can place whatever it wants in that part.
当然了，LLM模型自己去生成下一轮的Context这件事情，也不是随便让它自己瞎做的。我们人类是可以给他一定的指导。比如在这篇论文中，作者们给它的指导就有这么长（大概精神是过于具体的记录以后很可能用不上了，就没有必要保存了；只需要保留一些有效的经验策略、可重用的code和一些关键的发现）。
Of course, allowing the LLM to generate the next round of Context by itself does not mean letting it do so blindly. Humans can still give it certain guidance. For example, in this paper, the authors provide it with a fairly long set of instructions. The general idea is that overly specific records are unlikely to be useful later, and therefore do not need to be preserved. What should be retained are effective experience-based strategies, reusable code, and some key findings.
与上一篇论文不同，这篇论文做了更复杂的事情。（注意：在这篇论文中，他们把Context称作Playbook）。
Unlike the previous paper, this paper does something more complex. Note that in this paper, the authors refer to the Context as a Playbook.
在这个工作中，Context不是仅仅经过一个模型，而是要经过3个模型。这3个模型分别做过不同方面的检查之后，不会直接产生新的Context（因为直接产生新的Context可能会损坏一些关键内容）,而是会先生成一个如何修改Context的指令（Edit Instruction）。
In this work, the Context does not pass through only one model. Instead, it passes through three models. After these three models have carried out checks from different perspectives, they do not directly produce a new Context, because directly generating a new Context may damage some key content. Instead, they first generate an instruction for how to modify the Context, namely an Edit Instruction.
根据这个Edit Instruction，再去生成新的Context。
A new Context is then generated according to this Edit Instruction.
存在硬盘上的Context (Context Stored on the Hard Drive)
还有一种情况是，上下文特别特别特别的长，一直长到了要存在硬盘上再行。
There is another situation in which the context becomes extremely long, to the point where it has to be stored on the hard drive.
那在实际的运行过程中，其实只需要载入一小部分关键的信息到实时的Context中。在这篇论文中，这些关键信息叫做metadata。比如说：这个Context的全文到底有多长、被切成了几段、存放在了哪里等。
In actual operation, only a small amount of key information needs to be loaded into the real-time Context. In this paper, this key information is called metadata. For example, it may include how long the full Context is, how many segments it has been divided into, where it is stored, and so on.
LLM模型会根据这些metadata（P_t），看看需要读取硬盘（M_t）中的哪些资料。
The LLM uses this metadata, P_t, to determine which materials it needs to read from the hard drive, M_t.
当然了，LLM也可以自己去写以及执行程序，去对硬盘中的内容做搜寻，他自己可以去做RAG把必要的信息拿出来，然后去修改他的metadata。
Of course, the LLM can also write and execute programmes by itself to search the content stored on the hard drive. It can perform RAG on its own to retrieve the necessary information, and then modify its metadata.
总之，这其中怎么引导LLM或者让LLM去用更好的方法去完成Context Engineering这件事情，是需要做Prompt engineering的，也就是说在给LLM看的Prompt中，要不断的、努力的尝试和暗示LLM模型在必要的时候去调用RAG的方式，去搜索需要的资料。
In short, how to guide the LLM, or how to encourage it to use better methods to complete Context Engineering, still requires Prompt Engineering. In other words, in the prompt shown to the LLM, we need to keep trying to guide and hint to the model that, when necessary, it should use RAG to search for the materials it needs.
所以，其实并没有什么特别的魔法突然一下子让模型的Context Engieneering突然变得特别好。
Therefore, there is no special magic that suddenly makes the model’s Context Engineering become exceptionally good all at once.
虽然这个方法看起来没有什么太神奇的地方，不过实际的表现效果是好的。在这个图中，上半部分是不采用这篇论文方法的表现，可以看的出来，随着token越来越长，在一些任务上的效果是有明显的下降的。
Although this method may not seem particularly magical, its actual performance is good. In this figure, the upper half shows the performance without using the method proposed in this paper. We can see that, as the number of tokens increases, performance on some tasks declines noticeably.
而如果采取这篇论文的做法，即使token变得很长，仍然可以收获不错的效果。
By contrast, if the method proposed in this paper is adopted, the model can still achieve good results even when the number of tokens becomes very large.
更重要的是，这个方法其实更像是一种机制，可以应用到任何的LLM上，而不是只有特定的LLM可以使用。
More importantly, this method is more like a general mechanism. It can be applied to any LLM, rather than being limited to a specific model.
