夜雨聆风理解 AI Agent 的核心技术：Context Engineering （LLM Summary & Observation Masking & Memory）
Context Engineering Explained: A Core Technology Behind AI Agents （LLM Summary, Observation Masking & Memory）
Slides:
https://github.com/createmomo/Notes-on-AI-Agents
Previous Articles:
之前的文章：
Contents:
Why Do We Need Context Engineering? Compressing Context (LLM Summary & Observation Masking) Which Is Better, LLM Summary or Observation Masking? Memory
目录：
为什么需要Context Engineering 压缩Context (LLM Summary & Observation Masking) LLM Summary & Observation Masking 哪一个更好呢？ Memory
关于这篇笔记的说明：
主要参考了Hung-yi Lee老师2026年的机器学习课程（https://www.youtube.com/watch?v=urwDLyNa9FU），其中一小部分内容根据个人理解可能有微小的调整； 保留了最核心的内容，所以比较简短，但也尽量保留了知识的完整。所以适合快速的回顾复习，当然也适合新手快速理解。
About This Note:
This note is primarily based on Prof. Hung-yi Lee’s 2026 Machine Learning course (https://www.youtube.com/watch?v=urwDLyNa9FU). A small portion of the content may include slight adaptations based on personal understanding; Only the core concepts are retained, making this note relatively concise while still preserving the completeness of the knowledge. Therefore, it is well-suited for quick review and revision, and also for beginners to gain a fast understanding.
1. 为什么需要Context Engineering (Why Do We Need Context Engineering?)
在之前的文章中《从 OpenClaw 看 AI Agent 的运作原理》其实我们已经很简单的提到了一下context engineering在Openclaw中是如何被使用的。当然啦，如果你不阅读那篇文章，仅读这篇文章也不会耽误你理解context engineering的概念。
In a previous article, How AI Agents Work: The OpenClaw Case, we briefly introduced how context engineering is applied in OpenClaw. Of course, even if you haven’t read that article, reading this one alone will not prevent you from understanding the concept of context engineering.
大家应该都知道了，我们与OpenClaw的互动越多，就意味着语言模型需要承受的上下文Context就会越来越长。这些上下文中可能还包含了OpenClaw和主人之外的“世界”交互的很多信息，比如它调用了一些工具，得到一些工具的输出等等（这里把这些不是和主人的交互用图中那个小地球来表示）。
As is generally understood, the more interactions we have with OpenClaw, the longer the context that the language model needs to handle becomes. This context may also include a large amount of information from interactions between OpenClaw and the external “world”, beyond just the user, such as tool calls and their outputs. In the figure, these non-user interactions are represented by the small globe icon.
总之，Context会随着各种各样交互的轮数变多，它要包含越来越多的信息，从而变得越来越长。
In summary, as the number of interaction rounds increases, the context needs to include more and more information, and therefore becomes increasingly longer.
如果用数学符号的形式来表示这个过程，其实也很简单。
If we express this process in mathematical notation, it is actually quite straightforward.
的含义是输入，那在刚开始时，就是第一次输入；此时此刻，因为还没有产生上下文，所以这里也是空的。后续随着交互的次数越来越多(越来越大），我们得到的语言模型的输出也越来越多。当语言模型输出之后，我们就需要更新一下上下文，其实就是在上一次的上下文的基础上，追加上新的输入和模型的输出就可以了。
denotes the input. At the beginning, represents the first input. At this point, since no context has been formed yet, it is empty. As the number of interactions increases and becomes larger, the outputs generated by the language model also accumulate. After each model output, the context needs to be updated to , which is essentially done by appending the new input and the model output to the previous context .
现在我们回过头来继续说。这样一直交互下去，导致Context越来越长放手不管肯定是不行的。所以，OpenClaw在这里还起到了一个非常非常关键的作用：它需要来做决定，决定给语言模型看哪些内容。换句话说，OpenClaw传输给语言模型的输入要长度合适（不能太长，也不能太短）。
Now let us return to the main point. If interactions continue in this way, the context will keep growing, and it is clearly not feasible to leave it unmanaged. Therefore, OpenClaw plays a crucial role here: it must decide which information should be presented to the language model. In other words, the input passed from OpenClaw to the language model must be of an appropriate length, neither too long nor too short.
太长了可能会造成资源的浪费、语言模型也可能会表现越来越差等，但如果太短了，又可能会丢失很多关键信息，也可能会导致任务无法成功的完成。
If it is too long, it may lead to wasted computational resources and potentially degrade the performance of the language model. However, if it is too short, important information may be lost, which could result in the failure to complete the task successfully.
基于上面的讨论，如果我们把OpenClaw或者AI Agent选择传输给语言模型看哪些内容的这个行为，还是用数学符号公式化的形式表达出来，就是像下图右半边展示的那样：
Based on the discussion above, if we formalise the behaviour of OpenClaw or an AI agent in selecting which information to pass to the language model using mathematical notation, it can be represented as shown on the right-hand side of the figure below.
就表示了那个行为，简单来说，这个就表示了我们想尽的各种办法和方式，来经过各式各样的处理，来更新我们的上下文。
represents this behaviour. In simple terms, refers to the various methods and strategies we employ to process and update the context .
Compressing Context (LLM Summary & Observation Masking)
其实在之前的文章中，我们已经提到过一些压缩上下文的一些做法。比如，当Context太长的时候，我们可以让语言模型来进行summarise，然后替换原来的部分。
In fact, in previous articles, we have already discussed some approaches to compressing context. For example, when the context becomes too long, we can ask the language model to generate a summary and use it to replace parts of the original context.
除此之外，还有一种方法就是把Context中工具的输出内容，替换成“这里曾经有工具的输出”的表达。所以，这样也会大大减少Context的内容。
In addition, another approach is to replace the tool outputs in the context with a placeholder such as “there was tool output here”. This can significantly reduce the length of the context.
那这两种做法的效果如何呢？下面这篇论文给出了一定的答案。这篇论文是在SWE这个数据上做的验证，这里SWE其实就是指的软件工程，这个数据集大体就是包含了平时软件开发人员会做的事情：给模型一个github repository和issue，看看模型最终能不能解决。
How effective are these two approaches? The following paper provides some insights. The experiments are conducted on the SWE dataset, where SWE refers to software engineering. This dataset broadly consists of typical tasks performed by software developers, in which a model is given a GitHub repository and an issue, and the objective is to evaluate whether the model can successfully resolve it.
在看结果之前，我们先把结果盖住，然后理解一下这些图表的含义：5个图表表示在5个模型上验证效果的好坏；横轴表示解决问题需要付出的代价（这里使用钱USD来衡量的，但大概可以理解成token的使用量）；竖轴表示问题的解决率。
Before examining the results, let us first understand what these figures represent. The five charts correspond to evaluations on five different models. The horizontal axis indicates the cost required to solve the problem, measured in USD, which can be roughly interpreted as token usage. The vertical axis represents the success rate of problem solving.
在图中，黑色的圆圈表示让AI Agent自己去单纯的运行，不采用Context Engineering；红色的方块表示采用我们前面提到的Summary的方法；三角形则表示我们前面提到的把Tool Output替换成其他内容的做法，在这篇论文中，他们称作“Observation Masking”，你可能在不同的场景和文章中可能会看到不同的名字，但大体概念上就是之前我们讲到的那样。
In the figure, the black circles represent the baseline where the AI agent runs without applying context engineering. The red squares indicate the use of the summarisation approach mentioned earlier. The triangles represent the method of replacing tool outputs with alternative content, which is referred to as “observation masking” in this paper. Although different terms may be used in various contexts or publications, the underlying concept remains essentially the same as described earlier.
我们可以看到，整体来说，调用语言模型去做summary（红色方块）和直接替换掉ToolOutput（三角形）的效果是差不多的（解决问题的成功率和需要的开销）；
It can be observed that, overall, using the language model to generate summaries (red squares) and directly replacing tool outputs (triangles) yield similar results in terms of both problem-solving success rates and associated costs.
在上面的图中有一个特别的例子，在图表（e）中，三者的效果差不多，也就是说，我们使用了Context Engieering和不使用的效果是差不多的。作者给出的原因是，在这个模型上面，出现了“trajectory elongation”，轨迹延长的现象。因为我们把Context压缩后，其中一些步骤的内容就不见了，然后就可能会激起语言模型的反思：“唉，这个事情我到底有没有做过？”，再然后他可能会又自己做一遍来“弥补”这个步骤。所以就导致本来已经被节省掉的Context的内容，又被补上来了。
A notable example can be observed in chart (e), where all three approaches achieve similar performance. In other words, applying context engineering yields results comparable to not applying it. The authors attribute this to a phenomenon known as “trajectory elongation”. After the context is compressed, certain intermediate steps may no longer be present. This can prompt the language model to reconsider whether those steps have already been completed, potentially leading it to repeat them in order to compensate. As a result, the content that was originally reduced from the context may be reintroduced.
Which Is Better, LLM Summary or Observation Masking?
那哪一种效果更好呢？我们可以不做选择，通通都要，不同的压缩方法都是可以使用的。
Which approach performs better? In practice, it is not necessary to make a strict choice, as different compression methods can be used in combination.
上面那篇论文的建议是，我们可以考虑再前期可以使用替换Tooloutput的方法。当后期Context的长度到了一定程度时，在使用Summary的方法。
The paper suggests that, in the early stages, the method of replacing tool outputs can be adopted. As the context grows beyond a certain length in later stages, the summarisation approach can then be applied.
Memory
那你可能会想，在上面的论文中，直接把Tool Output都拿掉，会不会有些太粗暴了？万一里面真的有有用的信息呢？虽然一时我们可能用不到，但是真要使用的时候，我们可就无法再次得到了。
One may wonder whether directly removing all tool outputs, as described in the paper above, is somewhat too aggressive. What if some of that information is actually useful? Although it may not be needed immediately, once it is removed, it may no longer be accessible when it is required later.
这一篇论文的设计思路便缓解了这个问题。它不会简单粗暴的把Tool Output替代，而是会把Output存到一个文件里面，同时也会在原来的Context中留下信息：当有需要的时候，在哪里可以找到这些Output。
The design proposed in this paper addresses this issue. Instead of directly replacing tool outputs, it stores them in external files, while leaving references in the original context indicating where these outputs can be retrieved when needed.
当语言模型语言读取其中的内容的时候，它自然就会要求执行读取的指令。
When the language model needs to access this information, it will naturally issue a request to retrieve the corresponding content.
这些存放Output的文件，就存在文件系统中（也就是我们的硬盘中）。
These files storing the outputs are maintained within a file system, that is, on persistent storage such as a hard drive.
如果我们把上面那个思路说的再宽泛抽象一些，也就是说，在做Context Engineering的时候，我们可以引入Memory的理念。为了节省Context的使用，会把一些内容抽出来，存放到Memory中去。
If we generalise the idea further, it can be understood as introducing the concept of memory into context engineering. In order to reduce the usage of context, certain information can be extracted and stored in memory.
所以，这就引出了新的问题：
什么时候去把一些内容存到Memory中呢？ 把什么内容存起来呢？怎么存呢？
This naturally leads to new questions:
When should certain information be stored in memory? What information should be stored, and how should it be stored?
除此之外，还需要考虑：
什么时候要去重新读取Memory中呢？ 需要读哪些内容呢？要怎么读取呢？
In addition, it is also necessary to consider:
When should information be retrieved from memory? What information should be retrieved, and how should it be accessed?
以及还有，这些Memory又是怎么存储的呢？就是简单的存成txt吗？还是更加多样的文件系统（比如一些图结构的存储方式、或者按时序存等等）？
Furthermore, how should this memory be stored? Should it simply be stored as plain text files, or should more sophisticated storage methods be used, such as graph-based structures or time-ordered storage?
不同的论文呢，有不同的设计。这里在图中放了几篇，如果有兴趣可以深入看一看。
Different papers propose different designs. Several examples are shown in the figure, which can be explored in more detail if of interest.
说到现在，我们可以把之前提到的公式，再更新一下了！（↓）
At this point, we can update the formula introduced earlier.
C指的是Context上下文。由于我们引入了Memory的概念，所以这里的上下文可以又细分为P和M两个部分。
denotes the context. With the introduction of the concept of memory, the context can be further divided into two components, and .
P为要实时与语言模型沟通的部分，而M指的是Memory，它们只在需要的时候才会载入到P中。
represents the portion that interacts with the language model in real time, while denotes the memory. The contents of are only loaded into when required.
当与语言模型进行完一轮交流之后，我们也需要更新一下和，因为经过这轮对话之后，我们可能要变动一下我们的Memory，同时P中也会有一些变动。
After each round of interaction with the language model, both and need to be updated. This is because the interaction may lead to changes in the memory, as well as updates to the active context .
