Build AI Agents(1)-Concepts

Planning

The plan agent enables us to create detailed implementation plans before starting the actual implementation to ensure all requirements are met.

• • The plan agent can ensure it stays focused on the overall goals and tracks progress effectively.
• • We can use the build-in plan agent or third-party agent to plan a task.
• • The plan agent will generate a todo list based on the requirement.

We can also type /plan followed our task description to switch to the plan agent and start planning in one step.

For example, we can enter a high-level task (feature, refactoring, bug, etc.) and submit it to the plan agent.

Implement a user authentication system with OAuth2 and JWT

Use the /plan slash command to start planning directly from the chat input box:

/plan Add unit tests for all API endpoints

We need to answer any clarifying questions the agent asks after researching the plan task.

We will see that the plan agent generates a high-level plan summary, implementation and verification steps.

We can review the plan draft and submit follow-up prompts to iterate on the plan until it meets our requirements.

• • To implement the plan, we can continue in the same session or start a new Copilot CLI session to implement the plan in the background.
• • To implement the plan, you can continue in the same session or start a new Copilot CLI session to implement the plan in the background.
• • To implement the plan, we can continue in the same session or start a new session to implement the plan in the background.

Whenthe plan is finalized, we can choose to start the implementation or open the planning prompt in the editor for further review.

The plan agent automatically saves its implementation plan to a session memory file (/memories/session/plan.md).

• • To access this file, run the Chat: Show Memory Files command and select plan.md from the list.
• • Session memory is cleared when the conversation ends, so the plan is not available in subsequent sessions.

We can tailor the planning process to fit our team's workflow:

• • Create a custom planning agent. Define a custom agent with specific instructions for our planning process, such as enforcing architectural guidelines or requiring specific planning deliverables.
• • Choose models for planning and implementation. Use the chat.planAgent.defaultModel setting to select a default model for the plan agent, and github.copilot.chat.implementAgent.model for the implementation step.
• • Add extra tools to the plan agent. Use the github.copilot.chat.planAgent.additionalTools setting to give the plan agent access to additional tools during the research and planning phases. For example, use an MCP server to connect to internal data sources or tools.

Memory

The agent could use memory to retain context across conversation without starting from scratch each session, then it can recall our perferences, apply lessons from previous tasks, and build up knownledge about our codebase over time.

The memory tool is a built-in agent tool that allows agents to save and recall notes as they work, we can enable or disable the memory tool with the github.copilot.chat.tools.memory.enabled setting.

• • All memory data is stored locally on local machine.
• • The memory tool is enabled by default.

Each scope serves a different purpose, depending on how long the information should persist and where it applies.

User Memory

User memory persists across all workspaces and conversations.

• • The first 200 lines are automatically loaded into the agent's context at the start of every session.
• • Use user memory for general preferences and insights that apply regardless of which project we're working in.

For example, ask the agent to remember a coding preference:

Remember that I prefer tabs over spaces and always use single quotes in JavaScript

The agent recalls this preference and applies it to generated code in all the workspaces and conversations.

Repository Memory

Repository memory is scoped to the current workspace and persists across conversations in that workspace.

• • Use repository memory for facts about a specific codebase, such as architecture decisions, naming conventions, or build commands.

Remember that this project uses the repository pattern for data access and all API endpoints require authentication

Session Memory

Session memory is scoped to the current conversation and cleared when the conversation ends.

• • Use session memory for temporary working notes or task-specific context that the agent tracks while working through a multi-step task.

The plan agent uses session memory to persist its implementation plans in a plan.md file.

• • This plan is available during the session and can be viewed with the Chat: Show Memory Files command, but is not available in subsequent sessions.

Copilot Memory

Copilot Memory is a GitHub-hosted memory system that lets Copilot learn and retain repository-specific insights as it works.

• • Copilot Memory is shared across multiple GitHub Copilot surfaces, including Copilot cloud agent, Copilot code review, and Copilot CLI.

Copilot Memory automatically capture tightly scoped insights called "memories". These memories are:

• • Repository-scoped: memories are tied to a specific repository and can only be created by contributors with write access.
• • Cross-agent: what one Copilot agent learns is available to other agents. For example, a pattern discovered by Copilot code review can later guide Copilot cloud agent.
• • Verified before use: agents validate memories against the current codebase before applying them, preventing stale or incorrect information from affecting results.
• • Automatically expired: memories are deleted after 28 days to avoid outdated information.

Repository owners can review and delete stored memories in Repository Settings > Copilot > Memory.

• • Use the local memory tool for personal preferences and session-specific context in VS Code.
• • Use Copilot Memory for repository knowledge that benefits all Copilot agents across your development workflow.

Memory Management

To store a memory, ask the agent to remember something in natural language. The agent determines the appropriate scope and creates or updates the corresponding memory file.

Remember that our team uses conventional commits for all commit messages

To retrieve a memory, ask about it in a new conversation. The agent checks its memory files and recalls the relevant information.

What are our commit message conventions?

Memory file references in the agent's chat responses are clickable, so we can view the contents of the memory file directly.

VS Code provides commands to view and manage our memory files:

• • Chat: Show Memory Files: opens a list of all memory files across scopes. Select a file to view its contents.
• • Chat: Clear All Memory Files: removes all memory files across all scopes.

Tools

Tools extend agents with specialized functionality for accomplishing specific tasks like searching code, running commands, fetching web content, or invoking APIs.

VS Code supports three types of tools: built-in tools, Model Context Protocol (MCP) tools, and extension tools.

We can enable or disable tools on a per-request basis by using the tools picker, and we can add more tools by installing MCP servers or extensions that contribute tools.

When we customize chat with prompt files or custom agents, we can specify which tools are available for a given prompt or mode.

The agent can automatically determine which tools to use from the enabled tools based on our prompt and the context of our request.

• • The agent autonomously chooses and invokes relevant tools as needed to accomplish the task.
• • We can also explicitly reference tools in our prompts by typing # followed by the tool name.

Type # in the chat input field to see a list of available tools, including:

• • Built-in tools
• • MCP tools from installed servers
• • extension tools
• • tool sets

Here are some examples of explicit tool references:

• • "What is the latest version of Node.js #web"
• • "How does routing work in Next.js? #web"
• • "Fix the issues in #problems"
• • "Explain the authentication flow #codebase"

By default, tool call details are collapsed in the chat conversation, we can uncollapse them by selecting the tool summary line in chat.

The permissions picker in the Chat view controls how much autonomy the agent has during a session.

• • Select a permission level from the permissions dropdown in the chat input area to choose how tool calls and approvals are handled.
• • Default Approvals: Uses our configured approval settings. Tools that require approval show a confirmation dialog before they run. The agent might ask clarifying questions if needed.
• • Bypass Approvals: Auto-approves all tool calls without showing confirmation dialogs and automatically retries on errors. The agent might ask clarifying questions if needed.
• • Autopilot: Auto-approves all tool calls without showing confirmation dialogs and auto-responds to clarifying questions. The agent continues working autonomously until the task is completed.

Bypass Approvals and Autopilot bypass manual approval prompts, including for potentially destructive actions like file edits, terminal commands, and external tool calls. The first time we enable either level, a warning dialog asks us to confirm. Only use these levels if we understand the security implications.

The permission level applies to the current chat session. We can change it at any time during a session by selecting a different level from the permissions picker. We can stop the agent at any time by selecting the stop button.

By default, new chat sessions start with the Default Approvals level. To persist our preferred permission level across sessions, configure the chat.permissions.default setting.

Tool approval

Some tools require our approval before they can run. This is a security measure because tools can perform actions that modify files, our environment, or attempt prompt injection attacks through malicious tool output.

Tools and agent actions might result in file modifications.

• • When a tool requires approval, a confirmation dialog appears showing the tool details.
• • Review the information carefully before approving the tool.

We can approve the tool for a single use, for the current session, for the current workspace, or for all future invocations.

For each tool, we can configure two types of approvals:

• • Pre-approval ("without approval"): skip the confirmation dialog before the tool runs.
• • Post-approval ("without reviewing result"): skip reviewing the tool's output before it is added to the chat context. This is relevant for tools that return external data, where the content might contain prompt injection attempts.

Expand a source to configure approvals for individual tools, or select the top-level checkboxes to trust all tools from a specific MCP server or extension at once.

URL approval

When a tool attempts to access a URL, such as with the #web/fetch tool, a two-step approval process is used to protect us from malicious or unexpected content. VS Code shows a confirmation dialog with the URL details for our review in the Chat view.

Pre-approval: approving the request to the URL

This step ensures that we trust the domain being contacted and can prevent sensitive data to be sent to untrusted sites.

We have options for one-time approval or for automatically approving future requests to the specific URL or domain.

• • Selecting auto-approval does not influence the need for reviewing the results.
• • When we select Allow requests to, we can choose to configure both pre and post approvals for the URL or domain.
• • The pre-approval respects the "Trusted Domains" feature. If a domain is listed there, we are automatically approved to make requests to that domain and defer the response reviewing step.

Post-approval: approving the response content fetched from the URL

This step ensures that we review the fetched content before it is added to the chat or passed to other tools, preventing potential prompt injection attacks.

For example, we might approve a request to fetch content from a well-known site, like GitHub.com. But because the content, such as issue description or comments, is user-generated, it could contain harmful content that might manipulate the model's behavior.

We have options for one-time approval or for automatically approving future responses from the specific URL or domain.

The post-approval step is not linked to the "Trusted Domains" feature and always requires our review. This is a security measure to prevent issues with untrusted content on a domain that we would otherwise trust.

The chat.tools.urls.autoApprove setting is used to store our auto-approve URL patterns.

• • The setting value is either a boolean to enable or disable auto-approvals for both requests and responses, or an object with approveRequest and approveResponse properties for granular control.
• • We can use exact URLs, glob patterns, or wildcards.

{"chat.tools.urls.autoApprove": {"https://www.example.com": false,"https://*.contoso.com/*": true,"https://example.com/api/*": {"approveRequest":true,"approveResponse":false  }}

Tool parameters

We can review and edit the input parameters before a tool runs:

• • When the tool confirmation dialog appears, select the chevron next to the tool name to expand its details.
• • Edit any tool input parameters as needed.
• • Select Allow to run the tool with the modified parameters.

Terminal commands

The agent might use terminal commands as part of its workflow to accomplish tasks. When the agent decides to run terminal commands, it uses the built-in terminal tool to execute them in an integrated terminal within VS Code.

In the chat conversation, the agent displays the commands it ran.

• • We can view the output of the command inline in chat by selecting Show Output (>) next to the command.
• • We can also view the full output in the integrated terminal by selecting Show Terminal.

Use the chat.tools.terminal.outputLocation setting to configure where terminal command output appears: inline in chat, in the integrated terminal.

• • We can see the list of terminals that the agent has used for a chat session.
• • We can also distinguish agent terminals by the chat icon in the terminals list.

Background task

When the agent runs a long-running terminal command, such as starting a development server or running a build in watch mode, we can push the command to the background. This allows the agent to continue with other tasks without waiting for the command to finish.

While a command is running, a Continue in Background button appears next to the terminal command in the chat conversation. Select this button to move the command to the background. The command continues running, and the agent can check its output later or use the terminal for other tasks.

The agent can also specify a timeout when running terminal commands. When the timeout is reached, the agent stops waiting for the command and returns the output collected so far. Use the chat.tools.terminal.enforceTimeoutFromModel setting to control whether to enforce the timeout value that the agent specifies.

The agent can also choose to run commands directly in the background, without user interaction.

• • Background terminals that we have not revealed are automatically cleaned up when their command finishes, which prevents stale terminals from accumulating over a long session.

To reveal a background terminal and keep it open after the command completes, select the Show link in the chat tool invocation header. The terminal output remains visible in the chat conversation even after a terminal is cleaned up.

We can configure which terminal commands are automatically approved by using the chat.tools.terminal.autoApprove setting.

We can specify both allowed and denied commands:

• • Set commands to true to automatically approve them
• • Set commands to false to always require approval
• • Use regular expressions by wrapping patterns in / characters

{// Allow the `mkdir` command  "mkdir":true,  // Allow `git status` and commands starting with `git show`  "/^git (status|show\\b.*)$/":true,  // Block the `del` command  "del":false,  // Block any command containing "dangerous"  "/dangerous/":false}

By default, patterns match against individual subcommands. For a command to be auto-approved, all subcommands must match a true entry and must not match a false entry.

For advanced scenarios, use object syntax with the matchCommandLine property to match against the full command line instead of individual subcommands.

Automatically approving terminal commands provides best effort protections and assumes the agent is not acting maliciously. It's important to protect ourselves from prompt injection when we enable terminal auto approve, as it might be possible for some commands to slip through.

Here are some examples where the detection can fall over:

• • VS Code uses PowerShell and bash tree sitter grammars to extract sub-commands, so patterns are not detected if these grammars don't detect them.
• • VS Code uses bash grammar because there is no zsh or fish grammar, so some sub-commands are not detected.
• • Detection of file writes is currently minimal, so it might be possible to write to files with the terminal that would not be possible by using the file editing agent tools.
• • Subverting auto approval is possible through various techniques such as quote concatenation. For example find -exec is normally blocked, but find -e"x"ec is not, despite doing the same thing.

If prompt injection is a possibility or we're in a high-risk environment, consider enabling agent sandboxing or running VS Code within a container.

Sandbox agent

Agent sandboxing restricts file system and network access for commands executed by the agent. When sandboxing is enabled, terminal commands are auto-approved without requiring user confirmation, because they run in a controlled environment.

We can choose between full isolation, which restricts both file system and network access, or file-system-only isolation, which allows unrestricted outbound network traffic.

• • To configure agent sandboxing, set the chat.agent.sandbox.enabled

This setting is managed at the organization level.

• • off(default): Sandboxing is disabled.
• • on: Full sandboxing with file system and network isolation. All outbound network access is blocked unless domains are explicitly allowed.
• • allowNetwork: Sandboxing with file system isolation only. Outbound network traffic is allowed without requiring domain configuration, while file system restrictions still apply.

When sandboxing is enabled (on or allowNetwork):

When file system access is restricted, the following rules apply to agent commands:

• • Commands have read access to workspace folders, the sandbox runtime temp folder, and any per-command paths that VS Code adds automatically (for example, paths required by git, node, npm, dotnet). Reads from our home directory ($HOME) are denied by default.
• • Commands have write access only to the current working directory and its subdirectories
• • Commands run without the user confirmation prompt

When network access is restricted, the following rules apply to agent commands:

• • All outbound network access is blocked unless domains are explicitly allowed.
• • We can configure domain-level exceptions with chat.agent.allowedNetworkDomains and chat.agent.deniedNetworkDomains, denied domains take precedence over allowed domains.
• • When set to allowNetwork, all outbound network traffic is permitted and domain settings are ignored.

If the required OS dependencies for sandboxing are not installed, VS Code offers to install the necessary components. If we choose not to install them, sandboxing is not enabled.

Use the chat.agent.sandbox.FileSystem.linux or chat.agent.sandbox.FileSystem.mac setting to control file system access.

We can specify allow rules for read and write access, and deny rules for both read and write access. These rules don't support glob patterns. The denyWrite and denyRead rules take precedence over allowWrite and allowRead rules.

Workspace folders, the sandbox runtime temp folder, and per-command read paths are allowed automatically, so we typically only need allowRead to grant access to tool configurations or data outside our workspace.

{"chat.agent.sandbox.FileSystem.mac": {// Allow writes to the working directory"allowWrite": ["."],// Allow reads from an additional path outside the workspace"allowRead": ["/Users/me/.config/myapp"],// Block writes to specific subdirectories"denyWrite": ["./secrets/"],// Block reads from specific paths"denyRead": ["/etc/passwd"]  }}

We can restrict which domains agent tools (fetch tool, integrated browser) can access by enabling thechat.agent.networkFilter setting.

• • When enabled, network access is controlled by the chat.agent.allowedNetworkDomains andchat.agent.deniedNetworkDomains settings.
• • When both lists are empty, all domains are blocked.
• • When sandboxing is also enabled, these network rules additionally apply to terminal commands executed by the agent.

Denied domains always take precedence over allowed domains. Both settings support wildcards like *.example.com.

{"chat.agent.networkFilter":true,"chat.agent.allowedNetworkDomains":["api.github.com"],"chat.agent.deniedNetworkDomains":["example.com"]}

Group tools

A tool set is a collection of tools that we can reference as a single entity in our prompts.

• • Tool sets help us organize related tools and make them easier to use in a chat prompt, prompt files, and custom chat agents.
• • Some of the built-in tools are part of predefined tool sets, such as #edit and #search.

Create a tool set

We can create a tool set:

1. 1. Run the Chat: Configure Tool Sets command from the Command Palette and select Create new tool sets file, we can also select the ellipsis (...) menu in the Chat view, select Tool Sets, and then select Create new tool sets file.
2. 2. Define our tool set in the .jsonc file that opens.

{"reader":{"tools":["search/changes","search/codebase","read/problems","search/usages"],"description":"Tools for reading and gathering context","icon":"book"}}

• • tools: Array of tool names (built-in tools, MCP tools, or extension tools)
• • description: Brief description displayed in the tools picker
• • icon: Icon for the tool set

We can create our own tools in two ways:

• • Develop a VS Code extension that contributes tools using the Language Model Tools API
• • Create an MCP server that provides tools.

Use a tool set

Type # in the chat input field to see a list of all available tools. We can also use the tools picker in chat to view and manage the list of active tools.

Reference a tool set in our prompts by typing # followed by the tool set name:

• • "Analyze the codebase for security issues #reader"
• • "Where is the DB connection string defined? #search"

In the tools picker, tool sets are available as collapsible groups of related tools. We can select or deselect entire tool sets to quickly enable or disable multiple related tools at once.

A chat request can have a maximum of 128 tools enabled at a time. If we see an error about exceeding 128 tools per request:

• • Open the tools picker in the Chat view and deselect some tools or entire MCP servers to reduce the count.
• • Alternatively, enable virtual tools with the github.copilot.chat.virtualTools.threshold setting to automatically manage large tool sets.