Envoy HTTP 协议实现技术文档
1. 概述
HTTP 是 Envoy 代理的核心协议,负责处理应用层通信。Envoy 的 HTTP 实现建立在其强大的网络层基础之上,与 TCP 和 libevent 架构深度关联。
核心特性
-
多协议支持: HTTP/1.1、HTTP/2、HTTP/3
-
自动协议检测: 根据数据流自动识别协议版本
-
连接管理: 完整的连接生命周期管理
-
流控制: 支持多路复用和流量控制
-
过滤器链: 可扩展的请求/响应处理管道
2. HTTP 连接管理器 – ConnectionManagerImpl
2.1 类定义
class ConnectionManagerImpl : Logger::Loggable<Logger::Id::http>,public Network::ReadFilter,public ServerConnectionCallbacks,public Network::ConnectionCallbacks,public Http::ApiListener {public:ConnectionManagerImpl(ConnectionManagerConfig& config,const Network::DrainDecision& drain_close,Random::RandomGenerator& random_generator,Http::Context& http_context,Runtime::Loader& runtime,const LocalInfo::LocalInfo& local_info,Upstream::ClusterManager& cluster_manager,Server::OverloadManager& overload_manager,TimeSource& time_system);~ConnectionManagerImpl() override;// Network::ReadFilterNetwork::FilterStatus onData(Buffer::Instance& data, bool end_stream) override;Network::FilterStatus onNewConnection() override;void initializeReadFilterCallbacks(Network::ReadFilterCallbacks& callbacks) override;// Http::ServerConnectionCallbacksRequestDecoder& newStream(ResponseEncoder& response_encoder,bool is_internally_created = false) override;// Network::ConnectionCallbacksvoid onEvent(Network::ConnectionEvent event) override;void onAboveWriteBufferHighWatermark() override;void onBelowWriteBufferLowWatermark() override;};
2.2 核心功能
|
|
|
|---|---|
| HTTP 协议处理 |
|
| 与网络层关联 |
|
| 协议版本支持 |
|
2.3 实现细节
// HTTP 数据处理流程Network::FilterStatus ConnectionManagerImpl::onData(Buffer::Instance& data, bool end_stream) {// 延迟创建编解码器,直到有数据到达if(!codec_) {createCodec(data);}// 将数据传递给编解码器处理codec_->dispatch(data);// 继续处理下一个网络过滤器(如果有)return Network::FilterStatus::Continue;}// 编解码器创建流程void ConnectionManagerImpl::createCodec(Buffer::Instance& data) {// 根据数据判断协议版本if(isHttp2(data)) {codec_ = createHttp2Codec();} else if (isHttp1(data)) {codec_ = createHttp1Codec();} else {handleCodecError("Unsupported protocol");return;}codec_->initialize();}
3. HTTP 编解码器客户端 – CodecClient
3.1 类定义
class CodecClient : protected Logger::Loggable<Logger::Id::client>,public Http::ConnectionCallbacks,public Network::ConnectionCallbacks,public Event::DeferredDeletable {public:CodecClient(CodecType type, Network::ClientConnectionPtr&& connection,Upstream::HostDescriptionConstSharedPtr host, Event::Dispatcher& dispatcher);voidconnect();RequestEncoder& newStream(ResponseDecoder& response_decoder);voidclose();Protocol protocol(){ return codec_->protocol(); }size_tnumActiveRequests(){ return active_requests_.size(); }// Http::ConnectionCallbacksvoidonGoAway(GoAwayErrorCode error_code)override;voidonSettings(ReceivedSettings& settings)override;voidonMaxStreamsChanged(uint32_t num_streams)override;// Network::ConnectionCallbacksvoidonEvent(Network::ConnectionEvent event)override;voidonAboveWriteBufferHighWatermark()override;voidonBelowWriteBufferLowWatermark()override;};
3.2 核心功能
|
|
|
|---|---|
| HTTP 客户端抽象 |
|
| 与网络层关联 |
|
| 流管理 |
|
3.3 实现细节
// HTTP 流创建流程RequestEncoder& CodecClient::newStream(ResponseDecoder& response_decoder){auto active_request = std::make_unique<ActiveRequest>(*this, response_decoder);RequestEncoder& encoder = codec_->newStream(response_decoder);active_request->encoder_ = &encoder;active_requests_.push_back(std::move(active_request));return encoder;}// 网络事件处理voidCodecClient::onEvent(Network::ConnectionEvent event){switch (event) {case Network::ConnectionEvent::Connected:codec_->initialize();break;case Network::ConnectionEvent::RemoteClose:case Network::ConnectionEvent::LocalClose:remote_closed_ = true;resetAllStreams();break;}}
4. HTTP 协议栈架构
4.1 整体架构图

4.2 实体关系图 (ER)

5. HTTP 与 TCP 连接的协作流程
5.1 连接建立流程图

5.2 流程说明
|
|
|
|
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6. HTTP 与 libevent 事件驱动的关联
6.1 定时器管理
// ConnectionManagerImpl 中的定时器使用void ConnectionManagerImpl::initializeTimers() {// 连接空闲超时定时器connection_idle_timer_ = dispatcher_.createTimer([this]() { this->onIdleTimeout(); });connection_idle_timer_->enableTimer(config_.idle_timeout());// 连接持续时间定时器if (config_.connection_duration_timeout().has_value()) {connection_duration_timer_ = dispatcher_.createTimer([this]() { this->onConnectionDurationTimeout(); });connection_duration_timer_->enableTimer(config_.connection_duration_timeout().value());}}// ActiveStream 中的定时器使用ActiveStream::ActiveStream(ConnectionManagerImpl& connection_manager, uint32_t buffer_limit): connection_manager_(connection_manager) {// 请求超时定时器request_timer_ = connection_manager_.dispatcher_.createTimer([this]() { this->onRequestTimeout(); });request_timer_->enableTimer(config_.request_timeout());// 请求头超时定时器request_header_timer_ = connection_manager_.dispatcher_.createTimer([this]() { this->onRequestHeaderTimeout(); });request_header_timer_->enableTimer(config_.request_header_timeout());// 流空闲超时定时器stream_idle_timer_ = connection_manager_.dispatcher_.createTimer([this]() { this->onIdleTimeout(); });stream_idle_timer_->enableTimer(config_.stream_idle_timeout());}
6.2 定时器类型说明
|
|
|
|
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6.3 文件事件使用
// CodecClient 中的连接事件处理void CodecClient::onEvent(Network::ConnectionEvent event) {switch(event) {case Network::ConnectionEvent::Connected:connected_ = true;codec_->initialize();break;case Network::ConnectionEvent::RemoteClose:remote_closed_ = true;resetAllStreams(StreamResetReason::RemoteReset,"Connection closed by remote");break;case Network::ConnectionEvent::LocalClose:resetAllStreams(StreamResetReason::LocalReset,"Connection closed locally");break;case Network::ConnectionEvent::ConnectError:handleConnectionError();break;}}
7. HTTP 协议版本支持
7.1 HTTP/1.1 实现
特点:
-
文本协议,易于调试
-
无状态协议,通过 Cookie 保持会话
-
支持流水线请求,效率不高
-
每个请求/响应都有独立的头信息
配置示例:
http1_settings:enable_trailing_headers: trueallow_chunked_length: trueenable_keep_alives: truemax_concurrent_streams: 100initial_window_size: 65536
7.2 HTTP/2 实现
特点:
-
二进制协议,效率更高
-
支持多路复用,多个流共享一个 TCP 连接
-
头部压缩(HPACK),减少带宽消耗
-
流量控制和优先级支持
配置示例:
quic_options:connection_timeout: 5smax_concurrent_connections: 1000idle_timeout: 300smax_datagram_size: 1452
7.3 HTTP/3 实现
特点:
-
基于 QUIC 协议,提供可靠的 UDP 传输
-
减少连接建立延迟(0-RTT 握手)
-
更好的多路复用性能(无队头阻塞)
-
天然的加密支持
配置示例:
quic_options:connection_timeout: 5smax_concurrent_connections: 1000idle_timeout: 300smax_datagram_size: 1452
7.4 协议对比
|
|
|
|
|
|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
8. HTTP 与 TCP 实现的区别
8.1 协议层次
|
|
|
|
|---|---|---|
| 协议层次 |
|
|
| 依赖关系 |
|
|
| 数据模型 |
|
|
8.2 数据处理
|
|
|
|
|---|---|---|
| 数据单元 |
|
|
| 应用语义 |
|
|
| 内容感知 |
|
|
8.3 连接管理
|
|
|
|
|---|---|---|
| 连接类型 |
|
|
| 建立方式 |
|
|
| 关闭方式 |
|
|
9. 性能优化与架构设计
9.1 连接池管理
class ConnectionPoolBase : public std::enable_shared_from_this<ConnectionPoolBase> {public:virtual ClientConnectionPtr connect()= 0;virtualvoidaddConnectionCallback(ConnectionCallback& callback)= 0;virtualvoidremoveConnectionCallback(ConnectionCallback& callback)= 0;};class ConnectionPoolImpl : public ConnectionPoolBase {public:ClientConnectionPtr connect()override{// 从连接池中获取或创建新连接if (!idle_connections_.empty()) {auto connection = std::move(idle_connections_.front());idle_connections_.pop_front();return connection;}if (active_connections_.size() < max_connections_) {return createNewConnection();}// 连接池已满,等待空闲连接return nullptr;}private:std::list<ClientConnectionPtr> idle_connections_;std::unordered_set<ClientConnectionPtr> active_connections_;size_t max_connections_;};
9.2 流管理优化
struct ActiveStream final : LinkedObject<ActiveStream>,public Event::DeferredDeletable,public StreamCallbacks,public RequestDecoder {voidcompleteRequest(){// 标记流完成state_.codec_saw_local_complete_ = true;// 清理定时器disarmRequestTimeout();resetIdleTimer();// 统计信息更新updateStats();// 延迟删除流,避免在回调中删除对象connection_manager_.doDeferredStreamDestroy(*this);}voidonIdleTimeout(){resetStream(StreamResetReason::IdleTimeout);}};
9.3 优化技术总结
|
|
|
|
|---|---|---|
| 连接池 |
|
|
| 流多路复用 |
|
|
| 延迟删除 |
|
|
| 定时器优化 |
|
|
| 零拷贝 |
|
|
10. HTTP 过滤器链
10.1 过滤器架构
class FilterManager : public FilterManagerCallbacks {public:FilterManager(ActiveStream& stream, ConnectionManagerImpl& connection_manager,const ConnectionManagerConfig& config);// 应用请求过滤器链voidapplyRequestFilters(RequestHeaderMapPtr&& headers);// 应用响应过滤器链voidapplyResponseFilters(ResponseHeaderMapPtr&& headers);private:std::vector<RequestDecoderFilterPtr> request_filters_;std::vector<ResponseEncoderFilterPtr> response_filters_;ActiveStream& stream_;ConnectionManagerImpl& connection_manager_;};
10.2 内置过滤器
|
|
|
|---|---|
envoy.filters.http.router |
|
envoy.filters.http.jwt_authn |
|
envoy.filters.http.rate_limit |
|
envoy.filters.http.cors |
|
envoy.filters.http.gzip |
|
envoy.filters.http.ext_authz |
|
10.3 过滤器配置示例
http_filters:# 认证过滤器- name: envoy.filters.http.jwt_authntyped_config:"@type": type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.JwtAuthenticationproviders:my_provider:issuer: example.comaudiences:- api.example.comremote_jwks:http_uri:uri: https://example.com/.well-known/jwks.jsontimeout: 5scache_duration: 300s# 速率限制过滤器- name: envoy.filters.http.rate_limittyped_config:"@type": type.googleapis.com/envoy.extensions.filters.http.rate_limit.v3.RateLimitdomain: example_domainstage: 0# 路由过滤器(必须最后)- name: envoy.filters.http.routertyped_config:"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
11. 架构关联总结
11.1 与 TCP 架构的关联
-
连接管理: HTTP 连接建立在 TCP 连接之上,使用 ConnectionImpl 管理底层连接
-
数据传输: HTTP 数据通过 TCP 连接传输,使用 Network::ReadFilter 接收数据
-
错误处理: TCP 错误会导致 HTTP 连接关闭,HTTP 协议错误也会影响 TCP 连接状态
11.2 与 libevent 架构的关联
-
事件驱动: HTTP 连接和流管理使用 event_base 进行事件调度
-
定时器: HTTP 超时(连接、请求、响应)使用 Event::Timer 实现
-
文件事件: HTTP 数据传输通过 FileEventImpl 监听 socket 事件
11.3 架构优势
|
|
|
|---|---|
| 高性能 |
|
| 可靠性 |
|
| 可扩展性 |
|
12. 配置示例
12.1 完整 HTTP 代理配置
static_resources:listeners:- name: http_listeneraddress:socket_address: { address: 0.0.0.0, port_value: 8080 }filter_chains:- filters:- name: envoy.filters.network.http_connection_managertyped_config:"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManagerstat_prefix: ingress_httpcodec_type: AUTOroute_config:name: local_routevirtual_hosts:- name: local_servicedomains: ["*"]routes:- match: { prefix: "/" }route: { cluster: "example_cluster" }http_filters:- name: envoy.filters.http.routerclusters:- name: example_clusterconnect_timeout: 0.25stype: STRICT_DNSlb_policy: ROUND_ROBINhttp2_protocol_options: {}load_assignment:cluster_name: example_clusterendpoints:- lb_endpoints:- endpoint:address:socket_address: { address: 127.0.0.1, port_value: 80 }
12.2 关键配置参数说明
|
|
|
|
|---|---|---|
codec_type |
|
|
idle_timeout |
|
|
request_timeout |
|
|
stream_idle_timeout |
|
|
max_requests_per_connection |
|
|
13. HTTP 请求完整生命周期流程
13.1 阶段概览

13.2 详细处理流程
13.2.1 请求接受阶段
// ConnectionManagerImpl 构造函数ConnectionManagerImpl::ConnectionManagerImpl(ConnectionManagerConfig& config,const Network::DrainDecision& drain_close,Random::RandomGenerator& random_generator,Http::Context& http_context, Runtime::Loader& runtime,const LocalInfo::LocalInfo& local_info,Upstream::ClusterManager& cluster_manager,Server::OverloadManager& overload_manager,TimeSource& time_source): config_(config), stats_(config_.stats()),conn_length_(new Stats::HistogramCompletableTimespanImpl(stats_.named_.downstream_cx_length_ms_, time_source)),drain_close_(drain_close), user_agent_(http_context.userAgentContext()),random_generator_(random_generator), http_context_(http_context), runtime_(runtime),local_info_(local_info), cluster_manager_(cluster_manager),listener_stats_(config_.listenerStats()),overload_state_(overload_manager.getThreadLocalOverloadState()),time_source_(time_source) {}// 初始化读取过滤器回调void ConnectionManagerImpl::initializeReadFilterCallbacks(Network::ReadFilterCallbacks& callbacks) {read_callbacks_ = &callbacks;stats_.named_.downstream_cx_total_.inc();stats_.named_.downstream_cx_active_.inc();// 创建连接超时定时器if (config_.idleTimeout()) {connection_idle_timer_ = read_callbacks_->connection().dispatcher().createScaledTimer(Event::ScaledTimerType::HttpDownstreamIdleConnectionTimeout,[this]() -> void { onIdleTimeout(); });connection_idle_timer_->enableTimer(config_.idleTimeout().value());}}
13.2.2 请求解码阶段
RequestDecoder& ConnectionManagerImpl::newStream(ResponseEncoder& response_encoder,bool is_internally_created) {if (connection_idle_timer_) {connection_idle_timer_->disableTimer();}// 创建 ActiveStream 实例auto& buffer_factory = read_callbacks_->connection().dispatcher().getWatermarkFactory();Buffer::BufferMemoryAccountSharedPtr downstream_stream_account =buffer_factory.createAccount(response_encoder.getStream());response_encoder.getStream().setAccount(downstream_stream_account);ActiveStreamPtr new_stream(new ActiveStream(*this, response_encoder.getStream().bufferLimit(),std::move(downstream_stream_account)));accumulated_requests_++;if (config_.maxRequestsPerConnection() > 0 &&accumulated_requests_ >= config_.maxRequestsPerConnection()) {if (codec_->protocol() < Protocol::Http2) {new_stream->state_.saw_connection_close_ = true;drain_state_ = DrainState::Closing;}}new_stream->response_encoder_ = &response_encoder;new_stream->response_encoder_->getStream().addCallbacks(*new_stream);LinkedList::moveIntoList(std::move(new_stream), streams_);return **streams_.begin();}
13.2.3 请求处理阶段
void ConnectionManagerImpl::ActiveStream::decodeHeaders(RequestHeaderMapPtr&& headers,bool end_stream) {request_headers_ = std::move(headers);filter_manager_.requestHeadersInitialized();// 路径规范化处理const auto action =ConnectionManagerUtility::maybeNormalizePath(*request_headers_, connection_manager_.config_);if (action == ConnectionManagerUtility::NormalizePathAction::Reject) {sendLocalReply(Code::BadRequest, "", nullptr, absl::nullopt,StreamInfo::ResponseCodeDetails::get().PathNormalizationFailed);return;}// 应用过滤器链filter_manager_.decodeHeaders(*request_headers_, end_stream);}
13.2.4 响应生成与发送
void ConnectionManagerImpl::ActiveStream::encodeHeaders(ResponseHeaderMap& headers,bool end_stream) {// 设置 Date 头部if (!headers.Date()) {connection_manager_.config_.dateProvider().setDateHeader(headers);}// 设置 Server 头部const auto transformation = connection_manager_.config_.serverHeaderTransformation();if (transformation == ConnectionManagerConfig::HttpConnectionManagerProto::OVERWRITE ||(transformation == ConnectionManagerConfig::HttpConnectionManagerProto::APPEND_IF_ABSENT &&headers.Server() == nullptr)) {headers.setReferenceServer(connection_manager_.config_.serverName());}chargeStats(headers);response_encoder_->encodeHeaders(headers, end_stream);}void ConnectionManagerImpl::ActiveStream::encodeData(Buffer::Instance& data, bool end_stream) {filter_manager_.streamInfo().addBytesSent(data.length());response_encoder_->encodeData(data, end_stream);}
13.3 关键性能优化点
|
|
|
|
|---|---|---|
| 连接持续时间控制 | max_connection_duration
|
|
| 连接空闲超时 | idle_timeout
|
|
| 流空闲超时 | stream_idle_timeout
|
|
| 请求超时控制 | request_timeout
|
|
| 延迟删除 | deferredDelete
|
|
| 协议自动检测 |
|
|
13.4 配置优化建议
http_connection_manager:# 连接管理配置idle_timeout: 300smax_connection_duration: 3600smax_requests_per_connection: 100delayed_close_timeout: 15s# 超时配置stream_idle_timeout: 60srequest_timeout: 30srequest_headers_timeout: 5smax_stream_duration: 120s# 协议配置codec_type: AUTOhttp1_settings:accept_http_10: truedefault_host_for_http_10: "example.com"http2_settings:initial_stream_window_size: 65536initial_connection_window_size: 1048576max_concurrent_streams: 256
附录
A. 错误码说明
|
|
|
|
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
B. 相关源码文件
|
|
|
|---|---|
source/common/http/conn_manager_impl.cc |
|
source/common/http/codec_client.cc |
|
source/common/http/filter_manager.cc |
|
source/common/http/http1/codec_impl.cc |
|
source/common/http/http2/codec_impl.cc |
|
source/common/http/utility.cc |
|
夜雨聆风