WinNetFlow
WinNetFlow 采集器通过 Windows 事件追踪(ETW)监听 Microsoft-Windows-TCPIP Provider 的 TCP/UDP 事件,按进程统计主机四层网络流量(字节、包数、重传、RTT、TCP 状态)。核心字段和端点角色与 Linux 版 ebpf-net/netflow 兼容,可复用基于 src/dst、client/server、conn_side 和流量字段的主机网络观测页面。
同时监听 Microsoft-Windows-HttpService Provider 采集 HTTP 请求指标(httpflow,默认开启),覆盖 IIS、HttpListener、ASP.NET Core 等基于 HTTP.sys 的 Web 服务,字段与 Linux 版 ebpf-net/httpflow 兼容。
配置¶
前置条件¶
- 操作系统:Windows 10 / Windows Server 2016 及以上(64 位)
- 采集器需要以管理员权限运行(创建实时 ETW 会话需要管理员权限)
- 与 eBPF 采集器互不依赖,可独立启用;两者在同一主机上只会启用其一
进入 DataKit 安装目录下的 conf.d/samples 目录,复制 winnetflow.conf.sample 并命名为 winnetflow.conf。示例如下:
[[inputs.winnetflow]]
# Flow aggregation and report interval.
interval = "60s"
# Optional ETW session tuning. Buffer size is in KB; unsafe values are
# clamped to documented limits and a 256 MiB per-session memory budget.
# etw_buffer_size_kb = 64
# etw_min_buffers = 8
# etw_max_buffers = 256
# Max flows tracked per interval; excess new flows are dropped and counted.
# max_flows = 65536
# Collect L7 HTTP request metrics from the HTTP.sys ETW provider (IIS,
# HttpListener and other HTTP.sys clients) as the "httpflow" measurement.
# enable_httpflow = true
# Max in-flight HTTP requests tracked; excess new requests are dropped and
# counted in the periodic summary.
# max_http_requests = 65536
# Max request path length collected; longer paths are truncated and flagged.
# httpflow_path_limit = 256
[inputs.winnetflow.tags]
# some_tag = "some_value"
配置好后,重启 DataKit 即可。
可通过 ConfigMap 方式注入采集器配置 或 配置 ENV_DATAKIT_INPUTS 开启采集器。
也支持以环境变量的方式修改配置参数(需要在 ENV_DEFAULT_ENABLED_INPUTS 中加为默认采集器):
-
ENV_INPUT_WINNETFLOW_INTERVAL
采集器重复间隔时长
字段类型: Duration
采集器配置字段:
interval默认值: 10s
-
ENV_INPUT_WINNETFLOW_ETW_BUFFER_SIZE_K_B
ETW 会话缓冲区大小(KB,范围 4-1024)。
字段类型: Int
采集器配置字段:
etw_buffer_size_k_b示例: 64
-
ENV_INPUT_WINNETFLOW_ETW_MIN_BUFFERS
ETW 会话最小缓冲区数量(范围 2-4096)。
字段类型: Int
采集器配置字段:
etw_min_buffers示例: 8
-
ENV_INPUT_WINNETFLOW_ETW_MAX_BUFFERS
ETW 会话最大缓冲区数量,同时受每会话 256 MiB 内存预算限制。
字段类型: Int
采集器配置字段:
etw_max_buffers示例: 256
-
ENV_INPUT_WINNETFLOW_MAX_FLOWS
每个周期跟踪的最大并发流数,超出部分丢弃并计数。
字段类型: Int
采集器配置字段:
max_flows示例: 65536
-
ENV_INPUT_WINNETFLOW_ENABLE_HTTP_FLOW
是否启用 HTTP.sys ETW Provider 的 L7 httpflow 采集。
字段类型: Boolean
采集器配置字段:
enable_http_flow示例: true
-
ENV_INPUT_WINNETFLOW_MAX_HTTP_REQUESTS
跟踪的最大并发进行中 HTTP 请求数,超出部分丢弃并计数。
字段类型: Int
采集器配置字段:
max_http_requests示例: 65536
-
ENV_INPUT_WINNETFLOW_HTTP_FLOW_PATH_LIMIT
收集的最大请求路径长度,超长部分被截断并标记。
字段类型: Int
采集器配置字段:
http_flow_path_limit示例: 256
-
ENV_INPUT_WINNETFLOW_TAGS
自定义标签。如果配置文件有同名标签,将会覆盖它
字段类型: String
采集器配置字段:
tags示例:
'tag1=value1,tag2=value2'
指标¶
以下所有数据采集,默认会追加名为 host 的全局 tag(tag 值为 DataKit 所在主机名),也可以在配置中通过 [inputs.winnetflow.tags] 指定其它标签:
| Tags & Fields | Description |
|---|---|
| client_ip ( tag) |
Client endpoint IP address. |
| client_ip_type ( tag) |
Client IP type: private, loopback, multicast or other. |
| client_port ( tag) |
Client endpoint port. |
| conn_side ( tag) |
Local connection role: client or server. |
| direction ( tag) |
Flow direction: incoming or outgoing. |
| dst_ip ( tag) |
Destination IP address. |
| dst_ip_type ( tag) |
Destination IP type: private, loopback, multicast or other. |
| dst_nat_ip ( tag) |
Destination NAT IP; N/A because Windows ETW does not expose NAT translation. |
| dst_nat_port ( tag) |
Destination NAT port; N/A because Windows ETW does not expose NAT translation. |
| dst_port ( tag) |
Destination port. |
| family ( tag) |
IP family: IPv4 or IPv6. |
| pid ( tag) |
Process ID owning the connection. |
| process_name ( tag) |
Process name owning the connection. |
| server_ip ( tag) |
Server endpoint IP address. |
| server_ip_type ( tag) |
Server IP type: private, loopback, multicast or other. |
| server_port ( tag) |
Server endpoint port. |
| src_ip ( tag) |
Source IP address. |
| src_ip_type ( tag) |
Source IP type: private, loopback, multicast or other. |
| src_port ( tag) |
Source port. |
| transport ( tag) |
Transport protocol: tcp or udp. |
| bytes_read | Bytes read on this flow during the interval. Type: int | (gauge) Unit: digital,B |
| bytes_written | Bytes written on this flow during the interval. Type: int | (gauge) Unit: digital,B |
| client_sent | Bytes sent by the client during the interval. Type: int | (gauge) Unit: digital,B |
| packets_read | Packets read on this flow during the interval (approximate on Windows). Type: int | (gauge) Unit: count |
| packets_written | Packets/messages written during the interval; always 0 for TCP because TCPIP send events do not expose a packet count, and approximated by message count for UDP. Type: int | (gauge) Unit: count |
| retransmits | TCP retransmissions on this flow during the interval. Type: int | (gauge) Unit: count |
| rtt | Average smoothed TCP RTT in microseconds. Type: int | (gauge) Unit: time,μs |
| rtt_var | Average TCP RTT variance in microseconds. Type: int | (gauge) Unit: time,μs |
| server_sent | Bytes sent by the server during the interval. Type: int | (gauge) Unit: digital,B |
| tcp_close_wait | TCP transitions into CLOSE_WAIT state during the interval. Type: int | (gauge) Unit: count |
| tcp_closed | TCP connections closed during the interval. Type: int | (gauge) Unit: count |
| tcp_connect_attempts | TCP active connect attempts during the interval. Type: int | (gauge) Unit: count |
| tcp_connect_failures | TCP active connect failures during the interval. Type: int | (gauge) Unit: count |
| tcp_established | TCP connections established during the interval. Type: int | (gauge) Unit: count |
| tcp_last_ack | TCP transitions into LAST_ACK state during the interval. Type: int | (gauge) Unit: count |
| tcp_time_wait | TCP transitions into TIME_WAIT state during the interval. Type: int | (gauge) Unit: count |
| Tags & Fields | Description |
|---|---|
| client_ip ( tag) |
Client endpoint IP address. |
| client_ip_type ( tag) |
Client IP type: private, loopback, multicast or other. |
| client_port ( tag) |
Client endpoint port. |
| conn_side ( tag) |
Local connection role, always server for HTTP.sys. |
| direction ( tag) |
Flow direction, always incoming for the HTTP.sys server side. |
| dst_ip ( tag) |
Client IP address (remote endpoint). |
| dst_ip_type ( tag) |
Client destination IP type: private, loopback, multicast or other. |
| dst_nat_ip ( tag) |
Destination NAT IP; N/A because Windows ETW does not expose NAT translation. |
| dst_nat_port ( tag) |
Destination NAT port; N/A because Windows ETW does not expose NAT translation. |
| dst_port ( tag) |
Client port. |
| family ( tag) |
IP family: IPv4 or IPv6. |
| pid ( tag) |
Server process ID reported by HTTP.sys on the response event. |
| process_name ( tag) |
Server process name resolved from the HTTP.sys response event PID (for example, w3wp.exe). |
| server_ip ( tag) |
Server endpoint IP address. |
| server_ip_type ( tag) |
Server IP type: private, loopback, multicast or other. |
| server_port ( tag) |
Server endpoint port. |
| src_ip ( tag) |
Server IP address (local endpoint). |
| src_ip_type ( tag) |
Server source IP type: private, loopback, multicast or other. |
| src_port ( tag) |
Server port. |
| transport ( tag) |
Transport protocol, always tcp. |
| bytes_read | Bytes read on this flow during the interval (always 0; HTTP.sys does not expose request body sizes). Type: int | (gauge) Unit: digital,B |
| bytes_written | Bytes written on this flow during the interval (populated for cache-served responses). Type: int | (gauge) Unit: digital,B |
| client_sent | Bytes sent by the HTTP client; always 0 because HTTP.sys does not expose request body sizes. Type: int | (gauge) Unit: digital,B |
| count | Number of HTTP requests in this group during the interval. Type: int | (gauge) Unit: count |
| http_version | HTTP version; empty because HTTP.sys events do not carry it. Type: string | (string) Unit: N/A |
| latency | Average request processing time (receive to send complete) in nanoseconds. Type: int | (gauge) Unit: time,ns |
| method | HTTP method (GET/POST/...). Type: string | (string) Unit: N/A |
| path | Request path. Type: string | (string) Unit: N/A |
| server_sent | Bytes sent by the HTTP server during the interval. Type: int | (gauge) Unit: digital,B |
| status_code | HTTP response status code. Type: int | (gauge) Unit: N/A |
| truncated | Request path reached the configured length limit and was truncated. Type: bool | (gauge) Unit: bool |
已知限制¶
- TCP 发送事件不携带包数字段,因此 TCP 的
packets_written恒为 0;UDP 使用消息数近似包数; - UDP 无连接语义,方向通过“非临时端口上的绑定套接字”启发式判断:命中监听端口标记为
incoming,否则outgoing; - 采集启动前已存在的 TCP 连接通过本地 TCP 监听端口快照判断方向:本地端点命中监听端口时标记为
incoming,否则为outgoing;监听快照每 30 秒刷新一次; - Windows ETW 不提供网络命名空间、Kubernetes 端点元数据、DNS 域名或 NAT 转换信息;
dst_nat_ip和dst_nat_port固定输出N/A,需要的 Kubernetes 或命名空间标签可通过单独配置的全局标签补充; - 高流量场景下 ETW 实时会话可能丢事件,采集器每 10 分钟输出一次会话统计(解码/丢弃/解析错误/会话丢失计数),异常时会告警日志。
- 每个周期跟踪的流数有上限(默认 65536,可配置
max_flows),连接风暴下超出的新流会被丢弃并计入flows_skipped统计。 httpflow仅覆盖走 HTTP.sys 的 HTTP 流量(IIS / HttpListener / ASP.NET Core 等);自包含 socket 的 HTTP 服务(如部分 Go/Node 服务)不会被捕获。- HTTP.sys 事件不携带 HTTP 版本与请求体大小,
http_version恒为空、bytes_read恒为 0;bytes_written仅在缓存命中响应(event 16)时可得。 - HTTP 进程归因为尽力而为:连接事件中的 PID 属于客户端,因此会被忽略;采集器在 HTTP.sys 上报响应时立即解析服务端 PID。如果服务端在 Windows 允许读取名称前已退出,
process_name可能为unknown。 - HTTP 并发请求跟踪有上限(默认 65536,可配置
max_http_requests),超出的新请求会被丢弃并计入周期汇总日志;URL 查询参数不会被采集,以避免泄露敏感信息和产生无界指标基数;请求路径超过httpflow_path_limit(默认 256)会被截断并置truncated。