CloudFlare LogPush
本篇主要介绍配置和接收 LogPush 的日志到观测云,目前支持 LogPush 的数据类型有:
- HTTP Requests
- DNS Logs
在配置 HTTP Destination 时有三个请求参数需要填写:
- source: 作为中心的数据源标签,目前支持两种:
http_requests,dns_logs - service: 作为服务标签,建议使用域名,例如:
service=domain.com - token: 请求token。
HTTP Requests¶
http 请求日志,在配置目标地址时,要配置请求的类型: "source=http_requests"
创建作业¶
- 登录 Cloudflare 仪表板 → 选择目标网站 → 导航至 Analytics → Logs → 点击 Add Logpush Job。
- 选择数据集:HTTP Requests
- 配置目标:HTTP Destination
- 填写路径:
https://<endpoint>/v1/write/cf-logpush?source=http_requests&domain=<domain.com>&token=tkn_abcxxx - 作业名称:符合格式即可
- 发送字段:用户可以自定义选择,只要不为空的字段都会解析到一级字段。
- 高级选项:选择时间格式为纳秒单位
unixnano。
注意:填写目标路径的时候一定要填写 source,domain,token 否测数据无法按照类型进行筛选,高级选项一定要使用纳秒时间单位。
完整的字段有98个,这里是常用字段展示(部分):
| 字段 | 中心字段 | 说明 |
|---|---|---|
| ClientIP | ClientIP | client ip |
| EdgeResponseStatus | EdgeResponseStatus | HTTP status code returned by Cloudflare to the client. |
| - | duration | request duration,nano |
| - | country | country name |
| RayID | RayID | Identifier of the request |
| ClientCity | ClientCity | Approximate city of the client. |
| ClientCountry | ClientCountry | 2-letter ISO-3166 country code of the client IP address. |
| ClientRegionCode | ClientRegionCode | The ISO-3166-2 region code of the client IP address. |
| ClientRequestHost | ClientRequestHost | Host requested by the client. |
| ClientRequestMethod | ClientRequestMethod | HTTP method of client request. |
| ClientRequestURI | ClientRequestURI | URI requested by the client. |
| ClientDeviceType | ClientDeviceType | Client device type |
| ClientRequestBytes | ClientRequestBytes | Number of bytes in the client request. |
| ClientRequestPath | ClientRequestPath | URI path requested by the client. |
| ClientRequestProtocol | ClientRequestProtocol | HTTP protocol of client request. |
| ClientRequestScheme | ClientRequestScheme | The URL scheme requested by the visitor. |
| ClientRequestUserAgent | ClientRequestUserAgent | User agent reported by the client. |
| EdgeTimeToFirstByteMs | EdgeTimeToFirstByteMs | Total view of Time To First Byte as measured at Cloudflare's edge. |
| EdgeResponseBodyBytes | EdgeResponseBodyBytes | Size of the HTTP response body returned to clients. |
| EdgeResponseBytes | EdgeResponseBytes | Number of bytes returned by the edge to the client. |
| - | message | log json |
DNS 请求日志¶
操作步骤与 http 请求类似。但是注意,在配置目标地址时,要配置请求类型:"source=dns_logs"
创建作业¶
- 登录 Cloudflare 仪表板 → 选择目标网站 → 导航至 Analytics → Logs → 点击 Add Logpush Job。
- 选择数据集:DNS Logs.
- 配置目标:HTTP Destination
- 填写路径:
https://<endpoint>/v1/write/cf-logpush?source=dns_logs&domain=<domain.com>&token=tkn_abcxxx - 作业名称:符合格式即可
- 发送字段:可以全选,目前中心全部支持解析到一级字段。
- 高级选项:选择时间格式为纳秒单位。
注意:填写目标路径的时候一定要填写 source,domain,token 否测数据无法按照类型进行筛选,高级选项一定要使用纳秒时间单位。
选择字段的时候请查看目前支持的字段:
| Cloudflare Field | Target Field | Description |
|---|---|---|
| ColoCode | ColoCode | IATA airport code of the data center that received the request. |
| EDNSSubnet | EDNSSubnet | IPv4 or IPv6 address information corresponding to the EDNS Client Subnet (ECS) forwarded by recursive resolvers. Not all resolvers send this information. |
| EDNSSubnetLength | EDNSSubnetLength | Size of the EDNS Client Subnet (ECS) in bits. For example, if the last octet of an IPv4 address is omitted (192.0.2.x.), the subnet length will be 24. |
| QueryName | QueryName | Name of the query that was sent. |
| QueryType | QueryType | Integer value of query type. For more information refer to Query type |
| - | query_type | String value of query type:A,AAAA,NS,CNAME,SOA,PTR,MX,TXT,DNSKEY,HTTPS.other is "unknown" |
| ResponseCached | ResponseCached | Whether the response was cached or not. |
| ResponseCode | ResponseCode | Integer value of response code. For more information refer to Response code ↗. |
| SourceIP | SourceIP | IP address of the client (IPv4 or IPv6). |
| Timestamp | time_ns | Timestamp at which the query occurred. |
其他¶
- 建议开启采样。
- 如果需要添加、修改、删除字段,可以通过 pipeline 实现。
- API 同样也可以实现页面配置,具体请查看 LogPush API