DataKit 自身指标采集
DataKit 采集器用于自身运行指标采集,包括运行环境信息、CPU、内存占用、各个核心模块指标等。采集到的数据可用于 DataKit 内置视图、Bug Report 问题排查和运行状态归档。
配置¶
DataKit 启动后会暴露 Prometheus 指标。dk 采集器默认随 DataKit 启动,替代了之前的 self 采集器。
基础功能:
- 采集 DataKit 自身 CPU、内存、Goroutine、HTTP API、数据上传、Pipeline、过滤器、磁盘缓存等运行指标。
- 通过
interval调整采集间隔,通过metric_types限制采集的指标类型。 - 支持通过
[inputs.dk.tags]追加自定义标签。
Version-2.2.0 起,dk 默认采集除内部屏蔽指标外的全部自身指标,不再提供 metric_name_filter 配置;如需仅保留部分指标,建议在 Pipeline 中按指标名过滤。
如需关闭自身指标采集,可在 dk.conf 中设置:
如需调整采集间隔、指标类型、自身 Profile 采集等配置,进入 DataKit 安装目录下的 conf.d/samples 目录,复制 dk.conf.sample 并命名为 dk.conf。示例如下:
[[inputs.dk]]
# set false to disable Datakit self-metrics collection
enabled = true
# keep empty to collect all types(count/gauge/summary/...)
metric_types = []
# collect frequency
interval = "30s"
# Upload Datakit runtime profiles when resource thresholds are matched. Disabled by default.
[inputs.dk.self_profiling]
enabled = false # enable threshold-triggered self profiling
interval = "10s" # interval for checking process CPU and memory
cooldown = "5m" # minimum interval between two profile collections
# Profiles to collect on each trigger. CPU is sampled for duration/emergency_duration;
# other profile types are collected as snapshots.
enabled_types = ["cpu", "heap", "goroutine"] # cpu, heap, goroutine
duration = "30s" # CPU sample duration for normal threshold triggers
emergency_duration = "10s" # CPU sample duration for emergency threshold triggers
# Resource bases for percent thresholds. Same unit style as [resource_limit].
cpu_cores = 2.0 # CPU cores used as the 100% base
mem_max_mb = 4096 # memory MiB used as the 100% base
# Normal thresholds use the average of the latest recent_points samples.
recent_points = 5 # number of samples for average thresholds
cpu_usage_percent = 80 # average CPU percent threshold, based on cpu_cores; set 0 to disable this threshold
mem_usage_percent = 80 # average memory percent threshold, based on mem_max_mb; set 0 to disable this threshold
mem_usage_mb = 3072 # average RSS memory threshold in MiB; set 0 to disable this threshold
# Emergency thresholds use the current sample.
mem_usage_percent_emergency = 95 # current memory percent threshold, based on mem_max_mb; set 0 to disable this threshold
mem_usage_mb_emergency = 0 # current RSS memory threshold in MiB; set 0 to disable this threshold
# Local queue and upload settings. Profiles are queued locally before uploading to Dataway.
cache_path = "dk_self_profile" # disk queue path; relative path is under DataKit cache dir
cache_capacity_mb = 1024 # disk queue capacity in MiB
send_timeout = "60s" # timeout for each upload attempt
send_retry_count = 4 # max upload attempts for each queued profile payload
[inputs.dk.tags]
# tag1 = "val-1"
# tag2 = "val-2"
配置好后,重启 DataKit 即可。
可通过 ConfigMap 方式注入采集器配置 或 配置 ENV_DATAKIT_INPUTS 开启采集器。
也支持以环境变量的方式修改配置参数:
-
ENV_INPUT_DK_INTERVAL
采集间隔
字段类型: Duration
采集器配置字段:
interval示例:
10s默认值:
30s -
ENV_INPUT_DK_ENABLE_SELF_PROFILING
开启 DataKit 自身阈值触发 Profile 采集
字段类型: Boolean
采集器配置字段:
self_profiling.enabled示例:
true默认值:
false
指标¶
DataKit 自身指标主要是一些 Prometheus 指标,其文档参见这里
自身 Profile 采集¶
Version-2.2.0 起,dk 支持按资源阈值触发 DataKit 自身 Profile 采集。该功能默认关闭,可用于在 DataKit CPU 或内存达到阈值时采集 CPU、heap、goroutine 等 Profile。
主机部署可在 dk.conf 中开启:
Kubernetes 部署可通过环境变量开启:
更多阈值、采集类型、缓存和上报配置参见上方示例配置中的 [inputs.dk.self_profiling]。