跳转至

DataKit 自身指标采集

· Version-1.11.0


DataKit 采集器用于自身运行指标采集,包括运行环境信息、CPU、内存占用、各个核心模块指标等。采集到的数据可用于 DataKit 内置视图、Bug Report 问题排查和运行状态归档。

配置

DataKit 启动后会暴露 Prometheus 指标dk 采集器默认随 DataKit 启动,替代了之前的 self 采集器。

基础功能:

  • 采集 DataKit 自身 CPU、内存、Goroutine、HTTP API、数据上传、Pipeline、过滤器、磁盘缓存等运行指标。
  • 通过 interval 调整采集间隔,通过 metric_types 限制采集的指标类型。
  • 支持通过 [inputs.dk.tags] 追加自定义标签。

Version-2.2.0 起,dk 默认采集除内部屏蔽指标外的全部自身指标,不再提供 metric_name_filter 配置;如需仅保留部分指标,建议在 Pipeline 中按指标名过滤。

如需关闭自身指标采集,可在 dk.conf 中设置:

[[inputs.dk]]
  enabled = false

如需调整采集间隔、指标类型、自身 Profile 采集等配置,进入 DataKit 安装目录下的 conf.d/samples 目录,复制 dk.conf.sample 并命名为 dk.conf。示例如下:

[[inputs.dk]]

  # set false to disable Datakit self-metrics collection
  enabled = true

  # keep empty to collect all types(count/gauge/summary/...)
  metric_types = []

  # collect frequency
  interval = "30s"

  # Upload Datakit runtime profiles when resource thresholds are matched. Disabled by default.
  [inputs.dk.self_profiling]
    enabled  = false # enable threshold-triggered self profiling
    interval = "10s" # interval for checking process CPU and memory
    cooldown = "5m" # minimum interval between two profile collections

    # Profiles to collect on each trigger. CPU is sampled for duration/emergency_duration;
    # other profile types are collected as snapshots.
    enabled_types      = ["cpu", "heap", "goroutine"] # cpu, heap, goroutine
    duration           = "30s"                        # CPU sample duration for normal threshold triggers
    emergency_duration = "10s"                        # CPU sample duration for emergency threshold triggers

    # Resource bases for percent thresholds. Same unit style as [resource_limit].
    cpu_cores  = 2.0  # CPU cores used as the 100% base
    mem_max_mb = 4096 # memory MiB used as the 100% base

    # Normal thresholds use the average of the latest recent_points samples.
    recent_points     = 5     # number of samples for average thresholds
    cpu_usage_percent = 80    # average CPU percent threshold, based on cpu_cores; set 0 to disable this threshold
    mem_usage_percent = 80    # average memory percent threshold, based on mem_max_mb; set 0 to disable this threshold
    mem_usage_mb      = 3072  # average RSS memory threshold in MiB; set 0 to disable this threshold

    # Emergency thresholds use the current sample.
    mem_usage_percent_emergency = 95    # current memory percent threshold, based on mem_max_mb; set 0 to disable this threshold
    mem_usage_mb_emergency      = 0     # current RSS memory threshold in MiB; set 0 to disable this threshold

    # Local queue and upload settings. Profiles are queued locally before uploading to Dataway.
    cache_path        = "dk_self_profile" # disk queue path; relative path is under DataKit cache dir
    cache_capacity_mb = 1024              # disk queue capacity in MiB
    send_timeout      = "60s"             # timeout for each upload attempt
    send_retry_count  = 4                 # max upload attempts for each queued profile payload

[inputs.dk.tags]
   # tag1 = "val-1"
   # tag2 = "val-2"

配置好后,重启 DataKit 即可。

可通过 ConfigMap 方式注入采集器配置配置 ENV_DATAKIT_INPUTS 开启采集器。

也支持以环境变量的方式修改配置参数:

  • ENV_INPUT_DK_INTERVAL

    采集间隔

    字段类型: Duration

    采集器配置字段: interval

    示例: 10s

    默认值: 30s

  • ENV_INPUT_DK_ENABLE_SELF_PROFILING

    开启 DataKit 自身阈值触发 Profile 采集

    字段类型: Boolean

    采集器配置字段: self_profiling.enabled

    示例: true

    默认值: false

指标

DataKit 自身指标主要是一些 Prometheus 指标,其文档参见这里

自身 Profile 采集

Version-2.2.0 起,dk 支持按资源阈值触发 DataKit 自身 Profile 采集。该功能默认关闭,可用于在 DataKit CPU 或内存达到阈值时采集 CPU、heap、goroutine 等 Profile。

主机部署可在 dk.conf 中开启:

[inputs.dk.self_profiling]
  enabled = true

Kubernetes 部署可通过环境变量开启:

ENV_INPUT_DK_ENABLE_SELF_PROFILING=true

更多阈值、采集类型、缓存和上报配置参见上方示例配置中的 [inputs.dk.self_profiling]

文档评价

文档内容是否对您有帮助? ×