DataKit Self-Monitoring Metrics Collection

· Version-1.11.0

The DataKit collector is used to collect basic information about its own operation, including runtime environment information, CPU usage, memory usage, and various core module metrics.

Configuration¶

After starting DataKit, it will expose some Prometheus Metrics by default. No additional actions are required, and this collector is enabled by default, replacing the previous self collector.

Host DeploymentKubernetes

Navigate to the conf.d/host directory under the DataKit installation directory, copy dk.conf.sample, and rename it to dk.conf. An example is shown below:

[[inputs.dk]]

  # See https://docs.guance.com/datakit/datakit-metrics/#metrics for all metrics exported by DataKit.
  metric_name_filter = [
    ### Collect all metrics (these may collect over 300 metrics of DataKit)
    ### If you want to collect all, make this rule the first in the list.
    # ".*",

    "datakit_http.*",       # HTTP API
    "datakit_goroutine.*",  # Goroutine

    ### runtime related
    "datakit_cpu_.*",
    "datakit_.*_alloc_bytes", # Memory
    "datakit_open_files",
    "datakit_uptime_seconds",
    "datakit_data_overuse",
    "datakit_process_.*",

    ### election
    "datakit_election_status",

    ### Dataway related
    #"datakit_io_dataway_.*",
    #"datakit_io_http_retry_total",

    ### Filter
    #"datakit_filter_.*",

    ### dialtesting
    #"datakit_dialtesting_.*",

    ### Input feed
    #".*_feed_.*",
  ]

  # keep empty to collect all types(count/gauge/summary/...)
  metric_types = []

  # collection frequency
  interval = "30s"

[inputs.dk.tags]
   # tag1 = "val-1"
   # tag2 = "val-2"

After configuring, restart DataKit.

You can inject the collector configuration via ConfigMap or configure ENV_DATAKIT_INPUTS to enable the collector.

It also supports modifying configuration parameters via environment variables (you need to add it as a default collector in ENV_DEFAULT_ENABLED_INPUTS):

ENV_INPUT_DK_ENABLE_ALL_METRICS

Collect all metrics; any non-empty string

Field Type: Boolean

Collector Configuration Field: -

Example: true

Default Value: -
ENV_INPUT_DK_ADD_METRICS

Append the list of metrics; available metric names can be found here

Field Type: List

Collector Configuration Field: -

Example: ["datakit_io_.*", "datakit_pipeline_.*"]

Default Value: -
ENV_INPUT_DK_ONLY_METRICS

Enable only specified metrics

Field Type: List

Collector Configuration Field: -

Example: ["datakit_io_.*", "datakit_pipeline_.*"]

Default Value: -

Metrics¶

DataKit's self-monitoring metrics are primarily Prometheus metrics. Refer to the documentation here for more details.

DataKit Self-Monitoring Metrics Collection

Configuration¶

Metrics¶

Is this page helpful? ×