Skip to content

StatsD


The indicator data collected by the DDTrace agent will be sent to port 8125 of the DK through the StatsD data type.

This includes the JVM CPU, memory, threads, and class loading information of the JVM runtime, as well as various collected JMX indicators such as Kafka, Tomcat, RabbitMQ, etc.

Config

Preconditions

When DDTrace runs as an agent, there is no need for the user to specifically open the jmx port. If no port is opened, the agent will randomly open a local port.

DDTrace will collect JVM information by default. By default, it will be sent to 'localhost: 8125'

if k8s:

DD_JMXFETCH_STATSD_HOST=datakit_url
DD_JMXFETCH_STATSD_PORT=8125

You can use dd.jmxfetch.<INTEGRATION_NAME>.enabled=true Enable the specified collector.

for INTEGRATION_NAME, You can check the default supported third-party software before.

For example, Tomcat or Kafka:

-Ddd.jmxfetch.tomcat.enabled=true
# or
-Ddd.jmxfetch.kafka.enabled=true 

Collector Configuration

Go to the conf.d/samples directory under the DataKit installation directory, copy statsd.conf.sample and name it statsd.conf. Examples are as follows:

[[inputs.statsd]]
  ## Collector alias.
  # source = "statsd/-/-"

  ## Collect interval, default is 10 seconds. (optional)
  # interval = '10s'

  protocol = "udp"

  ## Address to host unix listener on, linux only
  service_unix_address = "/var/run/datakit/statsd.sock"

  ## Address and port to host UDP listener on
  service_address = ":8125"

  ## Tag request metric. Used for distinguish feed metric name.
  ## eg, DD_TAGS=source_key:tomcat,host_key:cn-shanghai-sq5ei
  ## eg, -Ddd.tags=source_key:tomcat,host_key:cn-shanghai-sq5ei
  # statsd_source_key = "source_key"
  # statsd_host_key   = "host_key"
  ## Indicate whether report tag statsd_source_key and statsd_host_key.
  # save_above_key    = false

  delete_gauges = true
  delete_counters = true
  delete_sets = true
  delete_timings = true

  ## Counter metric is float in new Datakit version, set true if want be int.
  # set_counter_int = false

  ## Percentiles to calculate for timing & histogram stats
  percentiles = [50.0, 90.0, 99.0, 99.9, 99.95, 100.0]

  ## separator to use between elements of a statsd metric
  metric_separator = "_"

  ## Parses tags in the datadog statsd format
  ## http://docs.datadoghq.com/guides/dogstatsd/
  parse_data_dog_tags = true

  ## Parses datadog extensions to the statsd format
  datadog_extensions = true

  ## Parses distributions metric as specified in the datadog statsd format
  ## https://docs.datadoghq.com/developers/metrics/types/?tab=distribution#definition
  datadog_distributions = true

  ## We do not need following tags(they may create tremendous of time-series under influxdb's logic)
  # Examples:
  # "runtime-id", "metric-type"
  drop_tags = [ ]

  # All metric-name prefixed with 'jvm_' are set to influxdb's measurement 'jvm'
  # All metric-name prefixed with 'stats_' are set to influxdb's measurement 'stats'
  # Examples:
  # "stats_:stats", "jvm_:jvm", "tomcat_:tomcat",
  metric_mapping = [ ]

  ## Number of UDP messages allowed to queue up, once filled,
  ## the statsd server will start dropping packets, default is 128.
  # allowed_pending_messages = 128

  ## Number of timing/histogram values to track per-measurement in the
  ## calculation of percentiles. Raising this limit increases the accuracy
  ## of percentiles but also increases the memory usage and cpu time.
  percentile_limit = 1000

  ## Max duration (TTL) for each metric to stay cached/reported without being updated.
  #max_ttl = "1000h"

  [inputs.statsd.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"

After configuration, restart DataKit.

The collector can now be turned on by configMap injection collector configuration.

Info

If find lot of Feed: io busy in the log, can configure interval='1s', minimum is 1s.

Tag data sources

If you want to tag the hosts captured by DDTrace, you can do so by injecting tags:

  • You can use environment variables, i.e. DD_TAGS, for example: DD_TAGS=source_key:tomcat,host_key:cn-shanghai-sq5ei
  • You can use the command line, i.e. dd.tags, for example:-Ddd.tags=source_key:tomcat,host_key:cn-shanghai-sq5ei

In the above example, you need to specify that the key for source is source_key and the key for host is host_key in the DataKit configuration. You can change it to something else, but you have to make sure that the field names in DataKit are the same as the field names in DDTrace.

The end result is that you can see statsd/tomcat/cn-shanghai-sq5ei in the datakit monitor, which distinguishes it from other data sources reported to the statsd collector. If the above configuration is not done, then the default display is seen on the datakit monitor: statsd/-/-.

In addition, there is a configuration switch save_above_key that determine whether the tags corresponding to statsd_source_key and statsd_host_key are reported to the center. The default is not to report(false).

Metric

For all of the following data collections, a global tag named host is appended by default (the tag value is the host name of the DataKit), or other tags can be specified in the configuration by [inputs.statsd.tags]:

 [inputs.statsd.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

jvm

Tags & Fields Description
host
(tag)
Host name.
instance
(tag)
Instance name.
jmx_domain
(tag)
JMX domain.
metric_type
(tag)
Metric type.
name
(tag)
Type name.
runtime-id
(tag)
Runtime id.
service
(tag)
Service name.
type
(tag)
Object type.
buffer_pool_direct_capacity Measure of total memory capacity of direct buffers.
Type: float | (gauge)
Unit: digital,B
buffer_pool_direct_count Number of direct buffers in the pool.
Type: float | (gauge)
Unit: count
buffer_pool_direct_used Measure of memory used by direct buffers.
Type: float | (gauge)
Unit: digital,B
buffer_pool_mapped_capacity Measure of total memory capacity of mapped buffers.
Type: float | (gauge)
Unit: digital,B
buffer_pool_mapped_count Number of mapped buffers in the pool.
Type: float | (gauge)
Unit: count
buffer_pool_mapped_used Measure of memory used by mapped buffers.
Type: float | (gauge)
Unit: digital,B
cpu_load_process Recent CPU utilization for the process.
Type: float | (gauge)
Unit: percent,percent
cpu_load_system Recent CPU utilization for the whole system.
Type: float | (gauge)
Unit: percent,percent
daemon_code_cache_used The number of daemon threads.
Type: float | (count)
Unit: count
daemon_thread_count Daemon thread count.
Type: float | (gauge)
Unit: count
gc_code_cache_used GC code cache used.
Type: float | (gauge)
Unit: count
gc_eden_size The 'eden' size in garbage collection.
Type: float | (gauge)
Unit: digital,B
gc_major_collection_count The rate of major garbage collections.
Type: float | (gauge)
Unit: count
gc_major_collection_time The fraction of time spent(rate) in major garbage collection.
Type: float | (gauge)
Unit: time,ms
gc_metaspace_size The metaspace size in garbage collection.
Type: float | (gauge)
Unit: digital,B
gc_minor_collection_count The rate of minor garbage collections.
Type: float | (gauge)
Unit: count
gc_minor_collection_time The fraction of time spent(rate) in minor garbage collection.
Type: float | (gauge)
Unit: time,ms
gc_old_gen_size The ond gen size in garbage collection.
Type: float | (gauge)
Unit: digital,B
gc_survivor_size The survivor size in garbage collection.
Type: float | (gauge)
Unit: digital,B
heap_memory The total Java heap memory used.
Type: float | (gauge)
Unit: digital,B
heap_memory_committed The total Java heap memory committed to be used.
Type: float | (gauge)
Unit: digital,B
heap_memory_init The initial Java heap memory allocated.
Type: float | (gauge)
Unit: digital,B
heap_memory_max The maximum Java heap memory available.
Type: float | (gauge)
Unit: digital,B
loaded_classes Number of classes currently loaded.
Type: float | (gauge)
Unit: count
non_heap_memory The total Java non-heap memory used. Non-heap memory is: Metaspace + CompressedClassSpace + CodeCache.
Type: float | (gauge)
Unit: digital,B
non_heap_memory_committed The total Java non-heap memory committed to be used.
Type: float | (gauge)
Unit: digital,B
non_heap_memory_init The initial Java non-heap memory allocated.
Type: float | (gauge)
Unit: digital,B
non_heap_memory_max The maximum Java non-heap memory available.
Type: float | (gauge)
Unit: digital,B
os_open_file_descriptors The number of file descriptors used by this process (only available for processes run as the dd-agent user)
Type: float | (gauge)
Unit: count
peak_thread_count The peak number of live threads.
Type: float | (count)
Unit: count
thread_count The number of live threads.
Type: float | (count)
Unit: count
total_thread_count The number of total threads.
Type: float | (count)
Unit: count

jmx

Tags & Fields Description
host
(tag)
Host name.
instance
(tag)
Instance name.
jmx_domain
(tag)
JMX domain.
metric_type
(tag)
Metric type.
name
(tag)
Type name.
runtime-id
(tag)
Runtime id.
service
(tag)
Service name.
type
(tag)
Object type.
gc_cms.count The total number of garbage collections that have occurred.
Type: float | (count)
Unit: count
gc_major_collection_count The rate of major garbage collections.
Type: float | (gauge)
Unit: count
gc_major_collection_time The fraction of time spent in major garbage collection. Set new_gc_metrics: true to receive this metric.
Type: float | (gauge)
Unit: PPM
gc_minor_collection_count The rate of minor garbage collections.
Type: float | (gauge)
Unit: count
gc_minor_collection_time The fraction of time spent in minor garbage collection. Set new_gc_metrics: true to receive this metric.
Type: float | (gauge)
Unit: PPM
gc_parnew.time The approximate accumulated garbage collection time elapsed.
Type: float | (gauge)
Unit: time,ms
heap_memory The total Java heap memory used.
Type: float | (gauge)
Unit: digital,B
heap_memory_committed The total Java heap memory committed to be used.
Type: float | (gauge)
Unit: digital,B
heap_memory_init The initial Java heap memory allocated.
Type: float | (gauge)
Unit: digital,B
heap_memory_max The maximum Java heap memory available.
Type: float | (gauge)
Unit: digital,B
non_heap_memory The total Java non-heap memory used. Non-heap memory is calculated as follows: 'Metaspace' + CompressedClassSpace + CodeCache
Type: float | (gauge)
Unit: digital,B
non_heap_memory_committed The total Java non-heap memory committed to be used.
Type: float | (gauge)
Unit: digital,B
non_heap_memory_init The initial Java non-heap memory allocated.
Type: float | (gauge)
Unit: digital,B
non_heap_memory_max The maximum Java non-heap memory available.
Type: float | (gauge)
Unit: digital,B
thread_count The number of live threads.
Type: float | (count)
Unit: count

ddtrace

Tags & Fields Description
endpoint
(tag)
Endpoint.
host
(tag)
Host name.
lang
(tag)
Lang type.
lang_interpreter
(tag)
Lang interpreter.
lang_interpreter_vendor
(tag)
Lang interpreter vendor.
lang_version
(tag)
Lang version.
metric_type
(tag)
Metric type.
priority
(tag)
Priority.
service
(tag)
Service name.
stat
(tag)
Stat.
tracer_version
(tag)
Tracer version.
tracer_agent_discovery_time Tracer agent discovery time.
Type: float | (gauge)
Unit: time,ms
tracer_api_errors_total Tracer api errors total.
Type: float | (gauge)
Unit: count
tracer_api_requests_total Tracer api requests total.
Type: float | (gauge)
Unit: count
tracer_flush_bytes_total Tracer flush bytes total.
Type: float | (gauge)
Unit: count
tracer_flush_traces_total Tracer flush traces total.
Type: float | (gauge)
Unit: count
tracer_queue_enqueued_bytes Tracer queue enqueued bytes.
Type: float | (gauge)
Unit: count
tracer_queue_enqueued_spans Tracer queue enqueued spans.
Type: float | (gauge)
Unit: count
tracer_queue_enqueued_traces Tracer queue enqueued traces.
Type: float | (gauge)
Unit: count
tracer_queue_max_length Tracer queue max length.
Type: float | (gauge)
Unit: count
tracer_scope_activate_count Tracer scope activate count.
Type: float | (gauge)
Unit: count
tracer_scope_close_count Tracer scope close count.
Type: float | (gauge)
Unit: count
tracer_span_pending_created Tracer span pending created.
Type: float | (gauge)
Unit: count
tracer_span_pending_finished Tracer span pending finished.
Type: float | (gauge)
Unit: count
tracer_trace_agent_discovery_time Tracer trace agent discovery time.
Type: float | (gauge)
Unit: count
tracer_trace_agent_send_time Tracer trace agent send time.
Type: float | (gauge)
Unit: count
tracer_trace_pending_created Tracer trace pending created.
Type: float | (gauge)
Unit: count
tracer_tracer_trace_buffer_fill_time Tracer trace buffer fill time.
Type: float | (gauge)
Unit: count

Feedback

Is this page helpful? ×