InfluxDB
The InfluxDB collector is used to collect the data of the InfluxDB.
Configuration¶
Preconditions¶
The influxdb collector is only applicable to influxdb v1.x, and the prom collector is required for influxdb v2.x.
Already tested version:
- 1.8.10
Collector Configuration¶
Go to the conf.d/influxdb directory under the DataKit installation directory, copy influxdb.conf.sample and name it influxdb.conf. Examples are as follows:
[[inputs.influxdb]]
url = "http://localhost:8086/debug/vars"
## (optional) collect interval, default is 10 seconds
interval = '10s'
## Username and password to send using HTTP Basic Authentication.
# username = ""
# password = ""
## http request & header timeout
timeout = "5s"
## Set true to enable election
election = true
## TLS config
# [inputs.influxdb.tlsconf]
# insecure_skip_verify = true
## Following ca_certs/cert/cert_key are optional, if insecure_skip_verify = true.
# ca_certs = ["/opt/tls/ca.crt"]
# cert = "/opt/tls/client.root.crt"
# cert_key = "/opt/tls/client.root.key"
## we can encode these file content in base64 format:
# ca_certs_base64 = ["LONG_BASE64_STRING......"]
# cert_base64 = "LONG_BASE64_STRING......"
# cert_key_base64 = "LONG_BASE64_STRING......"
# server_name = "your-SNI-name"
# [inputs.influxdb.log]
# files = []
# #grok pipeline script path
# pipeline = "influxdb.p"
[inputs.influxdb.tags]
# some_tag = "some_value"
# more_tag = "some_other_value"
Once configured, restart DataKit.
The collector can now be turned on by ConfigMap injection collector configuration.
InfluxDB v2.x¶
[[inputs.prom]]
## Exporter address
url = "http://127.0.0.1:8086/metrics"
metric_types = ["counter", "gauge"]
interval = "10s"
## TLS configuration.
tls_open = false
# tls_ca = "/tmp/ca.crt"
# tls_cert = "/tmp/peer.crt"
# tls_key = "/tmp/peer.key"
[[inputs.prom.measurements]]
prefix = "boltdb_"
name = "influxdb_v2_boltdb"
[[inputs.prom.measurements]]
prefix = "go_"
name = "influxdb_v2_go"
## Histogram type.
# [[inputs.prom.measurements]]
# prefix = "http_api_request_"
# name = "influxdb_v2_http_request"
[[inputs.prom.measurements]]
prefix = "influxdb_"
name = "influxdb_v2"
[[inputs.prom.measurements]]
prefix = "service_"
name = "influxdb_v2_service"
[[inputs.prom.measurements]]
prefix = "task_"
name = "influxdb_v2_task"
## Customize tags.
[inputs.prom.tags]
# some_tag = "some_value"
# more_tag = "some_other_value"
Metric¶
For all of the following data collections, the global election tags will added automatically, we can add extra tags in [inputs.influxdb.tags] if needed:
influxdb_cq¶
- Tags
| Tag | Description |
|---|---|
| host | System hostname. |
- Metrics
| Metric | Description |
|---|---|
| query_fail | The total number of continuous queries that executed but failed. Type: float Unit: - |
| query_ok | The total number of continuous queries that executed successfully. Type: float Unit: - |
influxdb_database¶
- Tags
| Tag | Description |
|---|---|
| database | Database name. |
| host | System hostname. |
- Metrics
| Metric | Description |
|---|---|
| num_measurements | The current number of measurements in the specified database. Type: float Unit: - |
| num_series | The current series cardinality of the specified database. Type: float Unit: - |
influxdb_httpd¶
- Tags
| Tag | Description |
|---|---|
| bind | Bind port. |
| host | System hostname. |
- Metrics
| Metric | Description |
|---|---|
| auth_fail | The number of HTTP requests that were aborted due to authentication being required, but not supplied or incorrect. Type: float Unit: - |
| client_error | The number of HTTP responses due to client errors, with a 4XX HTTP status code. Type: float Unit: - |
| flux_query_req | The number of Flux query requests served. Type: float Unit: - |
| flux_query_req_duration_ns | The duration (wall-time), in nanoseconds, spent executing Flux query requests. Type: float Unit: time,ns |
| ping_req | The number of times InfluxDB HTTP server served the /ping HTTP endpoint. Type: float Unit: - |
| points_written_dropped | The number of points dropped by the storage engine. Type: float Unit: - |
| points_written_fail | The number of points accepted by the HTTP /write endpoint, but unable to be persisted. Type: float Unit: - |
| points_written_ok | The number of points successfully accepted and persisted by the HTTP /write endpoint. Type: float Unit: - |
| prom_read_req | The number of read requests to the Prometheus /read endpoint. Type: float Unit: - |
| prom_write_req | The number of write requests to the Prometheus /write endpoint. Type: float Unit: - |
| query_req | The number of query requests. Type: float Unit: - |
| query_req_duration_ns | The total query request duration, in nanosecond (ns). Type: float Unit: time,ns |
| query_resp_bytes | The total number of bytes returned in query responses. Type: float Unit: digital,B |
| recovered_panics | The total number of panics recovered by the HTTP handler. Type: float Unit: - |
| req | The total number of HTTP requests served. Type: float Unit: - |
| req_active | The number of currently active requests. Type: float Unit: - |
| req_duration_ns | The duration (wall time), in nanoseconds, spent inside HTTP requests. Type: float Unit: time,ns |
| server_error | The number of HTTP responses due to server errors. Type: float Unit: - |
| status_req | The number of status requests served using the HTTP /status endpoint. Type: float Unit: - |
| values_written_ok | The number of values (fields) successfully accepted and persisted by the HTTP /write endpoint. Type: float Unit: - |
| write_req | The number of write requests served using the HTTP /write endpoint. Type: float Unit: - |
| write_req_active | The number of currently active write requests. Type: float Unit: - |
| write_req_bytes | The total number of bytes of line protocol data received by write requests, using the HTTP /write endpoint. Type: float Unit: digital,B |
| write_req_duration_ns | The duration (wall time), in nanoseconds, of write requests served using the /write HTTP endpoint. Type: float Unit: time,ns |
influxdb_memstats¶
- Tags
| Tag | Description |
|---|---|
| host | System hostname. |
- Metrics
| Metric | Description |
|---|---|
| alloc | The currently allocated number of bytes of heap objects. Type: float Unit: digital,B |
| buck_hash_sys | The bytes of memory in profiling bucket hash tables. Type: float Unit: digital,B |
| frees | The cumulative number of freed (live) heap objects. Type: float Unit: - |
| gc_cpu_fraction | The fraction of CPU time used by the garbage collection cycle. Type: float Unit: - |
| gc_sys | The bytes of memory in garbage collection metadata. Type: float Unit: digital,B |
| heap_alloc | The size, in bytes, of all heap objects. Type: float Unit: digital,B |
| heap_idle | The number of bytes of idle heap objects. Type: float Unit: digital,B |
| heap_inuse | The number of bytes in in-use spans. Type: float Unit: digital,B |
| heap_objects | The number of allocated heap objects. Type: float Unit: - |
| heap_released | The number of bytes of physical memory returned to the OS. Type: float Unit: digital,B |
| heap_sys | The number of bytes of heap memory obtained from the OS. Type: float Unit: digital,B |
| last_gc | Time the last garbage collection finished, as nanoseconds since 1970 (the UNIX epoch). Type: float Unit: timeStamp,nsec |
| lookups | The number of pointer lookups performed by the runtime. Type: float Unit: - |
| mallocs | The total number of heap objects allocated. Type: float Unit: - |
| mcache_inuse | The bytes of allocated mcache structures. Type: float Unit: digital,B |
| mcache_sys | The bytes of memory obtained from the OS for mcache structures. Type: float Unit: digital,B |
| mspan_inuse | The bytes of allocated mcache structures. Type: float Unit: digital,B |
| mspan_sys | The bytes of memory obtained from the OS for mspan.Type: float Unit: digital,B |
| next_gc | The target heap size of the next garbage collection cycle. Type: float Unit: - |
| num_forced_gc | The number of GC cycles that were forced by the application calling the GC function. Type: float Unit: - |
| num_gc | The number of completed garbage collection cycles. Type: float Unit: - |
| other_sys | The number of bytes of memory used other than heap_sys/stacks_sys/mspan_sys/mcache_sys/buckhash_sys/gc_sys.Type: float Unit: digital,B |
| pause_ns | The time garbage collection cycles are paused in nanoseconds. Type: float Unit: time,ns |
| pause_total_ns | The total time garbage collection cycles are paused in nanoseconds. Type: float Unit: time,ns |
| stack_inuse | The number of bytes in in-use stacks. Type: float Unit: digital,B |
| stack_sys | The total number of bytes of memory obtained from the stack in use. Type: float Unit: digital,B |
| sys | The cumulative bytes allocated for heap objects. Type: float Unit: digital,B |
| total_alloc | The cumulative bytes allocated for heap objects. Type: float Unit: digital,B |
influxdb_queryExecutor¶
- Tags
| Tag | Description |
|---|---|
| host | System hostname. |
- Metrics
| Metric | Description |
|---|---|
| queries_active | The number of active queries currently being handled. Type: float Unit: - |
| queries_executed | The number of queries executed (started). Type: float Unit: - |
| queries_finished | The number of queries that have finished executing. Type: float Unit: - |
| query_duration_ns | The duration (wall time), in nanoseconds, of every query executed. Type: float Unit: time,ns |
| recovered_panics | The number of panics recovered by the Query Executor. Type: float Unit: - |
influxdb_runtime¶
- Tags
| Tag | Description |
|---|---|
| host | System hostname. |
- Metrics
| Metric | Description |
|---|---|
| alloc | The currently allocated number of bytes of heap objects. Type: float Unit: digital,B |
| frees | The cumulative number of freed (live) heap objects. Type: float Unit: - |
| heap_alloc | The size, in bytes, of all heap objects. Type: float Unit: digital,B |
| heap_idle | The number of bytes of idle heap objects. Type: float Unit: digital,B |
| heap_inuse | The number of bytes in in-use spans. Type: float Unit: digital,B |
| heap_objects | The number of allocated heap objects. Type: float Unit: - |
| heap_released | The number of bytes of physical memory returned to the OS. Type: float Unit: digital,B |
| heap_sys | The number of bytes of heap memory obtained from the OS. Type: float Unit: digital,B |
| lookups | The number of pointer lookups performed by the runtime. Type: float Unit: - |
| mallocs | The total number of heap objects allocated. Type: float Unit: - |
| num_gc | The number of completed garbage collection cycles. Type: float Unit: - |
| num_goroutine | The total number of Go routines. Type: float Unit: - |
| pause_total_ns | The total time garbage collection cycles are paused in nanoseconds. Type: float Unit: time,ns |
| sys | The cumulative bytes allocated for heap objects. Type: float Unit: digital,B |
| total_alloc | The cumulative bytes allocated for heap objects. Type: float Unit: digital,B |
influxdb_shard¶
- Tags
| Tag | Description |
|---|---|
| database | Database name. |
| engine | Engine. |
| host | System hostname. |
| id | ID. |
| index_type | Index type. |
| path | Path. |
| retention_policy | Retention policy. |
| wal_path | Wal path. |
- Metrics
| Metric | Description |
|---|---|
| disk_bytes | The size, in bytes, of the shard, including the size of the data directory and the WAL directory. Type: float Unit: digital,B |
| fields_create | The number of fields created. Type: float Unit: - |
| series_create | Then number of series created. Type: float Unit: - |
| write_bytes | The number of bytes written to the shard. Type: float Unit: digital,B |
| write_points_dropped | The number of requests to write points t dropped from a write. Type: float Unit: - |
| write_points_err | The number of requests to write points that failed to be written due to errors. Type: float Unit: - |
| write_points_ok | The number of points written successfully. Type: float Unit: - |
| write_req | The total number of write requests. Type: float Unit: - |
| write_req_err | The total number of write requests that failed due to errors. Type: float Unit: - |
| write_req_ok | The total number of successful write requests. Type: float Unit: - |
| write_values_ok | The number of write values successfully. Type: float Unit: - |
influxdb_subscriber¶
- Tags
| Tag | Description |
|---|---|
| host | System hostname. |
- Metrics
| Metric | Description |
|---|---|
| create_failures | The number of subscriptions that failed to be created. Type: float Unit: - |
| points_written | The total number of points that were successfully written to subscribers. Type: float Unit: - |
| write_failures | The total number of batches that failed to be written to subscribers. Type: float Unit: - |
influxdb_tsm1_cache¶
- Tags
| Tag | Description |
|---|---|
| database | Database name. |
| engine | Engine. |
| host | System hostname. |
| id | ID. |
| index_type | Index type. |
| path | Path. |
| retention_policy | Retention policy. |
| wal_path | Wal path. |
- Metrics
| Metric | Description |
|---|---|
| cache_age_ms | The duration, in milliseconds, since the cache was last snapshotted at sample time. Type: float Unit: time,ms |
| cached_bytes | The total number of bytes that have been written into snapshots. Type: float Unit: digital,B |
| disk_bytes | The size, in bytes, of on-disk snapshots. Type: float Unit: digital,B |
| mem_bytes | The size, in bytes, of in-memory cache. Type: float Unit: digital,B |
| snapshot_count | The current level (number) of active snapshots. Type: float Unit: - |
| wal_compaction_time_ms | The duration, in milliseconds, that the commit lock is held while compacting snapshots. Type: float Unit: time,ms |
| write_dropped | The total number of writes dropped due to timeouts. Type: float Unit: - |
| write_err | The total number of writes that failed. Type: float Unit: - |
| write_ok | The total number of successful writes. Type: float Unit: - |
influxdb_tsm1_engine¶
- Tags
| Tag | Description |
|---|---|
| database | Database name. |
| engine | Engine. |
| host | System hostname. |
| id | ID. |
| index_type | Index type. |
| path | Path. |
| retention_policy | Retention policy. |
| wal_path | Wal path. |
- Metrics
| Metric | Description |
|---|---|
| cache_compaction_duration | The duration (wall time), in nanoseconds, spent in cache compactions. Type: float Unit: time,ns |
| cache_compaction_err | The number of cache compactions that have failed due to errors. Type: float Unit: - |
| cache_compactions | The total number of cache compactions that have ever run. Type: float Unit: - |
| cache_compactions_active | The number of cache compactions that are currently running. Type: float Unit: - |
| tsm_full_compaction_duration | The duration (wall time), in nanoseconds, spent in full compactions. Type: float Unit: - |
| tsm_full_compaction_err | The total number of TSM full compactions that have failed due to errors. Type: float Unit: - |
| tsm_full_compaction_queue | The current number of pending TMS Full compactions. Type: float Unit: - |
| tsm_full_compactions | The total number of TSM full compactions that have ever run. Type: float Unit: - |
| tsm_full_compactions_active | The number of TSM full compactions currently running. Type: float Unit: - |
| tsm_level1_compaction_duration | The duration (wall time), in nanoseconds, spent in TSM level 1 compactions. Type: float Unit: time,ns |
| tsm_level1_compaction_err | The total number of TSM level 1 compactions that have failed due to errors. Type: float Unit: - |
| tsm_level1_compaction_queue | The current number of pending TSM level 1 compactions. Type: float Unit: - |
| tsm_level1_compactions | The total number of TSM level 1 compactions that have ever run. Type: float Unit: - |
| tsm_level1_compactions_active | The number of TSM level 1 compactions that are currently running. Type: float Unit: - |
| tsm_level2_compaction_duration | The duration (wall time), in nanoseconds, spent in TSM level 2 compactions. Type: float Unit: time,ns |
| tsm_level2_compaction_err | The number of TSM level 2 compactions that have failed due to errors. Type: float Unit: - |
| tsm_level2_compaction_queue | The current number of pending TSM level 2 compactions. Type: float Unit: - |
| tsm_level2_compactions | The total number of TSM level 2 compactions that have ever run. Type: float Unit: - |
| tsm_level2_compactions_active | The number of TSM level 2 compactions that are currently running. Type: float Unit: - |
| tsm_level3_compaction_duration | The duration (wall time), in nanoseconds, spent in TSM level 3 compactions. Type: float Unit: time,ns |
| tsm_level3_compaction_err | The number of TSM level 3 compactions that have failed due to errors. Type: float Unit: - |
| tsm_level3_compaction_queue | The current number of pending TSM level 3 compactions. Type: float Unit: - |
| tsm_level3_compactions | The total number of TSM level 3 compactions that have ever run. Type: float Unit: - |
| tsm_level3_compactions_active | The number of TSM level 3 compactions that are currently running. Type: float Unit: - |
| tsm_optimize_compaction_duration | The duration (wall time), in nanoseconds, spent during TSM optimize compactions. Type: float Unit: time,ns |
| tsm_optimize_compaction_err | The total number of TSM optimize compactions that have failed due to errors. Type: float Unit: - |
| tsm_optimize_compaction_queue | The current number of pending TSM optimize compactions. Type: float Unit: - |
| tsm_optimize_compactions | The total number of TSM optimize compactions that have ever run. Type: float Unit: - |
| tsm_optimize_compactions_active | The number of TSM optimize compactions that are currently running. Type: float Unit: - |
influxdb_tsm1_filestore¶
- Tags
| Tag | Description |
|---|---|
| database | Database name. |
| engine | Engine. |
| host | System hostname. |
| id | ID. |
| index_type | Index type. |
| path | Path. |
| retention_policy | Retention policy. |
| wal_path | Wal path. |
- Metrics
| Metric | Description |
|---|---|
| disk_bytes | The size, in bytes, of disk usage by the TSM file store. Type: float Unit: digital,B |
| num_files | The total number of files in the TSM file store. Type: float Unit: - |
influxdb_tsm1_wal¶
- Tags
| Tag | Description |
|---|---|
| database | Database name. |
| engine | Engine. |
| host | System hostname. |
| id | ID. |
| index_type | Index type. |
| path | Path. |
| retention_policy | Retention policy. |
| wal_path | Wal path. |
- Metrics
| Metric | Description |
|---|---|
| current_segment_disk_bytes | The current size, in bytes, of the segment disk. Type: float Unit: digital,B |
| old_segments_disk_bytes | The size, in bytes, of the segment disk. Type: float Unit: digital,B |
| write_err | The number of writes that failed due to errors. Type: float Unit: - |
| write_ok | The number of writes that succeeded. Type: float Unit: - |
influxdb_write¶
- Tags
| Tag | Description |
|---|---|
| host | System hostname. |
- Metrics
| Metric | Description |
|---|---|
| point_req | The total number of every point requested to be written to this data node. Type: float Unit: - |
| point_req_local | The total number of point requests that have been attempted to be written into a shard on the same (local) node. Type: float Unit: - |
| req | The total number of batches of points requested to be written to this node. Type: float Unit: - |
| sub_write_drop | The total number of batches of points that failed to be sent to the subscription dispatcher. Type: float Unit: - |
| sub_write_ok | The total number of batches of points that were successfully sent to the subscription dispatcher. Type: float Unit: - |
| write_drop | The total number of write requests for points that have been dropped due to timestamps not matching any existing retention policies. Type: float Unit: - |
| write_error | The total number of batches of points that were not successfully written, due to a failure to write to a local or remote shard. Type: float Unit: - |
| write_ok | The total number of batches of points written at the requested consistency level. Type: float Unit: - |
| write_timeout | The total number of write requests that failed to complete within the default write timeout duration. Type: float Unit: - |
Log Collection¶
To collect the InfluxDB log, open files in influxdb.conf and write to the absolute path of the InfluxDB log file. For example: