Doris
Doris collector uses the MySQL-compatible protocol on FE Query Port to collect Doris database objects and custom SQL metrics. To collect full FE/BE Prometheus metrics, keep using the Prometheus collector configuration in the sample.
Configuration¶
Already tested version:
- 2.0.0
Preconditions¶
Connect to FE Query Port with a Doris admin account and create the datakit collection account:
Use the datakit account to check the FE SQL connection:
After connecting, run the following SQL statements to verify the data required by object collection:
If custom queries are configured, make sure the datakit account has query privileges on the related tables or views.
Collector Configuration¶
Go to the conf.d/samples directory under the DataKit installation directory, copy doris.conf.sample and name it doris.conf. Examples are as follows:
[[inputs.doris]]
## Doris FE SQL endpoint.
host = "127.0.0.1"
port = 9030
## Doris user and password.
user = "datakit"
password = "<PASS>"
## Gathering interval.
interval = "10s"
## Connection timeout for Doris SQL and metrics endpoints.
connect_timeout = "10s"
## Set true to enable election.
election = true
## FE metrics URL. Used to collect qps and avg_query_time for database object.
fe_metric_url = "http://127.0.0.1:8030/metrics"
## TLS Config for FE metrics endpoint. Used when fe_metric_url is HTTPS.
# [inputs.doris.metric_tls]
# tls_ca = "/etc/doris/metric-ca.pem"
# tls_cert = "/etc/doris/metric-cert.pem"
# tls_key = "/etc/doris/metric-key.pem"
## Use TLS but skip chain & host verification
# insecure_skip_verify = true
## by default, support TLS 1.2 and above.
## set to true if server side uses TLS 1.0 or TLS 1.1
# allow_tls10 = false
## collect Doris object
[inputs.doris.object]
## Set true to enable Doris object collection.
enabled = true
## Interval to collect Doris object which will be greater than collection interval.
interval = "600s"
## TLS Config
# [inputs.doris.tls]
# tls_ca = "/etc/doris/ca.pem"
# tls_cert = "/etc/doris/cert.pem"
# tls_key = "/etc/doris/key.pem"
## Use TLS but skip chain & host verification
# insecure_skip_verify = true
## by default, support TLS 1.2 and above.
## set to true if server side uses TLS 1.0 or TLS 1.1
# allow_tls10 = false
## Run a custom SQL query and collect corresponding metrics.
# [[inputs.doris.custom_queries]]
# sql = '''
# SELECT
# TABLE_SCHEMA AS table_schema,
# COUNT(*) AS table_count
# FROM information_schema.tables
# GROUP BY TABLE_SCHEMA
# '''
# metric = "doris_custom"
# tags = ["table_schema"]
# fields = ["table_count"]
# interval = "10s"
[inputs.doris.tags]
# some_tag = "some_value"
# more_tag = "some_other_value"
## The following Prometheus config collects full Doris FE/BE metrics.
## Keep it when you still want the existing doris_fe/doris_be/doris_common/doris_jvm metrics.
[[inputs.prom]]
## Collector alias.
source = "doris"
## (Optional) Collect interval: (defaults to "30s").
# interval = "15s"
## Exporter URLs.
urls = ["http://127.0.0.1:8030/metrics","http://127.0.0.1:8040/metrics"]
## Stream Size.
## The source stream segmentation size.
## Default 1, source stream undivided.
stream_size = 0
## TLS configuration.
tls_open = false
# tls_ca = "/tmp/ca.crt"
# tls_cert = "/tmp/peer.crt"
# tls_key = "/tmp/peer.key"
## Set to 'true' to enable election.
election = true
## disable setting host tag for this input
disable_host_tag = false
## disable setting instance tag for this input
disable_instance_tag = false
## Measurement name.
## If measurement_name is empty, split metric name by '_', the first field after split as measurement set name, the rest as current metric name.
## If measurement_name is not empty, using this as measurement set name.
## Always add 'measurement_prefix' prefix at last.
measurement_name = "doris_common"
## Customize measurement set name.
## Treat those metrics with prefix as one set.
## Prioritier over 'measurement_name' configuration.
[[inputs.prom.measurements]]
prefix = "doris_fe_"
name = "doris_fe"
[[inputs.prom.measurements]]
prefix = "doris_be_"
name = "doris_be"
[[inputs.prom.measurements]]
prefix = "jvm_"
name = "doris_jvm"
## Customize tags.
# [inputs.prom.tags]
# some_tag = "some_value"
# more_tag = "some_other_value"
After configuration, restart DataKit.
Can be turned on by ConfigMap Injection Collector Configuration or Config ENV_DATAKIT_INPUTS .
Custom Query¶
Use [[inputs.doris.custom_queries]] to run custom SQL and report the query result as metrics.
[[inputs.doris.custom_queries]]
sql = '''
SELECT
TABLE_SCHEMA AS table_schema,
COUNT(*) AS table_count
FROM information_schema.tables
GROUP BY TABLE_SCHEMA
'''
metric = "doris_custom"
tags = ["table_schema"]
fields = ["table_count"]
interval = "10s"
metric: measurement name to report.tags: result columns used as tags.fields: result columns used as fields. Field values must be numeric.interval: collection interval for this query. If not configured,inputs.doris.intervalis used.
Metric¶
doris_fe¶
| Tags & Fields | Description |
|---|---|
| catalog ( tag) |
Catalog. |
| host ( tag) |
Host name. |
| instance ( tag) |
Instance endpoint. |
| job ( tag) |
Job type. |
| method ( tag) |
Method type. |
| name ( tag) |
Metric name. |
| quantile ( tag) |
quantile. |
| state ( tag) |
State. |
| type ( tag) |
Metric type. |
| cache_added | Cumulative number of entries added to the FE cache. Type: float | (count) Unit: count |
| cache_hit | Cumulative number of FE cache hits. Type: float | (count) Unit: count |
| connection_total | Current number of FE MySQL port connections. Type: float | (gauge) Unit: count |
| counter_hit_sql_block_rule | Number of queries blocked by SQL BLOCK RULE. Type: float | (count) Unit: count |
| edit_log | Cumulative number of FE metadata log entries. Type: float | (count) Unit: count |
| edit_log_clean | The number of times the historical metadata log was cleared. Type: float | (count) Unit: count |
| editlog_write_latency_ms | metadata log write latency . For example, {quantile=0.75} indicates the 75th percentile write latency . Type: float | (summary) Unit: time,ms |
| external_schema_cache | SpecifyExternal Catalog _ The number of corresponding schema caches. Type: float | (gauge) Unit: count |
| hive_meta_cache | Specify External Hive Meta store Catalog The number of corresponding partition caches. Type: float | (gauge) Unit: count |
| image_clean | The number of times cleaning of historical metadata image files. Type: float | (count) Unit: count |
| image_push | The number of times cleaning of historical metadata image files. Type: float | (count) Unit: count |
| image_write | The Number of to generate metadata image files. Type: float | (count) Unit: count |
| job | Current count of different job types and different job statuses. For example, {job=load, type=INSERT, state=LOADING} represents an import job of type INSERT and the number of jobs in the LOADING state. Type: float | (gauge) Unit: count |
| max_journal_id | The maximum metadata log ID of the current FE node . If it is Master FE , it is the maximum ID currently written , if it is a non- Master FE , represents the maximum ID of the metadata log currently being played back. Type: float | (gauge) Unit: count |
| max_tablet_compaction_score | Largest tablet compaction score reported across all BE nodes. Type: float | (gauge) Unit: N/A |
| publish_txn_num | Specify the number of transactions being published by the DB . For example, { db =test} indicates the number of transactions currently being published by DB test. Type: float | (gauge) Unit: count |
| qps | Current number of FE queries per second ( only query requests are counted ). Type: float | (gauge) Unit: throughput,reqps |
| query_err | Cumulative number of failed FE query requests. Type: float | (count) Unit: count |
| query_err_rate | Error queries per second. Type: float | (gauge) Unit: throughput,reqps |
| query_instance_begin | Specify the fragment where the user request starts Number of instances . For example, {user=test_u} represents the user test_u Number of instances to start requesting. Type: float | (count) Unit: count |
| query_instance_num | Specifies the fragment that the user is currently requesting Number of instances . For example, {user=test_u} represents the user test_u The number of instances currently being requested. Type: float | (gauge) Unit: count |
| query_latency_ms | Percentile statistics of query request latency. For example, {quantile=0.75} indicates the query delay at the 75th percentile. Type: float | (summary) Unit: time,ms |
| query_latency_ms_db | Percentile statistics of query request delay of each DB . For example, {quantile=0.75,db=test} indicates the query delay of the 75th percentile of DB test. Type: float | (summary) Unit: time,ms |
| query_olap_table | The statistics of the number of requests for the internal table ( OlapTable ).Type: float | (count) Unit: count |
| query_rpc_failed | RPC failures sent to the specified BE . For example, { be=192.168.10.1} indicates the number of RPC failures sent to BE with IP address 192.168.10.1. Type: float | (count) Unit: count |
| query_rpc_size | Specify the RPC data size of BE . For example, { be=192.168.10.1} indicates the number of RPC data bytes sent to BE with IP address 192.168.10.1. Type: float | (count) Unit: digital,B |
| query_rpc_total | Of RPCs sent to the specified BE . For example, { be=192.168.10.1} indicates the number of RPCs sent to BE with IP address 192.168.10.1. Type: float | (count) Unit: count |
| query_total | Cumulative number of FE query requests. Type: float | (count) Unit: count |
| report_queue_size | The queue length of various periodic reporting tasks of BE on the FE side. Type: float | (gauge) Unit: count |
| request_total | All operation requests received through the MySQL port (including queries and other statements ). Type: float | (count) Unit: count |
| routine_load_error_rows | Count the total number of error rows for all Routine Load jobs in the cluster. Type: float | (count) Unit: count |
| routine_load_receive_bytes | The amount of data received by all Routine Load jobs in the cluster. Type: float | (count) Unit: digital,B |
| routine_load_rows | Count the number of data rows received by all Routine Load jobs in the cluster. Type: float | (count) Unit: count |
| rps | Current number of FE requests per second (including queries and other types of statements ). Type: float | (gauge) Unit: count |
| scheduled_tablet_num | Tablets being scheduled by the Master FE node . Includes replicas being repaired and replicas being balanced. Type: float | (gauge) Unit: count |
| tablet_max_compaction_score | Tablet compaction score reported by each BE node. Type: float | (gauge) Unit: N/A |
| tablet_num | Current total number of tablets on each BE node . For example, {backend=172.21.0.1:9556} indicates the current number of tablets of the BE 172.21.0.1:9556. Type: float | (gauge) Unit: count |
| tablet_status_count | Number of tablets by scheduler status on the master FE node. Type: float | (gauge) Unit: count |
| thread_pool | Count the number of working threads and queuing status of various thread pools . active_thread_num Indicates the number of tasks being executed . pool_size Indicates the total number of threads in the thread pool . task_in_queue Indicates the number of tasks being queued. Type: float | (gauge) Unit: count |
| thrift_rpc_latency_ms | The RPC requests received by each method of the FE thrift interface take time. For example, {method=report} indicates that the RPC request received by the report method takes time. Type: float | (summary) Unit: time,ms |
| thrift_rpc_total | RPC requests received by each method of the FE thrift interface . For example, {method=report} indicates the number of RPC requests received by the report method. Type: float | (count) Unit: count |
| txn_counter | Cumulative number of imported transactions by status. Type: float | (count) Unit: count |
| txn_exec_latency_ms | Percentile statistics of transaction execution time. For example, {quantile=0.75} indicates the 75th percentile transaction execution time. Type: float | (summary) Unit: time,ms |
| txn_num | Specifies the number of transactions being performed by the DB . For example, { db =test} indicates the number of transactions currently being executed by DB test. Type: float | (gauge) Unit: count |
| txn_publish_latency_ms | Percentile statistics of transaction publish time. For example, {quantile=0.75} indicates that the 75th percentile transaction publish time is. Type: float | (summary) Unit: time,ms |
| txn_replica_num | Specifies the number of replicas opened by the transaction being executed by the DB . For example, { db =test} indicates the number of copies opened by the transaction currently being executed by DB test. Type: float | (gauge) Unit: count |
| txn_status | Count the number of import transactions currently in various states. For example, {type=committed} indicates the number of transactions in the committed state. Type: float | (gauge) Unit: count |
doris_be¶
| Tags & Fields | Description |
|---|---|
| device ( tag) |
Device name. |
| host ( tag) |
Host name. |
| instance ( tag) |
Instance endpoint. |
| mode ( tag) |
Metric mode. |
| name ( tag) |
Metric name. |
| path ( tag) |
File path. |
| quantile ( tag) |
quantile. |
| status ( tag) |
Metric status. |
| type ( tag) |
Metric type. |
| active_scan_context_count | The number of scanners currently opened directly from the outside. Type: float | (gauge) Unit: count |
| add_batch_task_queue_size | When recording import, the queue size of the thread pool that receives the batch. Type: float | (gauge) Unit: count |
| agent_task_queue_size | Display the length of each Agent Task processing queue, such as {type=CREATE_TABLE} Indicates the length of the CREATE_TABLE task queue. Type: float | (gauge) Unit: count |
| all_rowsets_num | Current number of rowsets. Type: float | (gauge) Unit: count |
| all_segments_num | Current number of segments. Type: float | (gauge) Unit: count |
| brpc_endpoint_stub_count | Created _ The number of brpc stubs used for interaction between BEs.Type: float | (gauge) Unit: count |
| brpc_function_endpoint_stub_count | Created _ The number of brpc stubs used to interact with Remote RPC.Type: float | (gauge) Unit: count |
| cache_capacity | Record the capacity of the specified LRU Cache. Type: float | (gauge) Unit: digital,B |
| cache_hit_count | Record the number of hits in the specified LRU Cache. Type: float | (count) Unit: count |
| cache_hit_ratio | Hit ratio of the specified LRU cache as a value from 0 to 1. Type: float | (gauge) Unit: percent,percent_decimal |
| cache_lookup_count | Record the number of times the specified LRU Cache is searched. Type: float | (count) Unit: count |
| cache_usage | Record the usage of the specified LRU Cache. Type: float | (gauge) Unit: digital,B |
| cache_usage_ratio | Usage ratio of the specified LRU cache as a value from 0 to 1. Type: float | (gauge) Unit: percent,percent_decimal |
| chunk_pool_local_core_alloc_count | ChunkAllocator , the number of times memory is allocated from the memory queue of the bound core. Type: float | (count) Unit: count |
| chunk_pool_other_core_alloc_count | ChunkAllocator , the number of times memory is allocated from the memory queue of other cores. Type: float | (count) Unit: count |
| chunk_pool_reserved_bytes | ChunkAllocator The amount of memory reserved in. Type: float | (gauge) Unit: digital,B |
| chunk_pool_system_alloc_cost_ns | Cumulative time spent by SystemAllocator applying for memory. Type: float | (count) Unit: time,ns |
| chunk_pool_system_alloc_count | SystemAllocator Number of times to apply for memory. Type: float | (count) Unit: count |
| chunk_pool_system_free_cost_ns | Cumulative time spent by SystemAllocator releasing memory. Type: float | (count) Unit: time,ns |
| chunk_pool_system_free_count | SystemAllocator The number of times memory is released. Type: float | (count) Unit: count |
| compaction_bytes_total | Cumulative bytes processed by compaction. Type: float | (count) Unit: digital,B |
| compaction_deltas_total | Cumulative number of rowset deltas processed by compaction. Type: float | (count) Unit: count |
| compaction_used_permits | The number of tokens used by the Compaction task. Type: float | (gauge) Unit: count |
| compaction_waitting_permits | Number of compaction permits currently waiting. Type: float | (gauge) Unit: count |
| cpu | CPU counters collected from /proc/stat by logical core and CPU mode. Type: float | (count) Unit: count |
| data_stream_receiver_count | Number of data receiving terminals Receiver. Type: float | (gauge) Unit: count |
| disk_bytes_read | Cumulative bytes read from each disk, collected from /proc/diskstats. Type: float | (count) Unit: digital,B |
| disk_bytes_written | Cumulative bytes written to each disk. Type: float | (count) Unit: digital,B |
| disk_io_time_ms | Cumulative disk I/O time in milliseconds. Type: float | (count) Unit: time,ms |
| disk_io_time_weighted | Weighted cumulative disk I/O time in milliseconds. Type: float | (count) Unit: time,ms |
| disk_read_time_ms | Cumulative disk read time in milliseconds. Type: float | (count) Unit: time,ms |
| disk_reads_completed | Cumulative number of completed disk reads. Type: float | (count) Unit: count |
| disk_write_time_ms | Cumulative disk write time in milliseconds. Type: float | (count) Unit: time,ms |
| disk_writes_completed | Cumulative number of completed disk writes. Type: float | (count) Unit: count |
| disks_avail_capacity | Specify the remaining space on the disk where the specified data directory is located. like {path=path1} express /path1 The remaining space on the disk where the directory is located. Type: float | (gauge) Unit: digital,B |
| disks_compaction_num | Compaction tasks being executed on the specified data directory . like {path=path1} means /path1 The number of tasks being executed on the directory. Type: float | (gauge) Unit: count |
| disks_compaction_score | Compaction score for the specified data directory. Type: float | (gauge) Unit: N/A |
| disks_local_used_capacity | Local used capacity for the specified data directory. Type: float | (gauge) Unit: digital,B |
| disks_remote_used_capacity | Remote used capacity for the specified data directory. Type: float | (gauge) Unit: digital,B |
| disks_state | Specifies the disk status of the data directory . 1 means normal. 0 means abnormal. Type: float | (gauge) Unit: bool |
| disks_total_capacity | Capacity of the disk where the specified data directory is located. Type: float | (gauge) Unit: digital,B |
| engine_requests_total | Cumulative number of BE engine requests. Type: float | (count) Unit: count |
| fd_num_limit | System file handle limit upper limit. from /proc/sys/fs/file-nr collection. Type: float | (gauge) Unit: count |
| fd_num_used | The number of file handles used by the system . from /proc/sys/fs/file-nr collection. Type: float | (gauge) Unit: count |
| file_created_total | Cumulative number of local file creation times. Type: float | (count) Unit: count |
| fragment_endpoint_count | Number of fragment endpoints by execution status on the BE. Type: float | (gauge) Unit: count |
| fragment_instance_count | The number of fragment instances currently received. Type: float | (gauge) Unit: count |
| fragment_request_duration_us | All fragment intance The cumulative execution time of.Type: float | (count) Unit: time,μs |
| fragment_requests_total | The cumulative number of executed fragment instances. Type: float | (count) Unit: count |
| fragment_thread_pool_queue_size | Current query execution thread pool waiting queue. Type: float | (gauge) Unit: count |
| heavy_work_active_threads | Number of active threads in heavy thread pool. Type: float | (gauge) Unit: count |
| heavy_work_max_threads | Number of heavy thread pool threads. Type: float | (gauge) Unit: count |
| heavy_work_pool_queue_size | The maximum length of the heavy thread pool queue will block the submission of work if it exceeds it. Type: float | (gauge) Unit: count |
| light_work_active_threads | Number of active threads in light thread pool. Type: float | (gauge) Unit: count |
| light_work_max_threads | Number of light thread pool threads. Type: float | (gauge) Unit: count |
| light_work_pool_queue_size | The maximum length of the light thread pool queue . If it exceeds the maximum length, the submission of work will be blocked. Type: float | (gauge) Unit: count |
| load_average | Machine load average by interval, such as 15 minutes. Type: float | (gauge) Unit: count |
| load_bytes | Cumulative quantity sent through tablet Sink. Type: float | (count) Unit: digital,B |
| load_channel_count | The number of load channels currently open. Type: float | (gauge) Unit: count |
| load_rows | Cumulative number of rows sent through tablet Sink. Type: float | (count) Unit: count |
| local_bytes_read_total | Depend on LocalFileReader Number of bytes read. Type: float | (count) Unit: digital,B |
| local_bytes_written_total | Depend on LocalFileWriter Number of bytes written. Type: float | (count) Unit: digital,B |
| local_file_open_reading | Current number of open LocalFileReader instances. Type: float | (gauge) Unit: count |
| local_file_reader_total | Opened LocalFileReader Cumulative count of. Type: float | (count) Unit: count |
| local_file_writer_total | Cumulative number of opened LocalFileWriter instances. Type: float | (count) Unit: count |
| max_disk_io_util_percent | Maximum disk I/O utilization percentage across all disks. Type: float | (gauge) Unit: percent,percent |
| max_network_receive_bytes_rate | The maximum receive rate calculated among all network cards. Type: float | (gauge) Unit: traffic,B/S |
| max_network_send_bytes_rate | The calculated maximum sending rate among all network cards. Type: float | (gauge) Unit: traffic,B/S |
| mem_consumption | Specifies the current memory overhead of the module . For example, {type=compaction} represents the current total memory overhead of the compaction module. Type: float | (gauge) Unit: digital,B |
| memory_allocated_bytes | BE process physical memory size, taken from /proc/self/status/ VmRSS. Type: float | (gauge) Unit: digital,B |
| memory_jemalloc | Jemalloc stats, taken from je_mallctl.Type: float | (gauge) Unit: digital,B |
| memory_pgpgin | The amount of data written by the system from disk to memory page. Type: float | (gauge) Unit: digital,B |
| memory_pgpgout | The amount of data written to disk by system memory pages. Type: float | (gauge) Unit: digital,B |
| memory_pool_bytes_total | all MemPool The size of memory currently occupied. Statistical value, does not represent actual memory usage. Type: float | (gauge) Unit: digital,B |
| memory_pswpin | The number of times the system swapped from disk to memory. Type: float | (gauge) Unit: digital,B |
| memory_pswpout | The number of times the system swapped from memory to disk. Type: float | (gauge) Unit: digital,B |
| memtable_flush_duration_us | Cumulative time spent flushing memtables to disk. Type: float | (count) Unit: time,μs |
| memtable_flush_total | Cumulative number of memtable flushes written to disk. Type: float | (count) Unit: count |
| meta_request_duration | Access RocksDB The cumulative time consumption of meta in. Type: float | (count) Unit: time,μs |
| meta_request_total | Access RocksDB The cumulative number of meta requests. Type: float | (count) Unit: count |
| network_receive_bytes | each network card are accumulated. Collected from /proc/net/dev. Type: float | (count) Unit: digital,B |
| network_receive_packets | each network card is accumulated. Collected from /proc/net/dev. Type: float | (count) Unit: count |
| network_send_bytes | each network card . Collected from /proc/net/dev. Type: float | (count) Unit: digital,B |
| network_send_packets | The total number of packets sent by each network card is accumulated. Collected from /proc/net/dev. Type: float | (count) Unit: count |
| proc | Current number of processes. Type: float | (count) Unit: count |
| process_fd_num_limit_hard | BE process. pass /proc/ pid /limits collection. Type: float | (gauge) Unit: count |
| process_fd_num_limit_soft | BE process. pass /proc/ pid /limits collection. Type: float | (gauge) Unit: count |
| process_fd_num_used | The number of file handles used by the BE process. pass /proc/ pid /limits collection. Type: float | (gauge) Unit: count |
| process_thread_num | BE process threads. pass /proc/ pid /task collection. Type: float | (gauge) Unit: count |
| query_cache_memory_total_byte | Bytes currently occupied by Query Cache. Type: float | (gauge) Unit: digital,B |
| query_cache_partition_total_count | Current number of Partition Cache caches. Type: float | (count) Unit: count |
| query_cache_sql_total_count | Current number of SQL cache entries. Type: float | (gauge) Unit: count |
| query_scan_bytes | Cumulative bytes read from Olap tables by query scan operations. Type: float | (count) Unit: digital,B |
| query_scan_bytes_per_second | According to doris_be_query_scan_bytes Calculated read rate. Type: float | (gauge) Unit: traffic,B/S |
| query_scan_rows | Cumulative raw rows read from Olap tables by query scan operations, including rows skipped by indexes. Type: float | (count) Unit: count |
| result_block_queue_count | The number of fragment instances in the current query result cache. Type: float | (gauge) Unit: count |
| result_buffer_block_count | The number of queries in the current query result cache. Type: float | (gauge) Unit: count |
| routine_load_task_count | The number of routine load tasks currently being executed. Type: float | (gauge) Unit: count |
| rowset_count_generated_and_in_use | New and in use since the last startup The number of rowset ids.Type: float | (gauge) Unit: count |
| s3_bytes_read_total | Cumulative bytes read by S3FileReader. Type: float | (count) Unit: digital,B |
| s3_file_open_reading | Current number of open S3FileReader instances. Type: float | (gauge) Unit: count |
| scanner_thread_pool_queue_size | used for OlapScanner The current queued number of thread pools.Type: float | (gauge) Unit: count |
| segment_read | Cumulative number of segments read. Type: float | (count) Unit: count |
| send_batch_thread_pool_queue_size | The number of queues in the thread pool used to send data packets when importing. Type: float | (gauge) Unit: count |
| send_batch_thread_pool_thread_num | The number of threads in the thread pool used to send packets when importing. Type: float | (gauge) Unit: count |
| small_file_cache_count | Current number of small files cached by the BE. Type: float | (gauge) Unit: count |
| snmp_tcp_in_errs | tcp packet reception errors. Collected from /proc/net/ SNMP. Type: float | (count) Unit: count |
| snmp_tcp_in_segs | tcp packets sent . Collected from /proc/net/ SNMP. Type: float | (count) Unit: count |
| snmp_tcp_out_segs | tcp packets sent. Collected from /proc/net/ SNMP. Type: float | (count) Unit: count |
| snmp_tcp_retrans_segs | TCP packet retransmissions . Collected from /proc/net/ SNMP. Type: float | (count) Unit: count |
| stream_load | Cumulative number of stream load requests received. Type: float | (count) Unit: count |
| stream_load_pipe_count | Current number of stream load data pipelines. Type: float | (gauge) Unit: count |
| stream_load_txn_request | Cumulative number of stream load transaction requests. Type: float | (count) Unit: count |
| streaming_load_current_processing | Number of stream load tasks currently running. Type: float | (count) Unit: count |
| streaming_load_duration_ms | Cumulative execution time of all stream load tasks. Type: float | (count) Unit: time,ms |
| streaming_load_requests_total | Cumulative number of stream load tasks. Type: float | (count) Unit: count |
| tablet_base_max_compaction_score | Current largest base compaction score. Type: float | (gauge) Unit: N/A |
| tablet_cumulative_max_compaction_score | Current largest cumulative compaction score. Type: float | (gauge) Unit: N/A |
| tablet_version_num_distribution | The histogram of the number of tablet versions. Type: float | (histogram) Unit: count |
| thrift_connections_total | Thrift connections created . like {name=heartbeat} Indicates the cumulative number of connections to the heartbeat service. Type: float | (count) Unit: count |
| thrift_current_connections | Current number of thrift connections. like {name=heartbeat} Indicates the current number of connections to the heartbeat service. Type: float | (gauge) Unit: count |
| thrift_opened_clients | Thrift clients currently open . like {name=frontend} Indicates the number of clients accessing the FE service. Type: float | (gauge) Unit: count |
| thrift_used_clients | Thrift clients currently in use . like {name=frontend} Indicates the number of clients being used to access the FE service. Type: float | (gauge) Unit: count |
| timeout_canceled_fragment_count | Cumulative number of fragment instances canceled due to timeout. Type: float | (gauge) Unit: count |
| unused_rowsets_count | The number of currently abandoned rowsets.Type: float | (gauge) Unit: count |
| upload_fail_count | Cumulative number of rowset upload failures to remote storage. Type: float | (count) Unit: count |
| upload_rowset_count | Cumulative number of rowsets successfully uploaded to remote storage.Type: float | (count) Unit: count |
| upload_total_byte | Bytes of rowset data successfully uploaded to remote storage. Type: float | (gauge) Unit: digital,B |
doris_common¶
| Tags & Fields | Description |
|---|---|
| host ( tag) |
Host name. |
| instance ( tag) |
Instance endpoint. |
| name ( tag) |
Metric name. |
| state ( tag) |
Metric state. |
| type ( tag) |
Metric type. |
| node_info | Number of Doris nodes by node type and state. Type: float | (gauge) Unit: count |
| system_meminfo | FE node machines. Collected from /proc/meminfo . include buffers , cached , memory_available , memory_free , memory_total. Type: float | (gauge) Unit: digital,B |
| system_snmp | FE node machines. Collected from /proc/net/ SNMP. Type: float | (gauge) Unit: count |
doris_jvm¶
| Tags & Fields | Description |
|---|---|
| host ( tag) |
Host name. |
| instance ( tag) |
Instance endpoint. |
| type ( tag) |
Metric type. |
| heap_size_bytes | JVM memory metrics. The tags include max, used, committed , corresponding to the maximum value, used and requested memory respectively. Type: float | (gauge) Unit: digital,B |
| non_heap_size_bytes | JVM off-heap memory statistics. Type: float | (gauge) Unit: digital,B |
| old_gc | JVM old-generation garbage collection count. Type: float | (gauge) Unit: count |
| old_size_bytes | JVM old generation memory statistics. Type: float | (gauge) Unit: digital,B |
| thread | JVM thread count statistics. Type: float | (gauge) Unit: count |
| young_size_bytes | JVM new generation memory statistics. Type: float | (gauge) Unit: digital,B |
collector¶
| Tags & Fields | Description |
|---|---|
| instance ( tag) |
Server addr of the instance |
| job ( tag) |
Server name of the instance |
| up | Whether the collector successfully scraped the target during the last collection cycle: 1 means true and 0 means false. Type: int | (gauge) Unit: bool |
Object¶
database¶
| Tags & Fields | Description |
|---|---|
| database_instance ( tag) |
Doris instance identifier from configured tag, current FE HostName, or the FE SQL endpoint. |
| database_type ( tag) |
The type of the database. The value is Doris. |
| host ( tag) |
The configured Doris FE host. |
| name ( tag) |
The object identifier. |
| port ( tag) |
The FE query port. |
| server ( tag) |
The Doris FE SQL endpoint. The value is host:query_port. |
| version ( tag) |
The Doris FE version. |
| avg_query_time | The average query latency in milliseconds. Collected from FE metrics when summary count/sum is available. Type: float Unit: time,ms |
| message | Summary of Doris FE/BE and metric source information. Type: string Unit: - |
| qps | The number of queries executed by Doris per second. Collected from FE metrics. Type: float Unit: gauge |