ClickHouse
ClickHouse collector can collect various metrics actively exposed by ClickHouse server instances, such as the number of statements executed, memory storage, IO interaction and other metrics, and collect the metrics into Guance to help you monitor and analyze various abnormal situations of ClickHouse.
Configuration¶
Preconditions¶
ClickHouse version >=v20.1.2.4
Find the following code snippet in the config.xml configuration file of ClickHouse-server, uncomment it, and set the port number exposed by metrics (which is unique if you choose it yourself). Restart after modification (if it is a cluster, every machine needs to operate).
<prometheus>
<endpoint>/metrics</endpoint>
<port>9363</port>
<metrics>true</metrics>
<events>true</events>
<asynchronous_metrics>true</asynchronous_metrics>
</prometheus>
Field description:
- HTTP Routing of
endpoint
Prometheus Server Fetch Metrics port
number of the port endpointmetrics
grabs exposed metrics flags from ClickHouse'ssystem.metrics
tableevents
grabs exposed event flags from ClickHouse'stable.events
.asynchronous_metrics
grabs exposed asynchronous_metrics flags from ClickHouse'ssystem.asynchronous_metrics
table
See ClickHouse official documents
Collector Configuration¶
Go to the conf.d/clickhousev1
directory under the DataKit installation directory, copy clickhousev1.conf.sample
and name it clickhousev1.conf
. Examples are as follows:
[[inputs.clickhousev1]]
## Exporter URLs.
urls = ["http://127.0.0.1:9363/metrics"]
## Unix Domain Socket URL. Using socket to request data when not empty.
uds_path = ""
## Ignore URL request errors.
ignore_req_err = false
## Collect data output.
## Fill this when want to collect the data to local file nor center.
## After filling, could use 'datakit debug --prom-conf /path/to/this/conf' to debug local storage measurement set.
## Using '--prom-conf' when priority debugging data in 'output' path.
# output = "/abs/path/to/file"
## Collect data upper limit as bytes.
## Only available when set output to local file.
## If collect data exceeded the limit, the data would be dropped.
## Default is 32MB.
# max_file_size = 0
## Metrics type whitelist. Optional: counter, gauge, histogram, summary
## Example: metric_types = ["counter", "gauge"], only collect 'counter' and 'gauge'.
## Default collect all.
# metric_types = []
## Metrics name whitelist.
## Regex supported. Multi supported, conditions met when one matched.
## Collect all if empty.
# metric_name_filter = ["cpu"]
## Metrics name blacklist.
## If a word both in blacklist and whitelist, blacklist priority.
## Regex supported. Multi supported, conditions met when one matched.
## Collect all if empty.
# metric_name_filter_ignore = ["foo","bar"]
## Measurement prefix.
## Add prefix to measurement set name.
measurement_prefix = ""
## Measurement name.
## If measurement_name is empty, split metric name by '_', the first field after split as measurement set name, the rest as current metric name.
## If measurement_name is not empty, using this as measurement set name.
## Always add 'measurement_prefix' prefix at last.
# measurement_name = "clickhouse"
## TLS configuration.
tls_open = false
# tls_ca = "/tmp/ca.crt"
# tls_cert = "/tmp/peer.crt"
# tls_key = "/tmp/peer.key"
## Set to 'true' to enable election.
election = true
## disable setting host tag for this input
disable_host_tag = false
## disable setting instance tag for this input
disable_instance_tag = false
## disable info tag for this input
disable_info_tag = false
## Ignore tags. Multi supported.
## The matched tags would be dropped, but the item would still be sent.
# tags_ignore = ["xxxx"]
## Customize authentification. For now support Bearer Token only.
## Filling in 'token' or 'token_file' is acceptable.
# [inputs.clickhousev1.auth]
# type = "bearer_token"
# token = "xxxxxxxx"
# token_file = "/tmp/token"
## Customize measurement set name.
## Treat those metrics with prefix as one set.
## Prioritier over 'measurement_name' configuration.
[[inputs.clickhousev1.measurements]]
prefix = "ClickHouseProfileEvents_"
name = "ClickHouseProfileEvents"
[[inputs.clickhousev1.measurements]]
prefix = "ClickHouseMetrics_"
name = "ClickHouseMetrics"
[[inputs.clickhousev1.measurements]]
prefix = "ClickHouseAsyncMetrics_"
name = "ClickHouseAsyncMetrics"
[[inputs.clickhousev1.measurements]]
prefix = "ClickHouseStatusInfo_"
name = "ClickHouseStatusInfo"
## Not collecting those data when tag matched.
[inputs.clickhousev1.ignore_tag_kv_match]
# key1 = [ "val1.*", "val2.*"]
# key2 = [ "val1.*", "val2.*"]
## Add HTTP headers to data pulling.
[inputs.clickhousev1.http_headers]
# Root = "passwd"
# Michael = "1234"
## Rename tag key in clickhouse data.
[inputs.clickhousev1.tags_rename]
overwrite_exist_tags = false
[inputs.clickhousev1.tags_rename.mapping]
# tag1 = "new-name-1"
# tag2 = "new-name-2"
# tag3 = "new-name-3"
## Customize tags.
[inputs.clickhousev1.tags]
# some_tag = "some_value"
# more_tag = "some_other_value"
## (Optional) Collect interval: (defaults to "30s").
# interval = "30s"
## (Optional) Timeout: (defaults to "30s").
# timeout = "30s"
After configuration, restart DataKit.
At present, you can inject collector configuration in ConfigMap mode。
Metric¶
For all of the following data collections, the global election tags will added automatically, we can add extra tags in [inputs.clickhousev1.tags]
if needed:
ClickHouseAsyncMetrics
¶
- Tags
Tag | Description |
---|---|
cpu | Cpu id |
disk | Disk name |
eth | Eth id |
host | Host name |
instance | Instance endpoint |
unit | Unit name |
- Metrics
Metric | Description |
---|---|
AsynchronousHeavyMetricsCalculationTimeSpent | Time in seconds spent for calculation of asynchronous heavy tables related metrics this is the overhead of asynchronous metrics. Type: float Unit: time,s |
AsynchronousHeavyMetricsUpdateInterval | Heavy (tables related) metrics update interval Type: float Unit: time,s |
AsynchronousMetricsCalculationTimeSpent | Time in seconds spent for calculation of asynchronous metrics this is the overhead of asynchronous metrics. Type: float Unit: time,s |
AsynchronousMetricsUpdateInterval | Metrics update interval Type: float Unit: time,s |
BlockActiveTime | Time in seconds the block device had the IO requests queued. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: time,s |
BlockDiscardBytes | Number of discarded bytes on the block device. These operations are relevant for SSD. Discard operations are not used by ClickHouse, but can be used by other processes on the system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: digital,B |
BlockDiscardMerges | Number of discard operations requested from the block device and merged together by the OS IO scheduler. These operations are relevant for SSD. Discard operations are not used by ClickHouse, but can be used by other processes on the system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: count |
BlockDiscardOps | Number of discard operations requested from the block device. These operations are relevant for SSD. Discard operations are not used by ClickHouse, but can be used by other processes on the system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: count |
BlockDiscardTime | Time in seconds spend in discard operations requested from the block device, summed across all the operations. These operations are relevant for SSD. Discard operations are not used by ClickHouse, but can be used by other processes on the system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: time,s |
BlockInFlightOps | This value counts the number of I/O requests that have been issued to the device driver but have not yet completed. It does not include IO requests that are in the queue but not yet issued to the device driver. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: count |
BlockQueueTime | This value counts the number of milliseconds that IO requests have waited on this block device. If there are multiple IO requests waiting, this value will increase as the product of the number of milliseconds times the number of requests waiting. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: time,ms |
BlockReadBytes | Number of bytes read from the block device. It can be lower than the number of bytes read from the filesystem due to the usage of the OS page cache, that saves IO. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: digital,B |
BlockReadMerges | Number of read operations requested from the block device and merged together by the OS IO scheduler. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: count |
BlockReadOps | Number of read operations requested from the block device. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: count |
BlockReadTime | Time in seconds spend in read operations requested from the block device, summed across all the operations. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: time,s |
BlockWriteBytes | Number of bytes written to the block device. It can be lower than the number of bytes written to the filesystem due to the usage of the OS page cache, that saves IO. A write to the block device may happen later than the corresponding write to the filesystem due to write-through caching. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: digital,B |
BlockWriteMerges | Number of write operations requested from the block device and merged together by the OS IO scheduler. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: count |
BlockWriteOps | Number of write operations requested from the block device. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: count |
BlockWriteTime | Time in seconds spend in write operations requested from the block device, summed across all the operations. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Source: /sys/block .Type: float Unit: time,s |
CPUFrequencyMHz | The current frequency of the CPU, in MHz. Most of the modern CPUs adjust the frequency dynamically for power saving and Turbo Boosting. Type: float Unit: frequency,MHz |
CompiledExpressionCacheBytes | Total bytes used for the cache of JIT-compiled code. Type: float Unit: digital,B |
CompiledExpressionCacheCount | Total entries in the cache of JIT-compiled code. Type: float Unit: count |
DiskAvailable | Available bytes on the disk (virtual filesystem). Remote filesystems can show a large value like 16 EiB.Type: float Unit: digital,B |
DiskTotal | The total size in bytes of the disk (virtual filesystem). Remote filesystems can show a large value like 16 EiB.Type: float Unit: digital,B |
DiskUnreserved | Available bytes on the disk (virtual filesystem) without the reservations for merges, fetches, and moves. Remote filesystems can show a large value like 16 EiB.Type: float Unit: digital,B |
DiskUsed | Used bytes on the disk (virtual filesystem). Remote filesystems not always provide this information.Type: float Unit: digital,B |
FilesystemCacheBytes | Total bytes in the cache virtual filesystem. This cache is hold on disk.Type: float Unit: digital,B |
FilesystemCacheFiles | Total number of cached file segments in the cache virtual filesystem. This cache is hold on disk.Type: float Unit: count |
FilesystemLogsPathAvailableBytes | Available bytes on the volume where ClickHouse logs path is mounted. If this value approaches zero, you should tune the log rotation in the configuration file. Type: float Unit: digital,B |
FilesystemLogsPathAvailableINodes | The number of available inodes on the volume where ClickHouse logs path is mounted.Type: float Unit: count |
FilesystemLogsPathTotalBytes | The size of the volume where ClickHouse logs path is mounted, in bytes. It's recommended to have at least 10 GB for logs. Type: float Unit: digital,B |
FilesystemLogsPathTotalINodes | The total number of inodes on the volume where ClickHouse logs path is mounted.Type: float Unit: count |
FilesystemLogsPathUsedBytes | Used bytes on the volume where ClickHouse logs path is mounted. Type: float Unit: digital,B |
FilesystemLogsPathUsedINodes | The number of used inodes on the volume where ClickHouse logs path is mounted.Type: float Unit: count |
FilesystemMainPathAvailableBytes | Available bytes on the volume where the main ClickHouse path is mounted. Type: float Unit: digital,B |
FilesystemMainPathAvailableINodes | The number of available inodes on the volume where the main ClickHouse path is mounted. If it is close to zero, it indicates a misconfiguration, and you will get 'no space left on device' even when the disk is not full.Type: float Unit: count |
FilesystemMainPathTotalBytes | The size of the volume where the main ClickHouse path is mounted, in bytes. Type: float Unit: digital,B |
FilesystemMainPathTotalINodes | The total number of inodes on the volume where the main ClickHouse path is mounted. If it is less than 25 million, it indicates a misconfiguration.Type: float Unit: count |
FilesystemMainPathUsedBytes | Used bytes on the volume where the main ClickHouse path is mounted. Type: float Unit: digital,B |
FilesystemMainPathUsedINodes | The number of used inodes on the volume where the main ClickHouse path is mounted. This value mostly corresponds to the number of files.Type: float Unit: count |
HTTPThreads | Number of threads in the server of the HTTP interface (without TLS). Type: float Unit: count |
InterserverThreads | Number of threads in the server of the replicas communication protocol (without TLS). Type: float Unit: count |
Jitter | The difference in time the thread for calculation of the asynchronous metrics was scheduled to wake up and the time it was in fact, woken up. A proxy-indicator of overall system latency and responsiveness. Type: float Unit: time,s |
LoadAverage | The whole system load, averaged with exponential smoothing over 1 minute. The load represents the number of threads across all the processes (the scheduling entities of the OS kernel), that are currently running by CPU or waiting for IO, or ready to run but not being scheduled at this point of time. This number includes all the processes, not only clickhouse-server . The number can be greater than the number of CPU cores, if the system is overloaded, and many processes are ready to run but waiting for CPU or IO.Type: float Unit: count |
MMapCacheCells | The number of files opened with mmap (mapped in memory). This is used for queries with the setting local_filesystem_read_method set to mmap . The files opened with mmap are kept in the cache to avoid costly TLB flushes.Type: float Unit: count |
MarkCacheBytes | Total size of mark cache in bytes Type: float Unit: digital,B |
MarkCacheFiles | Total number of mark files cached in the mark cache Type: float Unit: count |
MaxPartCountForPartition | Maximum number of parts per partition across all partitions of all tables of MergeTree family. Values larger than 300 indicates misconfiguration, overload, or massive data loading. Type: float Unit: count |
MemoryCode | The amount of virtual memory mapped for the pages of machine code of the server process, in bytes. Type: float Unit: digital,B |
MemoryDataAndStack | The amount of virtual memory mapped for the use of stack and for the allocated memory, in bytes. It is unspecified whether it includes the per-thread stacks and most of the allocated memory, that is allocated with the mmap system call. This metric exists only for completeness reasons. I recommend to use the MemoryResident metric for monitoring.Type: float Unit: digital,B |
MemoryResident | The amount of physical memory used by the server process, in bytes. Type: float Unit: digital,B |
MemoryShared | The amount of memory used by the server process, that is also shared by another processes, in bytes. ClickHouse does not use shared memory, but some memory can be labeled by OS as shared for its own reasons. This metric does not make a lot of sense to watch, and it exists only for completeness reasons. Type: float Unit: digital,B |
MemoryVirtual | The size of the virtual address space allocated by the server process, in bytes. The size of the virtual address space is usually much greater than the physical memory consumption, and should not be used as an estimate for the memory consumption. The large values of this metric are totally normal, and makes only technical sense. Type: float Unit: digital,B |
MySQLThreads | Number of threads in the server of the MySQL compatibility protocol. Type: float Unit: count |
NetworkReceiveBytes | Number of bytes received via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: digital,B |
NetworkReceiveDrop | Number of bytes a packet was dropped while received via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: digital,B |
NetworkReceiveErrors | Number of times error happened receiving via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: count |
NetworkReceivePackets | Number of network packets received via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: count |
NetworkSendBytes | Number of bytes sent via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: digital,B |
NetworkSendDrop | Number of times a packed was dropped while sending via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: count |
NetworkSendErrors | Number of times error (e.g. TCP retransmit) happened while sending via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: count |
NetworkSendPackets | Number of network packets sent via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: count |
NumberOfDatabases | Total number of databases on the server. Type: float Unit: count |
NumberOfDetachedByUserParts | The total number of parts detached from MergeTree tables by users with the ALTER TABLE DETACH query (as opposed to unexpected, broken or ignored parts). The server does not care about detached parts and they can be removed.Type: float Unit: count |
NumberOfDetachedParts | The total number of parts detached from MergeTree tables. A part can be detached by a user with the ALTER TABLE DETACH query or by the server itself it the part is broken, unexpected or unneeded. The server does not care about detached parts and they can be removed.Type: float Unit: count |
NumberOfTables | Total number of tables summed across the databases on the server, excluding the databases that cannot contain MergeTree tables. The excluded database engines are those who generate the set of tables on the fly, like Lazy , MySQL , PostgreSQL , SQlite .Type: float Unit: count |
OSContextSwitches | The number of context switches that the system underwent on the host machine. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: count |
OSGuestNiceTime | The ratio of time spent running a virtual CPU for guest operating systems under the control of the Linux kernel, when a guest was set to a higher priority (See man procfs ). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . This metric is irrelevant for ClickHouse, but still exists for completeness. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSGuestNiceTimeCPU | The ratio of time spent running a virtual CPU for guest operating systems under the control of the Linux kernel, when a guest was set to a higher priority (See man procfs ). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . This metric is irrelevant for ClickHouse, but still exists for completeness. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSGuestNiceTimeNormalized | The value is similar to OSGuestNiceTime but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.Type: float Unit: rate |
OSGuestTime | The ratio of time spent running a virtual CPU for guest operating systems under the control of the Linux kernel (See man procfs ). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . This metric is irrelevant for ClickHouse, but still exists for completeness. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSGuestTimeCPU | The ratio of time spent running a virtual CPU for guest operating systems under the control of the Linux kernel (See man procfs ). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . This metric is irrelevant for ClickHouse, but still exists for completeness. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSGuestTimeNormalized | The value is similar to OSGuestTime but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.Type: float Unit: rate |
OSIOWaitTime | The ratio of time the CPU core was not running the code but when the OS kernel did not run any other process on this CPU as the processes were waiting for IO. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSIOWaitTimeCPU | The ratio of time the CPU core was not running the code but when the OS kernel did not run any other process on this CPU as the processes were waiting for IO. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSIOWaitTimeNormalized | The value is similar to OSIOWaitTime but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.Type: float Unit: rate |
OSIdleTime | The ratio of time the CPU core was idle (not even ready to run a process waiting for IO) from the OS kernel standpoint. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . This does not include the time when the CPU was under-utilized due to the reasons internal to the CPU (memory loads, pipeline stalls , branch mispredictions , running another SMT core). The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSIdleTimeCPU | The ratio of time the CPU core was idle (not even ready to run a process waiting for IO) from the OS kernel standpoint. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . This does not include the time when the CPU was under-utilized due to the reasons internal to the CPU (memory loads, pipeline stalls , branch mispredictions , running another SMT core). The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSIdleTimeNormalized | The value is similar to OSIdleTime but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.Type: float Unit: rate |
OSInterrupts | The number of interrupts on the host machine. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: count |
OSIrqTime | The ratio of time spent for running hardware interrupt requests on the CPU. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . A high number of this metric may indicate hardware misconfiguration or a very high network load. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSIrqTimeCPU | The ratio of time spent for running hardware interrupt requests on the CPU. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . A high number of this metric may indicate hardware misconfiguration or a very high network load. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSIrqTimeNormalized | The value is similar to OSIrqTime but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.Type: float Unit: rate |
OSMemoryAvailable | The amount of memory available to be used by programs, in bytes. This is very similar to the OSMemoryFreePlusCached metric. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: digital,B |
OSMemoryBuffers | The amount of memory used by OS kernel buffers, in bytes. This should be typically small, and large values may indicate a misconfiguration of the OS. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: digital,B |
OSMemoryCached | The amount of memory used by the OS page cache, in bytes. Typically, almost all available memory is used by the OS page cache - high values of this metric are normal and expected. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: digital,B |
OSMemoryFreePlusCached | The amount of free memory plus OS page cache memory on the host system, in bytes. This memory is available to be used by programs. The value should be very similar to OSMemoryAvailable . This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: digital,B |
OSMemoryFreeWithoutCached | The amount of free memory on the host system, in bytes. This does not include the memory used by the OS page cache memory, in bytes. The page cache memory is also available for usage by programs, so the value of this metric can be confusing. See the OSMemoryAvailable metric instead. For convenience we also provide the OSMemoryFreePlusCached metric, that should be somewhat similar to OSMemoryAvailable. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: digital,B |
OSMemorySwapCached | The amount of memory in swap that was also loaded in RAM. Swap should be disabled on production systems. If the value of this metric is large, it indicates a misconfiguration. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: digital,B |
OSMemoryTotal | The total amount of memory on the host system, in bytes. Type: float Unit: digital,B |
OSNiceTime | The ratio of time the CPU core was running userspace code with higher priority. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSNiceTimeCPU | The ratio of time the CPU core was running userspace code with higher priority. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSNiceTimeNormalized | The value is similar to OSNiceTime but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.Type: float Unit: rate |
OSOpenFiles | The total number of opened files on the host machine. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: count |
OSProcessesBlocked | Number of threads blocked waiting for I/O to complete (man procfs ). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: count |
OSProcessesCreated | The number of processes created. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: count |
OSProcessesRunning | The number of runnable (running or ready to run) threads by the operating system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server .Type: float Unit: count |
OSSoftIrqTime | The ratio of time spent for running software interrupt requests on the CPU. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . A high number of this metric may indicate inefficient software running on the system. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSSoftIrqTimeCPU | The ratio of time spent for running software interrupt requests on the CPU. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . A high number of this metric may indicate inefficient software running on the system. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSSoftIrqTimeNormalized | The value is similar to OSSoftIrqTime but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.Type: float Unit: rate |
OSStealTime | The ratio of time spent in other operating systems by the CPU when running in a virtualized environment. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Not every virtualized environments present this metric, and most of them don't. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSStealTimeCPU | The ratio of time spent in other operating systems by the CPU when running in a virtualized environment. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . Not every virtualized environments present this metric, and most of them don't. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSStealTimeNormalized | The value is similar to OSStealTime but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.Type: float Unit: rate |
OSSystemTime | The ratio of time the CPU core was running OS kernel (system) code. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSSystemTimeCPU | The ratio of time the CPU core was running OS kernel (system) code. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSSystemTimeNormalized | The value is similar to OSSystemTime but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.Type: float Unit: rate |
OSThreadsRunnable | The total number of 'runnable' threads, as the OS kernel scheduler seeing it. Type: float Unit: count |
OSThreadsTotal | The total number of threads, as the OS kernel scheduler seeing it. Type: float Unit: count |
OSUptime | The uptime of the host server (the machine where ClickHouse is running), in seconds. Type: float Unit: time,s |
OSUserTime | The ratio of time the CPU core was running userspace code. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . This includes also the time when the CPU was under-utilized due to the reasons internal to the CPU (memory loads, pipeline stalls , branch mispredictions , running another SMT core). The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSUserTimeCPU | The ratio of time the CPU core was running userspace code. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server . This includes also the time when the CPU was under-utilized due to the reasons internal to the CPU (memory loads, pipeline stalls , branch mispredictions , running another SMT core). The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].Type: float Unit: rate |
OSUserTimeNormalized | The value is similar to OSUserTime but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.Type: float Unit: rate |
PostgreSQLThreads | Number of threads in the server of the PostgreSQL compatibility protocol. Type: float Unit: count |
PrometheusThreads | Number of threads in the server of the Prometheus endpoint. Note: Prometheus endpoints can be also used via the usual HTTP/HTTPs ports. Type: float Unit: count |
ReplicasMaxAbsoluteDelay | Maximum difference in seconds between the most fresh replicated part and the most fresh data part still to be replicated, across Replicated tables. A very high value indicates a replica with no data. Type: float Unit: time,s |
ReplicasMaxInsertsInQueue | Maximum number of INSERT operations in the queue (still to be replicated) across Replicated tables. Type: float Unit: count |
ReplicasMaxMergesInQueue | Maximum number of merge operations in the queue (still to be applied) across Replicated tables. Type: float Unit: count |
ReplicasMaxQueueSize | Maximum queue size (in the number of operations like get, merge) across Replicated tables. Type: float Unit: count |
ReplicasMaxRelativeDelay | Maximum difference between the replica delay and the delay of the most up-to-date replica of the same table, across Replicated tables. Type: float Unit: time,s |
ReplicasSumInsertsInQueue | Sum of INSERT operations in the queue (still to be replicated) across Replicated tables. Type: float Unit: count |
ReplicasSumMergesInQueue | Sum of merge operations in the queue (still to be applied) across Replicated tables. Type: float Unit: count |
ReplicasSumQueueSize | Sum queue size (in the number of operations like get, merge) across Replicated tables. Type: float Unit: count |
TCPThreads | Number of threads in the server of the TCP protocol (without TLS). Type: float Unit: count |
Temperature | The temperature of the corresponding device in ℃. A sensor can return an unrealistic value. Source: /sys/class/thermal Type: float Unit: temperature,C |
TotalBytesOfMergeTreeTables | Total amount of bytes (compressed, including data and indices) stored in all tables of MergeTree family. Type: float Unit: digital,B |
TotalPartsOfMergeTreeTables | Total amount of data parts in all tables of MergeTree family. Numbers larger than 10 000 will negatively affect the server startup time and it may indicate unreasonable choice of the partition key. Type: float Unit: count |
TotalRowsOfMergeTreeTables | Total amount of rows (records) stored in all tables of MergeTree family. Type: float Unit: count |
UncompressedCacheBytes | Total size of uncompressed cache in bytes. Uncompressed cache does not usually improve the performance and should be mostly avoided. Type: float Unit: digital,B |
UncompressedCacheCells | Total number of entries in the uncompressed cache. Each entry represents a decompressed block of data. Uncompressed cache does not usually improve performance and should be mostly avoided. Type: float Unit: digital,B |
Uptime | The server uptime in seconds. It includes the time spent for server initialization before accepting connections. Type: float Unit: time,s |
jemalloc_active | Total number of bytes in active pages allocated by the application. This is a multiple of the page size, and greater than or equal to. Type: float Unit: digital,B |
jemalloc_allocated | Total number of bytes allocated by the application. Type: float Unit: digital,B |
jemalloc_arenas_all_dirty_purged | Number of madvise() or similar calls made to purge dirty pages.Type: float Unit: count |
jemalloc_arenas_all_muzzy_purged | Number of muzzy page purge sweeps performed. Type: float Unit: count |
jemalloc_arenas_all_pactive | Number of pages in active extents. Type: float Unit: count |
jemalloc_arenas_all_pdirty | Number of pages within unused extents that are potentially dirty, and for which madvise() or similar has not been called. Type: float Unit: count |
jemalloc_arenas_all_pmuzzy | Number of pages within unused extents that are muzzy. Type: float Unit: count |
jemalloc_background_thread_num_runs | Total number of runs from all background threads. Type: float Unit: count |
jemalloc_background_thread_num_threads | Number of background threads running currently. Type: float Unit: count |
jemalloc_background_thread_run_intervals | Average run interval in nanoseconds of background threads. Type: float Unit: time,ns |
jemalloc_epoch | An internal incremental update number of the statistics of jemalloc (Jason Evans' memory allocator), used in all other jemalloc metrics. Type: float Unit: count |
jemalloc_mapped | If a value is passed in, refresh the data from which the mallctl*() functions report values, and increment the epoch. Return the current epoch. This is useful for detecting whether another thread caused a refresh..Type: float Unit: digital,B |
jemalloc_metadata | Total number of bytes dedicated to metadata, which comprise base allocations used for bootstrap-sensitive allocator metadata structures . Type: float Unit: digital,B |
jemalloc_metadata_thp | Number of transparent huge pages (THP) used for metadata. Type: float Unit: count |
jemalloc_resident | Maximum number of bytes in physically resident data pages mapped by the allocator. Type: float Unit: digital,B |
jemalloc_retained | Total number of bytes in virtual memory mappings. Type: float Unit: digital,B |
ClickHouseMetrics
¶
- Tags
Tag | Description |
---|---|
host | Host name |
instance | Instance endpoint |
- Metrics
Metric | Description |
---|---|
ActiveAsyncDrainedConnections | Number of active connections drained asynchronously. Type: float Unit: count |
ActiveSyncDrainedConnections | Number of active connections drained synchronously. Type: float Unit: count |
AggregatorThreads | Number of threads in the Aggregator thread pool. Type: float Unit: count |
AggregatorThreadsActive | Number of threads in the Aggregator thread pool running a task. Type: float Unit: count |
AsyncDrainedConnections | Number of connections drained asynchronously. Type: float Unit: count |
AsyncInsertCacheSize | Number of async insert hash id in cache Type: float Unit: count |
AsynchronousInsertThreads | Number of threads in the AsynchronousInsert thread pool. Type: float Unit: count |
AsynchronousInsertThreadsActive | Number of threads in the AsynchronousInsert thread pool running a task. Type: float Unit: count |
AsynchronousReadWait | Number of threads waiting for asynchronous read. Type: float Unit: count |
BackgroundBufferFlushSchedulePoolSize | Limit on number of tasks in BackgroundBufferFlushSchedulePool Type: float Unit: count |
BackgroundBufferFlushSchedulePoolTask | Number of active tasks in BackgroundBufferFlushSchedulePool. This pool is used for periodic Buffer flushes Type: float Unit: count |
BackgroundCommonPoolSize | Limit on number of tasks in an associated background pool Type: float Unit: count |
BackgroundCommonPoolTask | Number of active tasks in an associated background pool Type: float Unit: count |
BackgroundDistributedSchedulePoolSize | Limit on number of tasks in BackgroundDistributedSchedulePool Type: float Unit: count |
BackgroundDistributedSchedulePoolTask | Number of active tasks in BackgroundDistributedSchedulePool. This pool is used for distributed sends that is done in background. Type: float Unit: count |
BackgroundFetchesPoolSize | Limit on number of simultaneous fetches in an associated background pool Type: float Unit: count |
BackgroundFetchesPoolTask | Number of active fetches in an associated background pool Type: float Unit: count |
BackgroundMergesAndMutationsPoolSize | Limit on number of active merges and mutations in an associated background pool Type: float Unit: count |
BackgroundMergesAndMutationsPoolTask | Number of active merges and mutations in an associated background pool Type: float Unit: count |
BackgroundMessageBrokerSchedulePoolSize | Limit on number of tasks in BackgroundProcessingPool for message streaming Type: float Unit: count |
BackgroundMessageBrokerSchedulePoolTask | Number of active tasks in BackgroundProcessingPool for message streaming Type: float Unit: count |
BackgroundMovePoolSize | Limit on number of tasks in BackgroundProcessingPool for moves Type: float Unit: count |
BackgroundMovePoolTask | Number of active tasks in BackgroundProcessingPool for moves Type: float Unit: count |
BackgroundPoolTask | Number of active tasks in BackgroundProcessingPool (merges, mutations, or replication queue bookkeeping) Type: float Unit: count |
BackgroundSchedulePoolSize | Limit on number of tasks in BackgroundSchedulePool. This pool is used for periodic ReplicatedMergeTree tasks, like cleaning old data parts, altering data parts, replica re-initialization, etc. Type: float Unit: count |
BackgroundSchedulePoolTask | Number of active tasks in BackgroundSchedulePool. This pool is used for periodic ReplicatedMergeTree tasks, like cleaning old data parts, altering data parts, replica re-initialization, etc. Type: float Unit: count |
BackupsIOThreads | Number of threads in the BackupsIO thread pool. Type: float Unit: count |
BackupsIOThreadsActive | Number of threads in the BackupsIO thread pool running a task. Type: float Unit: count |
BackupsThreads | Number of threads in the thread pool for BACKUP. Type: float Unit: count |
BackupsThreadsActive | Number of threads in thread pool for BACKUP running a task. Type: float Unit: count |
BrokenDistributedFilesToInsert | Number of files for asynchronous insertion into Distributed tables that has been marked as broken. This metric will starts from 0 on start. Number of files for every shard is summed. Type: float Unit: count |
CacheDetachedFileSegments | Number of existing detached cache file segments Type: float Unit: count |
CacheDictionaryThreads | Number of threads in the CacheDictionary thread pool. Type: float Unit: count |
CacheDictionaryThreadsActive | Number of threads in the CacheDictionary thread pool running a task. Type: float Unit: count |
CacheDictionaryUpdateQueueBatches | Number of 'batches' (a set of keys) in update queue in CacheDictionaries. Type: float Unit: count |
CacheDictionaryUpdateQueueKeys | Exact number of keys in update queue in CacheDictionaries. Type: float Unit: count |
CacheFileSegments | Number of existing cache file segments Type: float Unit: count |
ContextLockWait | Number of threads waiting for lock in Context. This is global lock. Type: float Unit: count |
DDLWorkerThreads | Number of threads in the DDLWorker thread pool for ON CLUSTER queries. Type: float Unit: count |
DDLWorkerThreadsActive | Number of threads in the DDLWORKER thread pool for ON CLUSTER queries running a task.Type: float Unit: count |
DatabaseCatalogThreads | Number of threads in the DatabaseCatalog thread pool. Type: float Unit: count |
DatabaseCatalogThreadsActive | Number of threads in the DatabaseCatalog thread pool running a task. Type: float Unit: count |
DatabaseOnDiskThreads | Number of threads in the DatabaseOnDisk thread pool. Type: float Unit: count |
DatabaseOnDiskThreadsActive | Number of threads in the DatabaseOnDisk thread pool running a task. Type: float Unit: count |
DatabaseOrdinaryThreads | Number of threads in the Ordinary database thread pool. Type: float Unit: count |
DatabaseOrdinaryThreadsActive | Number of threads in the Ordinary database thread pool running a task. Type: float Unit: count |
DelayedInserts | Number of INSERT queries that are throttled due to high number of active data parts for partition in a MergeTree table. Type: float Unit: count |
DestroyAggregatesThreads | Number of threads in the thread pool for destroy aggregate states. Type: float Unit: count |
DestroyAggregatesThreadsActive | Number of threads in the thread pool for destroy aggregate states running a task. Type: float Unit: count |
DictCacheRequests | Number of requests in fly to data sources of dictionaries of cache type. Type: float Unit: count |
DiskObjectStorageAsyncThreads | Number of threads in the async thread pool for DiskObjectStorage. Type: float Unit: count |
DiskObjectStorageAsyncThreadsActive | Number of threads in the async thread pool for DiskObjectStorage running a task. Type: float Unit: count |
DiskSpaceReservedForMerge | Disk space reserved for currently running background merges. It is slightly more than the total size of currently merging parts. Type: float Unit: digital,B |
DistributedFilesToInsert | Number of pending files to process for asynchronous insertion into Distributed tables. Number of files for every shard is summed. Type: float Unit: count |
DistributedInsertThreads | Number of threads used for INSERT into Distributed. Type: float Unit: count |
DistributedInsertThreadsActive | Number of threads used for INSERT into Distributed running a task. Type: float Unit: count |
DistributedSend | Number of connections to remote servers sending data that was INSERTed into Distributed tables. Both synchronous and asynchronous mode. Type: float Unit: count |
EphemeralNode | Number of ephemeral nodes hold in ZooKeeper. Type: float Unit: count |
FilesystemCacheElements | Filesystem cache elements (file segments) Type: float Unit: count |
FilesystemCacheReadBuffers | Number of active cache buffers Type: float Unit: count |
FilesystemCacheSize | Filesystem cache size in bytes Type: float Unit: digital,B |
GlobalThread | Number of threads in global thread pool. Type: float Unit: count |
GlobalThreadActive | Number of threads in global thread pool running a task. Type: float Unit: count |
HTTPConnection | Number of connections to HTTP server Type: float Unit: count |
HashedDictionaryThreads | Number of threads in the HashedDictionary thread pool. Type: float Unit: count |
HashedDictionaryThreadsActive | Number of threads in the HashedDictionary thread pool running a task. Type: float Unit: count |
IOPrefetchThreads | Number of threads in the IO prefertch thread pool.Type: float Unit: count |
IOPrefetchThreadsActive | Number of threads in the IO prefetch thread pool running a task. Type: float Unit: count |
IOThreads | Number of threads in the IO thread pool. Type: float Unit: count |
IOThreadsActive | Number of threads in the IO thread pool running a task. Type: float Unit: count |
IOUringInFlightEvents | Number of io_uring SQEs in flight Type: float Unit: count |
IOUringPendingEvents | Number of io_uring SQEs waiting to be submitted Type: float Unit: count |
IOWriterThreads | Number of threads in the IO writer thread pool. Type: float Unit: count |
IOWriterThreadsActive | Number of threads in the IO writer thread pool running a task. Type: float Unit: count |
InterserverConnection | Number of connections from other replicas to fetch parts Type: float Unit: count |
KafkaAssignedPartitions | Number of partitions Kafka tables currently assigned to Type: float Unit: count |
KafkaBackgroundReads | Number of background reads currently working (populating materialized views from Kafka) Type: float Unit: count |
KafkaConsumers | Number of active Kafka consumers Type: float Unit: count |
KafkaConsumersInUse | Number of consumers which are currently used by direct or background reads Type: float Unit: count |
KafkaConsumersWithAssignment | Number of active Kafka consumers which have some partitions assigned. Type: float Unit: count |
KafkaLibrdkafkaThreads | Number of active librdkafka threadsType: float Unit: count |
KafkaProducers | Number of active Kafka producer created Type: float Unit: count |
KafkaWrites | Number of currently running inserts to Kafka Type: float Unit: count |
KeeperAliveConnections | Number of alive connections Type: float Unit: count |
KeeperOutstandingRequets | Number of outstanding requests Type: float Unit: count |
LocalThread | Number of threads in local thread pools. The threads in local thread pools are taken from the global thread pool. Type: float Unit: count |
LocalThreadActive | Number of threads in local thread pools running a task. Type: float Unit: count |
MMappedAllocBytes | Sum bytes of mmapped allocations Type: float Unit: digital,B |
MMappedAllocs | Total number of mmapped allocations Type: float Unit: count |
MMappedFileBytes | Sum size of mmapped file regions. Type: float Unit: digital,B |
MMappedFiles | Total number of mmapped files. Type: float Unit: count |
MarksLoaderThreads | Number of threads in thread pool for loading marks. Type: float Unit: count |
MarksLoaderThreadsActive | Number of threads in the thread pool for loading marks running a task. Type: float Unit: count |
MaxDDLEntryID | Max processed DDL entry of DDLWorker. Type: float Unit: count |
MaxPushedDDLEntryID | Max DDL entry of DDLWorker that pushed to zookeeper. Type: float Unit: count |
MemoryTracking | Total amount of memory (bytes) allocated by the server. Type: float Unit: digital,B |
Merge | Number of executing background merges Type: float Unit: count |
MergeTreeAllRangesAnnouncementsSent | The current number of announcement being sent in flight from the remote server to the initiator server about the set of data parts (for MergeTree tables). Measured on the remote server side. Type: float Unit: count |
MergeTreeBackgroundExecutorThreads | Number of threads in the MergeTreeBackgroundExecutor thread pool. Type: float Unit: count |
MergeTreeBackgroundExecutorThreadsActive | Number of threads in the MergeTreeBackgroundExecutor thread pool running a task. Type: float Unit: count |
MergeTreeDataSelectExecutorThreads | Number of threads in the MergeTreeDataSelectExecutor thread pool. Type: float Unit: count |
MergeTreeDataSelectExecutorThreadsActive | Number of threads in the MergeTreeDataSelectExecutor thread pool running a task. Type: float Unit: count |
MergeTreePartsCleanerThreads | Number of threads in the MergeTree parts cleaner thread pool. Type: float Unit: count |
MergeTreePartsCleanerThreadsActive | Number of threads in the MergeTree parts cleaner thread pool running a task. Type: float Unit: count |
MergeTreePartsLoaderThreads | Number of threads in the MergeTree parts loader thread pool. Type: float Unit: count |
MergeTreePartsLoaderThreadsActive | Number of threads in the MergeTree parts loader thread pool running a task. Type: float Unit: count |
MergeTreeReadTaskRequestsSent | The current number of callback requests in flight from the remote server back to the initiator server to choose the read task (for MergeTree tables). Measured on the remote server side. Type: float Unit: count |
Move | Number of currently executing moves Type: float Unit: count |
MySQLConnection | Number of client connections using MySQL protocol Type: float Unit: count |
NetworkReceive | Number of threads receiving data from network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. Type: float Unit: count |
NetworkSend | Number of threads sending data to network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. Type: float Unit: count |
OpenFileForRead | Number of files open for reading Type: float Unit: count |
OpenFileForWrite | Number of files open for writing Type: float Unit: count |
ParallelFormattingOutputFormatThreads | Number of threads in the ParallelFormattingOutputFormatThreads thread pool. Type: float Unit: count |
ParallelFormattingOutputFormatThreadsActive | Number of threads in the ParallelFormattingOutputFormatThreads thread pool running a task. Type: float Unit: count |
ParallelParsingInputFormatThreads | Number of threads in the ParallelParsingInputFormat thread pool. Type: float Unit: count |
ParallelParsingInputFormatThreadsActive | Number of threads in the ParallelParsingInputFormat thread pool running a task. Type: float Unit: count |
ParquetDecoderThreads | Number of threads in the ParquetBlockInputFormat thread pool running a task. Type: float Unit: count |
ParquetDecoderThreadsActive | Number of threads in the ParquetBlockInputFormat thread pool. Type: float Unit: count |
PartMutation | Number of mutations (ALTER DELETE/UPDATE) Type: float Unit: count |
PartsActive | Active data part, used by current and upcoming SELECTs. Type: float Unit: count |
PartsCommitted | Deprecated. See PartsActive. Type: float Unit: count |
PartsCompact | Compact parts. Type: float Unit: count |
PartsDeleteOnDestroy | Part was moved to another disk and should be deleted in own destructor. Type: float Unit: count |
PartsDeleting | Not active data part with identity refcounter , it is deleting right now by a cleaner.Type: float Unit: count |
PartsInMemory | In-memory parts. Type: float Unit: count |
PartsOutdated | Not active data part, but could be used by only current SELECTs, could be deleted after SELECTs finishes. Type: float Unit: count |
PartsPreActive | The part is in data_parts, but not used for SELECTs. Type: float Unit: count |
PartsPreCommitted | Deprecated. See PartsPreActive. Type: float Unit: count |
PartsTemporary | The part is generating now, it is not in data_parts list. Type: float Unit: count |
PartsWide | Wide parts. Type: float Unit: count |
PendingAsyncInsert | Number of asynchronous inserts that are waiting for flush. Type: float Unit: count |
PostgreSQLConnection | Number of client connections using PostgreSQL protocol Type: float Unit: count |
Query | Number of executing queries Type: float Unit: count |
QueryPipelineExecutorThreads | Number of threads in the PipelineExecutor thread pool. Type: float Unit: count |
QueryPipelineExecutorThreadsActive | Number of threads in the PipelineExecutor thread pool running a task. Type: float Unit: count |
QueryPreempted | Number of queries that are stopped and waiting due to 'priority' setting. Type: float Unit: count |
QueryThread | Number of query processing threads Type: float Unit: count |
RWLockActiveReaders | Number of threads holding read lock in a table RWLock. Type: float Unit: count |
RWLockActiveWriters | Number of threads holding write lock in a table RWLock. Type: float Unit: count |
RWLockWaitingReaders | Number of threads waiting for read on a table RWLock. Type: float Unit: count |
RWLockWaitingWriters | Number of threads waiting for write on a table RWLock. Type: float Unit: count |
Read | Number of read (read , pread , io_getevents , etc.) syscalls in flyType: float Unit: count |
ReadTaskRequestsSent | The current number of callback requests in flight from the remote server back to the initiator server to choose the read task (for s3Cluster table function and similar). Measured on the remote server side. Type: float Unit: count |
ReadonlyReplica | Number of Replicated tables that are currently in readonly state due to re-initialization after ZooKeeper session loss or due to startup without ZooKeeper configured. Type: float Unit: count |
RemoteRead | Number of read with remote reader in fly Type: float Unit: count |
ReplicatedChecks | Number of data parts checking for consistency Type: float Unit: count |
ReplicatedFetch | Number of data parts being fetched from replica Type: float Unit: count |
ReplicatedSend | Number of data parts being sent to replicas Type: float Unit: count |
RestartReplicaThreads | Number of threads in the RESTART REPLICA thread pool. Type: float Unit: count |
RestartReplicaThreadsActive | Number of threads in the RESTART REPLICA thread pool running a task. Type: float Unit: count |
RestoreThreads | Number of threads in the thread pool for RESTORE. Type: float Unit: count |
RestoreThreadsActive | Number of threads in the thread pool for RESTORE running a task. Type: float Unit: count |
Revision | Revision of the server. It is a number incremented for every release or release candidate except patch releases. Type: float Unit: count |
S3Requests | S3 requests Type: float Unit: count |
SendExternalTables | Number of connections that are sending data for external tables to remote servers. External tables are used to implement GLOBAL IN and GLOBAL JOIN operators with distributed subqueries .Type: float Unit: count |
SendScalars | Number of connections that are sending data for scalars to remote servers. Type: float Unit: count |
StartupSystemTablesThreads | Number of threads in the StartupSystemTables thread pool. Type: float Unit: count |
StartupSystemTablesThreadsActive | Number of threads in the StartupSystemTables thread pool running a task. Type: float Unit: count |
StorageBufferBytes | Number of bytes in buffers of Buffer tables Type: float Unit: digital,B |
StorageBufferRows | Number of rows in buffers of Buffer tables Type: float Unit: count |
StorageDistributedThreads | Number of threads in the StorageDistributed thread pool. Type: float Unit: count |
StorageDistributedThreadsActive | Number of threads in the StorageDistributed thread pool running a task. Type: float Unit: count |
StorageHiveThreads | Number of threads in the StorageHive thread pthreadpool ool.Type: float Unit: count |
StorageHiveThreadsActive | Number of threads in the StorageHive thread pool running a task. Type: float Unit: count |
StorageS3Threads | Number of threads in the StorageS3 thread pool. Type: float Unit: count |
StorageS3ThreadsActive | Number of threads in the StorageS3 thread pool running a task. Type: float Unit: count |
SyncDrainedConnections | Number of connections drained synchronously. Type: float Unit: count |
SystemReplicasThreads | Number of threads in the system.replicas thread pool. Type: float Unit: count |
SystemReplicasThreadsActive | Number of threads in the system.replicas thread pool running a task. Type: float Unit: count |
TCPConnection | Number of connections to TCP server (clients with native interface), also included server-server distributed query connections Type: float Unit: count |
TablesLoaderThreads | Number of threads in the tables loader thread pool. Type: float Unit: count |
TablesLoaderThreadsActive | Number of threads in the tables loader thread pool running a task. Type: float Unit: count |
TablesToDropQueueSize | Number of dropped tables, that are waiting for background data removal. Type: float Unit: count |
TemporaryFilesForAggregation | Number of temporary files created for external aggregation Type: float Unit: count |
TemporaryFilesForJoin | Number of temporary files created for JOIN Type: float Unit: count |
TemporaryFilesForSort | Number of temporary files created for external sorting Type: float Unit: count |
TemporaryFilesUnknown | Number of temporary files created without known purpose Type: float Unit: count |
ThreadPoolFSReaderThreads | Number of threads in the thread pool for local_filesystem_read_method=threadpool .Type: float Unit: count |
ThreadPoolFSReaderThreadsActive | Number of threads in the thread pool for local_filesystem_read_method=threadpool running a task.Type: float Unit: count |
ThreadPoolRemoteFSReaderThreads | Number of threads in the thread pool for remote_filesystem_read_method=threadpool .Type: float Unit: count |
ThreadPoolRemoteFSReaderThreadsActive | Number of threads in the thread pool for remote_filesystem_read_method=threadpool running a task.Type: float Unit: count |
ThreadsInOvercommitTracker | Number of waiting threads inside of OvercommitTracker Type: float Unit: count |
TotalTemporaryFiles | Number of temporary files created Type: float Unit: count |
VersionInteger | Version of the server in a single integer number in base-1000. For example, version 11.22.33 is translated to 11022033. Type: float Unit: count |
Write | Number of write (write /pwrite /io_getevents , etc.) syscalls in flyType: float Unit: count |
ZooKeeperRequest | Number of requests to ZooKeeper in fly. Type: float Unit: count |
ZooKeeperSession | Number of sessions (connections) to ZooKeeper. Should be no more than one, because using more than one connection to ZooKeeper may lead to bugs due to lack of linearizability (stale reads) that ZooKeeper consistency model allows.Type: float Unit: count |
ZooKeeperWatch | Number of watches (event subscriptions) in ZooKeeper. Type: float Unit: count |
ClickHouseProfileEvents
¶
- Tags
Tag | Description |
---|---|
host | Host name |
instance | Instance endpoint |
- Metrics
Metric | Description |
---|---|
AIORead | Number of reads with Linux or FreeBSD AIO interface Type: float Unit: count |
AIOReadBytes | Number of bytes read with Linux or FreeBSD AIO interface Type: float Unit: digital,B |
AIOWrite | Number of writes with Linux or FreeBSD AIO interface Type: float Unit: count |
AIOWriteBytes | Number of bytes written with Linux or FreeBSD AIO interface Type: float Unit: digital,B |
AggregationHashTablesInitializedAsTwoLevel | How many hash tables were inited as two-level for aggregation.Type: float Unit: count |
AggregationPreallocatedElementsInHashTables | How many elements were preallocated in hash tables for aggregation. Type: float Unit: count |
ArenaAllocBytes | Number of bytes allocated for memory Arena (used for GROUP BY and similar operations) Type: float Unit: count |
ArenaAllocChunks | Number of chunks allocated for memory Arena (used for GROUP BY and similar operations) Type: float Unit: count |
AsyncInsertBytes | Data size in bytes of asynchronous INSERT queries. Type: float Unit: digital,B |
AsyncInsertCacheHits | Number of times a duplicate hash id has been found in asynchronous INSERT hash id cache. Type: float Unit: count |
AsyncInsertQuery | Same as InsertQuery, but only for asynchronous INSERT queries. Type: float Unit: count |
AsynchronousReadWaitMicroseconds | Time spent in waiting for asynchronous reads. Type: float Unit: time,ms |
AsynchronousRemoteReadWaitMicroseconds | Time spent in waiting for asynchronous remote reads. Type: float Unit: time,ms |
BackgroundLoadingMarksTasks | Number of background tasks for loading marks Type: float Unit: count |
CachedReadBufferCacheWriteBytes | Bytes written from source (remote fs, etc) to filesystem cache Type: float Unit: digital,B |
CachedReadBufferCacheWriteMicroseconds | Time spent writing data into filesystem cache Type: float Unit: time,ms |
CachedReadBufferReadFromCacheBytes | Bytes read from filesystem cache Type: float Unit: digital,B |
CachedReadBufferReadFromCacheMicroseconds | Time reading from filesystem cache Type: float Unit: time,ms |
CachedReadBufferReadFromSourceBytes | Bytes read from filesystem cache source (from remote fs, etc) Type: float Unit: digital,B |
CachedReadBufferReadFromSourceMicroseconds | Time reading from filesystem cache source (from remote filesystem, etc) Type: float Unit: time,ms |
CachedWriteBufferCacheWriteBytes | Bytes written from source (remote fs, etc) to filesystem cache Type: float Unit: digital,B |
CachedWriteBufferCacheWriteMicroseconds | Time spent writing data into filesystem cache Type: float Unit: time,ms |
CannotRemoveEphemeralNode | Number of times an error happened while trying to remove ephemeral node. This is not an issue, because our implementation of ZooKeeper library guarantee that the session will expire and the node will be removed. Type: float Unit: count |
CannotWriteToWriteBufferDiscard | Number of stack traces dropped by query profiler or signal handler because pipe is full or cannot write to pipe. Type: float Unit: count |
CompileExpressionsBytes | Number of bytes used for expressions compilation. Type: float Unit: digital,B |
CompileExpressionsMicroseconds | Total time spent for compilation of expressions to LLVM code. Type: float Unit: time,ms |
CompileFunction | Number of times a compilation of generated LLVM code (to create fused function for complex expressions) was initiated. Type: float Unit: count |
CompiledFunctionExecute | Number of times a compiled function was executed. Type: float Unit: count |
CompressedReadBufferBlocks | Number of compressed blocks (the blocks of data that are compressed independent of each other) read from compressed sources (files, network). Type: float Unit: count |
CompressedReadBufferBytes | Number of uncompressed bytes (the number of bytes after decompression) read from compressed sources (files, network). Type: float Unit: digital,B |
ContextLock | Number of times the lock of Context was acquired or tried to acquire. This is global lock. Type: float Unit: count |
CreatedHTTPConnections | Total amount of created HTTP connections (counter increase every time connection is created). Type: float Unit: count |
CreatedLogEntryForMerge | Successfully created log entry to merge parts in ReplicatedMergeTree. Type: float Unit: count |
CreatedLogEntryForMutation | Successfully created log entry to mutate parts in ReplicatedMergeTree. Type: float Unit: count |
CreatedReadBufferAIO | Created read buffer AIO Type: float Unit: count |
CreatedReadBufferAIOFailed | Created read buffer AIO Failed Type: float Unit: count |
CreatedReadBufferDirectIO | Number of times a read buffer with O_DIRECT was created for reading data (while choosing among other read methods). Type: float Unit: count |
CreatedReadBufferDirectIOFailed | Number of times a read buffer with O_DIRECT was attempted to be created for reading data (while choosing among other read methods), but the OS did not allow it (due to lack of filesystem support or other reasons) and we fallen back to the ordinary reading method. Type: float Unit: count |
CreatedReadBufferMMap | Number of times a read buffer using mmap was created for reading data (while choosing among other read methods).Type: float Unit: count |
CreatedReadBufferMMapFailed | Number of times a read buffer with mmap was attempted to be created for reading data (while choosing among other read methods), but the OS did not allow it (due to lack of filesystem support or other reasons) and we fallen back to the ordinary reading method.Type: float Unit: count |
CreatedReadBufferOrdinary | Number of times ordinary read buffer was created for reading data (while choosing among other read methods). Type: float Unit: count |
DNSError | Total count of errors in DNS resolution Type: float Unit: count |
DataAfterMergeDiffersFromReplica | Number of times data after merge is not byte-identical to the data on another replicas. There could be several reasons Type: float Unit: count |
DataAfterMutationDiffersFromReplica | Number of times data after mutation is not byte-identical to the data on another replicas. In addition to the reasons described in 'DataAfterMergeDiffersFromReplica', it is also possible due to non-deterministic mutation. Type: float Unit: count |
DelayedInserts | Number of times the INSERT of a block to a MergeTree table was throttled due to high number of active data parts for partition. Type: float Unit: count |
DelayedInsertsMilliseconds | Total number of milliseconds spent while the INSERT of a block to a MergeTree table was throttled due to high number of active data parts for partition. Type: float Unit: time,ms |
DictCacheKeysExpired | Number of keys looked up in the dictionaries of 'cache' types and found in the cache but they were obsolete. Type: float Unit: count |
DictCacheKeysHit | Number of keys looked up in the dictionaries of 'cache' types and found in the cache. Type: float Unit: count |
DictCacheKeysNotFound | Number of keys looked up in the dictionaries of 'cache' types and not found. Type: float Unit: count |
DictCacheKeysRequested | Number of keys requested from the data source for the dictionaries of 'cache' types. Type: float Unit: count |
DictCacheKeysRequestedFound | Number of keys requested from the data source for dictionaries of 'cache' types and found in the data source. Type: float Unit: count |
DictCacheKeysRequestedMiss | Number of keys requested from the data source for dictionaries of 'cache' types but not found in the data source. Type: float Unit: count |
DictCacheLockReadNs | Number of nanoseconds spend in waiting for read lock to lookup the data for the dictionaries of 'cache' types. Type: float Unit: time,ms |
DictCacheLockWriteNs | Number of nanoseconds spend in waiting for write lock to update the data for the dictionaries of 'cache' types. Type: float Unit: time,ms |
DictCacheRequestTimeNs | Number of nanoseconds spend in querying the external data sources for the dictionaries of 'cache' types. Type: float Unit: time,ms |
DictCacheRequests | Number of bulk requests to the external data sources for the dictionaries of 'cache' types. Type: float Unit: count |
DirectorySync | Number of times the F_FULLFSYNC/fsync/fdatasync function was called for directories.Type: float Unit: count |
DirectorySyncElapsedMicroseconds | Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for directories.Type: float Unit: time,ms |
DiskReadElapsedMicroseconds | Total time spent waiting for read syscall. This include reads from page cache. Type: float Unit: time,ms |
DiskS3AbortMultipartUpload | Number of DiskS3 API AbortMultipartUpload calls. Type: float Unit: count |
DiskS3CompleteMultipartUpload | Number of DiskS3 API CompleteMultipartUpload calls. Type: float Unit: count |
DiskS3CopyObject | Number of DiskS3 API CopyObject calls. Type: float Unit: count |
DiskS3CreateMultipartUpload | Number of DiskS3 API CreateMultipartUpload calls. Type: float Unit: count |
DiskS3DeleteObjects | Number of DiskS3 API DeleteObject(s) calls. Type: float Unit: count |
DiskS3GetObject | Number of DiskS3 API GetObject calls. Type: float Unit: count |
DiskS3GetObjectAttributes | Number of DiskS3 API GetObjectAttributes calls. Type: float Unit: count |
DiskS3GetRequestThrottlerCount | Number of DiskS3 GET and SELECT requests passed through throttler. Type: float Unit: count |
DiskS3GetRequestThrottlerSleepMicroseconds | Total time a query was sleeping to conform DiskS3 GET and SELECT request throttling. Type: float Unit: time,ms |
DiskS3HeadObject | Number of DiskS3 API HeadObject calls. Type: float Unit: count |
DiskS3ListObjects | Number of DiskS3 API ListObjects calls. Type: float Unit: count |
DiskS3PutObject | Number of DiskS3 API PutObject calls. Type: float Unit: count |
DiskS3PutRequestThrottlerCount | Number of DiskS3 PUT, COPY, POST and LIST requests passed through throttler. Type: float Unit: count |
DiskS3PutRequestThrottlerSleepMicroseconds | Total time a query was sleeping to conform DiskS3 PUT, COPY, POST and LIST request throttling. Type: float Unit: time,ms |
DiskS3ReadMicroseconds | Time of GET and HEAD requests to DiskS3 storage. Type: float Unit: time,ms |
DiskS3ReadRequestsCount | Number of GET and HEAD requests to DiskS3 storage. Type: float Unit: count |
DiskS3ReadRequestsErrors | Number of non-throttling errors in GET and HEAD requests to DiskS3 storage. Type: float Unit: count |
DiskS3ReadRequestsRedirects | Number of redirects in GET and HEAD requests to DiskS3 storage. Type: float Unit: count |
DiskS3ReadRequestsThrottling | Number of 429 and 503 errors in GET and HEAD requests to DiskS3 storage. Type: float Unit: count |
DiskS3UploadPart | Number of DiskS3 API UploadPart calls. Type: float Unit: count |
DiskS3UploadPartCopy | Number of DiskS3 API UploadPartCopy calls. Type: float Unit: count |
DiskS3WriteMicroseconds | Time of POST, DELETE, PUT and PATCH requests to DiskS3 storage. Type: float Unit: count |
DiskS3WriteRequestsCount | Number of POST, DELETE, PUT and PATCH requests to DiskS3 storage. Type: float Unit: count |
DiskS3WriteRequestsErrors | Number of non-throttling errors in POST, DELETE, PUT and PATCH requests to DiskS3 storage. Type: float Unit: count |
DiskS3WriteRequestsRedirects | Number of redirects in POST, DELETE, PUT and PATCH requests to DiskS3 storage. Type: float Unit: count |
DiskS3WriteRequestsThrottling | Number of 429 and 503 errors in POST, DELETE, PUT and PATCH requests to DiskS3 storage. Type: float Unit: count |
DiskWriteElapsedMicroseconds | Total time spent waiting for write syscall. This include writes to page cache. Type: float Unit: time,ms |
DistributedConnectionFailAtAll | Total count when distributed connection fails after all retries finished. Type: float Unit: time,ms |
DistributedConnectionFailTry | Total count when distributed connection fails with retry. Type: float Unit: time,ms |
DistributedConnectionMissingTable | Number of times we rejected a replica from a distributed query, because it did not contain a table needed for the query. Type: float Unit: count |
DistributedConnectionStaleReplica | Number of times we rejected a replica from a distributed query, because some table needed for a query had replication lag higher than the configured threshold. Type: float Unit: count |
DistributedDelayedInserts | Number of times the INSERT of a block to a Distributed table was throttled due to high number of pending bytes. Type: float Unit: count |
DistributedDelayedInsertsMilliseconds | Total number of milliseconds spent while the INSERT of a block to a Distributed table was throttled due to high number of pending bytes. Type: float Unit: time,ms |
DistributedRejectedInserts | Number of times the INSERT of a block to a Distributed table was rejected with 'Too many bytes' exception due to high number of pending bytes. Type: float Unit: count |
DistributedSyncInsertionTimeoutExceeded | A timeout has exceeded while waiting for shards during synchronous insertion into a Distributed table (with 'insert_distributed_sync' = 1) Type: float Unit: count |
DuplicatedInsertedBlocks | Number of times the INSERTed block to a ReplicatedMergeTree table was deduplicated. Type: float Unit: count |
ExecuteShellCommand | Number of shell command executions. Type: float Unit: count |
ExternalAggregationCompressedBytes | Number of bytes written to disk for aggregation in external memory. Type: float Unit: digital,B |
ExternalAggregationMerge | Number of times temporary files were merged for aggregation in external memory. Type: float Unit: count |
ExternalAggregationUncompressedBytes | Amount of data (uncompressed, before compression) written to disk for aggregation in external memory. Type: float Unit: digital,B |
ExternalAggregationWritePart | Number of times a temporary file was written to disk for aggregation in external memory. Type: float Unit: count |
ExternalDataSourceLocalCacheReadBytes | Bytes read from local cache buffer in RemoteReadBufferCache Type: float Unit: digital,B |
ExternalJoinCompressedBytes | Number of compressed bytes written for JOIN in external memory. Type: float Unit: digital,B |
ExternalJoinMerge | Number of times temporary files were merged for JOIN in external memory. Type: float Unit: count |
ExternalJoinUncompressedBytes | Amount of data (uncompressed, before compression) written for JOIN in external memory. Type: float Unit: digital,B |
ExternalJoinWritePart | Number of times a temporary file was written to disk for JOIN in external memory. Type: float Unit: count |
ExternalProcessingCompressedBytesTotal | Number of compressed bytes written by external processing (sorting/aggravating/joining) Type: float Unit: digital,B |
ExternalProcessingFilesTotal | Number of files used by external processing (sorting/aggravating/joining) Type: float Unit: count |
ExternalProcessingUncompressedBytesTotal | Amount of data (uncompressed, before compression) written by external processing (sorting/aggravating/joining) Type: float Unit: digital,B |
ExternalSortCompressedBytes | Number of compressed bytes written for sorting in external memory. Type: float Unit: digital,B |
ExternalSortMerge | Number of times temporary files were merged for sorting in external memory. Type: float Unit: count |
ExternalSortUncompressedBytes | Amount of data (uncompressed, before compression) written for sorting in external memory. Type: float Unit: digital,B |
ExternalSortWritePart | Number of times a temporary file was written to disk for sorting in external memory. Type: float Unit: count |
FailedAsyncInsertQuery | Number of failed ASYNC INSERT queries. Type: float Unit: count |
FailedInsertQuery | Same as FailedQuery, but only for INSERT queries. Type: float Unit: count |
FailedQuery | Number of failed queries. Type: float Unit: count |
FailedSelectQuery | Same as FailedQuery, but only for SELECT queries. Type: float Unit: count |
FileOpen | Number of files opened. Type: float Unit: count |
FileSegmentCacheWriteMicroseconds | Metric per file segment. Time spend writing data to cache Type: float Unit: time,ms |
FileSegmentPredownloadMicroseconds | Metric per file segment. Time spent predownloading data to cache (predownloading - finishing file segment download (after someone who failed to do that) up to the point current thread was requested to do)Type: float Unit: time,ms |
FileSegmentReadMicroseconds | Metric per file segment. Time spend reading from file Type: float Unit: time,ms |
FileSegmentUsedBytes | Metric per file segment. How many bytes were actually used from current file segment Type: float Unit: digital,B |
FileSegmentWaitReadBufferMicroseconds | Metric per file segment. Time spend waiting for internal read buffer (includes cache waiting) Type: float Unit: time,ms |
FileSegmentWriteMicroseconds | Metric per file segment. Time spend writing cache Type: float Unit: time,ms |
FileSync | Number of times the F_FULLFSYNC/fsync/fdatasync function was called for files.Type: float Unit: count |
FileSyncElapsedMicroseconds | Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for files.Type: float Unit: time,ms |
FunctionExecute | Number of SQL ordinary function calls (SQL functions are called on per-block basis, so this number represents the number of blocks). Type: float Unit: count |
HardPageFaults | The number of hard page faults in query execution threads. High values indicate either that you forgot to turn off swap on your server, or eviction of memory pages of the ClickHouse binary during very high memory pressure, or successful usage of the mmap read method for the tables data.Type: float Unit: count |
HedgedRequestsChangeReplica | Total count when timeout for changing replica expired in hedged requests. Type: float Unit: count |
IOBufferAllocBytes | Number of bytes allocated for IO buffers (for ReadBuffer/WriteBuffer). Type: float Unit: digital,B |
IOBufferAllocs | Number of allocations of IO buffers (for ReadBuffer/WriteBuffer). Type: float Unit: count |
IOUringCQEsCompleted | Total number of successfully completed io_uring CQEs Type: float Unit: count |
IOUringCQEsFailed | Total number of completed io_uring CQEs with failures Type: float Unit: count |
IOUringSQEsResubmits | Total number of io_uring SQE resubmits performed Type: float Unit: count |
IOUringSQEsSubmitted | Total number of io_uring SQEs submitted Type: float Unit: count |
InsertQuery | Same as Query, but only for INSERT queries. Type: float Unit: count |
InsertQueryTimeMicroseconds | Total time of INSERT queries. Type: float Unit: time,ms |
InsertedBytes | Number of bytes (uncompressed; for columns as they stored in memory) INSERTed to all tables. Type: float Unit: digital,B |
InsertedCompactParts | Number of parts inserted in Compact format. Type: float Unit: count |
InsertedInMemoryParts | Number of parts inserted in InMemory format. Type: float Unit: count |
InsertedRows | Number of rows INSERTed to all tables. Type: float Unit: count |
InsertedWideParts | Number of parts inserted in Wide format. Type: float Unit: count |
InvoluntaryContextSwitches | Involuntary context switches Type: float Unit: count |
KafkaBackgroundReads | Number of background reads populating materialized views from Kafka since server start Type: float Unit: count |
KafkaCommitFailures | Number of failed commits of consumed offsets to Kafka (usually is a sign of some data duplication) Type: float Unit: count |
KafkaCommits | Number of successful commits of consumed offsets to Kafka (normally should be the same as KafkaBackgroundReads) Type: float Unit: count |
KafkaConsumerErrors | Number of errors reported by librdkafka during pollsType: float Unit: count |
KafkaDirectReads | Number of direct selects from Kafka tables since server start Type: float Unit: count |
KafkaMessagesFailed | Number of Kafka messages ClickHouse failed to parse Type: float Unit: count |
KafkaMessagesPolled | Number of Kafka messages polled from librdkafka to ClickHouseType: float Unit: count |
KafkaMessagesProduced | Number of messages produced to Kafka Type: float Unit: count |
KafkaMessagesRead | Number of Kafka messages already processed by ClickHouse Type: float Unit: count |
KafkaProducerErrors | Number of errors during producing the messages to Kafka Type: float Unit: count |
KafkaProducerFlushes | Number of explicit flushes to Kafka producer Type: float Unit: count |
KafkaRebalanceAssignments | Number of partition assignments (the final stage of consumer group rebalance )Type: float Unit: count |
KafkaRebalanceErrors | Number of failed consumer group rebalances Type: float Unit: count |
KafkaRebalanceRevocations | Number of partition revocations (the first stage of consumer group rebalance )Type: float Unit: count |
KafkaRowsRead | Number of rows parsed from Kafka messages Type: float Unit: count |
KafkaRowsRejected | Number of parsed rows which were later rejected (due to rebalances / errors or similar reasons). Those rows will be consumed again after the rebalance .Type: float Unit: count |
KafkaRowsWritten | Number of rows inserted into Kafka tables Type: float Unit: count |
KafkaWrites | Number of writes (inserts) to Kafka tables Type: float Unit: count |
KeeperCheckRequest | Number of check requests Type: float Unit: count |
KeeperCommits | Number of successful commits Type: float Unit: count |
KeeperCommitsFailed | Number of failed commits Type: float Unit: count |
KeeperCreateRequest | Number of create requests Type: float Unit: count |
KeeperExistsRequest | Number of exists requests Type: float Unit: count |
KeeperGetRequest | Number of get requests Type: float Unit: count |
KeeperLatency | Keeper latency Type: float Unit: count |
KeeperListRequest | Number of list requests Type: float Unit: count |
KeeperMultiReadRequest | Number of multi read requests Type: float Unit: count |
KeeperMultiRequest | Number of multi requests Type: float Unit: count |
KeeperPacketsReceived | Packets received by keeper server Type: float Unit: count |
KeeperPacketsSent | Packets sent by keeper server Type: float Unit: count |
KeeperReadSnapshot | Number of snapshot read(serialization) Type: float Unit: count |
KeeperRemoveRequest | Number of remove requests Type: float Unit: count |
KeeperRequestTotal | Total requests number on keeper server Type: float Unit: count |
KeeperSaveSnapshot | Number of snapshot save Type: float Unit: count |
KeeperSetRequest | Number of set requests Type: float Unit: count |
KeeperSnapshotApplys | Number of snapshot applying Type: float Unit: count |
KeeperSnapshotApplysFailed | Number of failed snapshot applying Type: float Unit: count |
KeeperSnapshotCreations | Number of snapshots creations Type: float Unit: count |
KeeperSnapshotCreationsFailed | Number of failed snapshot creations Type: float Unit: count |
LoadedMarksCount | Number of marks loaded (total across columns). Type: float Unit: count |
LoadedMarksMemoryBytes | Size of in-memory representations of loaded marks. Type: float Unit: digital,B |
LocalReadThrottlerBytes | Bytes passed through 'max_local_read_bandwidth_for_server'/'max_local_read_bandwidth' throttler. Type: float Unit: digital,B |
LocalReadThrottlerSleepMicroseconds | Total time a query was sleeping to conform 'max_local_read_bandwidth_for_server'/'max_local_read_bandwidth' throttling. Type: float Unit: time,ms |
LocalWriteThrottlerBytes | Bytes passed through 'max_local_write_bandwidth_for_server'/'max_local_write_bandwidth' throttler. Type: float Unit: digital,B |
LocalWriteThrottlerSleepMicroseconds | Total time a query was sleeping to conform 'max_local_write_bandwidth_for_server'/'max_local_write_bandwidth' throttling. Type: float Unit: time,ms |
LogDebug | Number of log messages with level Debug Type: float Unit: count |
LogError | Number of log messages with level Error Type: float Unit: count |
LogFatal | Number of log messages with level Fatal Type: float Unit: count |
LogInfo | Number of log messages with level Info Type: float Unit: count |
LogTest | Number of log messages with level Test Type: float Unit: count |
LogTrace | Number of log messages with level Trace Type: float Unit: count |
LogWarning | Number of log messages with level Warning Type: float Unit: count |
MMappedFileCacheHits | Number of times a file has been found in the MMap cache (for the mmap read_method), so we didn't have to mmap it again.Type: float Unit: count |
MMappedFileCacheMisses | Number of times a file has not been found in the MMap cache (for the mmap read_method), so we had to mmap it again.Type: float Unit: count |
MainConfigLoads | Number of times the main configuration was reloaded. Type: float Unit: count |
MarkCacheHits | Number of times an entry has been found in the mark cache, so we didn't have to load a mark file. Type: float Unit: count |
MarkCacheMisses | Number of times an entry has not been found in the mark cache, so we had to load a mark file in memory, which is a costly operation, adding to query latency. Type: float Unit: count |
MemoryAllocatorPurge | Total number of times memory allocator purge was requested Type: float Unit: time,ms |
MemoryAllocatorPurgeTimeMicroseconds | Total number of times memory allocator purge was requested Type: float Unit: time,ms |
MemoryOvercommitWaitTimeMicroseconds | Total time spent in waiting for memory to be freed in OvercommitTracker .Type: float Unit: time,ms |
Merge | Number of launched background merges. Type: float Unit: count |
MergeTreeAllRangesAnnouncementsSent | The number of announcement sent from the remote server to the initiator server about the set of data parts (for MergeTree tables). Measured on the remote server side. Type: float Unit: count |
MergeTreeAllRangesAnnouncementsSentElapsedMicroseconds | Time spent in sending the announcement from the remote server to the initiator server about the set of data parts (for MergeTree tables). Measured on the remote server side. Type: float Unit: time,ms |
MergeTreeDataProjectionWriterBlocks | Number of blocks INSERTed to MergeTree tables projection. Each block forms a data part of level zero. Type: float Unit: count |
MergeTreeDataProjectionWriterBlocksAlreadySorted | Number of blocks INSERTed to MergeTree tables projection that appeared to be already sorted. Type: float Unit: count |
MergeTreeDataProjectionWriterCompressedBytes | Bytes written to filesystem for data INSERTed to MergeTree tables projection. Type: float Unit: digital,B |
MergeTreeDataProjectionWriterRows | Number of rows INSERTed to MergeTree tables projection. Type: float Unit: count |
MergeTreeDataProjectionWriterUncompressedBytes | Uncompressed bytes (for columns as they stored in memory) INSERTed to MergeTree tables projection. Type: float Unit: digital,B |
MergeTreeDataWriterBlocks | Number of blocks INSERTed to MergeTree tables. Each block forms a data part of level zero. Type: float Unit: count |
MergeTreeDataWriterBlocksAlreadySorted | Number of blocks INSERTed to MergeTree tables that appeared to be already sorted. Type: float Unit: count |
MergeTreeDataWriterCompressedBytes | Bytes written to filesystem for data INSERTed to MergeTree tables. Type: float Unit: digital,B |
MergeTreeDataWriterRows | Number of rows INSERTed to MergeTree tables. Type: float Unit: count |
MergeTreeDataWriterUncompressedBytes | Uncompressed bytes (for columns as they stored in memory) INSERTed to MergeTree tables. Type: float Unit: digital,B |
MergeTreeMetadataCacheDelete | Number of rocksdb deletes(used for merge tree metadata cache)Type: float Unit: count |
MergeTreeMetadataCacheGet | Number of rocksdb reads(used for merge tree metadata cache)Type: float Unit: count |
MergeTreeMetadataCacheHit | Number of times the read of meta file was done from MergeTree metadata cache Type: float Unit: count |
MergeTreeMetadataCacheMiss | Number of times the read of meta file was not done from MergeTree metadata cache Type: float Unit: count |
MergeTreeMetadataCachePut | Number of rocksdb puts(used for merge tree metadata cache)Type: float Unit: count |
MergeTreeMetadataCacheSeek | Number of rocksdb seeks(used for merge tree metadata cache)Type: float Unit: count |
MergeTreePrefetchedReadPoolInit | Time spent preparing tasks in MergeTreePrefetchedReadPool Type: float Unit: time,ms |
MergeTreeReadTaskRequestsReceived | The number of callbacks requested from the remote server back to the initiator server to choose the read task (for MergeTree tables). Measured on the initiator server side. Type: float Unit: count |
MergeTreeReadTaskRequestsSent | The number of callbacks requested from the remote server back to the initiator server to choose the read task (for MergeTree tables). Measured on the remote server side. Type: float Unit: count |
MergeTreeReadTaskRequestsSentElapsedMicroseconds | Time spent in callbacks requested from the remote server back to the initiator server to choose the read task (for MergeTree tables). Measured on the remote server side. Type: float Unit: count |
MergedIntoCompactParts | Number of parts merged into Compact format. Type: float Unit: count |
MergedIntoInMemoryParts | Number of parts in merged into InMemory format. Type: float Unit: count |
MergedIntoWideParts | Number of parts merged into Wide format. Type: float Unit: count |
MergedRows | Rows read for background merges. This is the number of rows before merge. Type: float Unit: count |
MergedUncompressedBytes | Uncompressed bytes (for columns as they stored in memory) that was read for background merges. This is the number before merge. Type: float Unit: digital,B |
MergesTimeMilliseconds | Total time spent for background merges. Type: float Unit: time,ms |
NetworkReceiveBytes | Total number of bytes received from network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. Type: float Unit: digital,B |
NetworkReceiveElapsedMicroseconds | Total time spent waiting for data to receive or receiving data from network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. Type: float Unit: time,ms |
NetworkSendBytes | Total number of bytes send to network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. Type: float Unit: digital,B |
NetworkSendElapsedMicroseconds | Total time spent waiting for data to send to network or sending data to network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. Type: float Unit: time,ms |
NotCreatedLogEntryForMerge | Log entry to merge parts in ReplicatedMergeTree is not created due to concurrent log update by another replica. Type: float Unit: count |
NotCreatedLogEntryForMutation | Log entry to mutate parts in ReplicatedMergeTree is not created due to concurrent log update by another replica. Type: float Unit: count |
OSCPUVirtualTimeMicroseconds | CPU time spent seen by OS. Does not include involuntary waits due to virtualization. Type: float Unit: time,ms |
OSCPUWaitMicroseconds | Total time a thread was ready for execution but waiting to be scheduled by OS, from the OS point of view. Type: float Unit: time,ms |
OSIOWaitMicroseconds | Total time a thread spent waiting for a result of IO operation, from the OS point of view. This is real IO that does not include page cache. Type: float Unit: time,ms |
OSReadBytes | Number of bytes read from disks or block devices. Does not include bytes read from page cache. May include excessive data due to block size, readahead , etc.Type: float Unit: digital,B |
OSReadChars | Number of bytes read from filesystem, including page cache. Type: float Unit: digital,B |
OSWriteBytes | Number of bytes written to disks or block devices. Does not include bytes that are in page cache dirty pages. May not include data that was written by OS asynchronously. Type: float Unit: digital,B |
OSWriteChars | Number of bytes written to filesystem, including page cache. Type: float Unit: digital,B |
ObsoleteReplicatedParts | Number of times a data part was covered by another data part that has been fetched from a replica (so, we have marked a covered data part as obsolete and no longer needed). Type: float Unit: count |
OpenedFileCacheHits | Number of times a file has been found in the opened file cache, so we didn't have to open it again. Type: float Unit: count |
OpenedFileCacheMisses | Number of times a file has been found in the opened file cache, so we had to open it again. Type: float Unit: count |
OtherQueryTimeMicroseconds | Total time of queries that are not SELECT or INSERT. Type: float Unit: time,ms |
OverflowAny | Number of times approximate GROUP BY was in effect: when aggregation was performed only on top of first 'max_rows_to_group_by' unique keys and other keys were ignored due to 'group_by_overflow_mode' = 'any'. Type: float Unit: count |
OverflowBreak | Number of times, data processing was canceled by query complexity limitation with setting '_overflow_mode' = 'break' and the result is incomplete. Type: float Unit: count* |
OverflowThrow | Number of times, data processing was canceled by query complexity limitation with setting '_overflow_mode' = 'throw' and exception was thrown. Type: float Unit: count* |
PerfAlignmentFaults | Number of alignment faults. These happen when unaligned memory accesses happen; the kernel can handle these but it reduces performance. This happens only on some architectures (never on x86). Type: float Unit: count |
PerfBranchInstructions | Retired branch instructions. Prior to Linux 2.6.35, this used the wrong event on AMD processors. Type: float Unit: count |
PerfBranchMisses | Mispredicted branch instructions.Type: float Unit: count |
PerfBusCycles | Bus cycles, which can be different from total cycles. Type: float Unit: count |
PerfCacheMisses | Cache misses. Usually this indicates Last Level Cache misses; this is intended to be used in con-junction with the PERFCOUNTHWCACHEREFERENCES event to calculate cache miss rates.Type: float Unit: count |
PerfCacheReferences | Cache accesses. Usually this indicates Last Level Cache accesses but this may vary depending on your CPU. This may include prefetches and coherency messages; again this depends on the design of your CPU. Type: float Unit: count |
PerfContextSwitches | Number of context switches Type: float Unit: count |
PerfCpuClock | The CPU clock, a high-resolution per-CPU timer Type: float Unit: count |
PerfCpuCycles | Total cycles. Be wary of what happens during CPU frequency scaling. Type: float Unit: count |
PerfCpuMigrations | Number of times the process has migrated to a new CPU Type: float Unit: count |
PerfDataTLBMisses | Data TLB misses Type: float Unit: count |
PerfDataTLBReferences | Data TLB references Type: float Unit: count |
PerfEmulationFaults | Number of emulation faults. The kernel sometimes traps on unimplemented instructions and emulates them for user space. This can negatively impact performance. Type: float Unit: count |
PerfInstructionTLBMisses | Instruction TLB misses Type: float Unit: count |
PerfInstructionTLBReferences | Instruction TLB references Type: float Unit: count |
PerfInstructions | Retired instructions. Be careful, these can be affected by various issues, most notably hardware interrupt counts. Type: float Unit: count |
PerfLocalMemoryMisses | Local NUMA node memory read missesubqueriess Type: float Unit: digital,B |
PerfLocalMemoryReferences | Local NUMA node memory reads Type: float Unit: digital,B |
PerfMinEnabledRunningTime | Running time for event with minimum enabled time. Used to track the amount of event multiplexing Type: float Unit: time,ms |
PerfMinEnabledTime | For all events, minimum time that an event was enabled. Used to track event multiplexing influence Type: float Unit: time,ms |
PerfRefCpuCycles | Total cycles; not affected by CPU frequency scaling. Type: float Unit: count |
PerfStalledCyclesBackend | Stalled cycles during retirement. Type: float Unit: count |
PerfStalledCyclesFrontend | Stalled cycles during issue. Type: float Unit: count |
PerfTaskClock | A clock count specific to the task that is running Type: float Unit: count |
PolygonsAddedToPool | A polygon has been added to the cache (pool) for the 'pointInPolygon' function. Type: float Unit: count |
PolygonsInPoolAllocatedBytes | The number of bytes for polygons added to the cache (pool) for the 'pointInPolygon' function. Type: float Unit: digital,B |
Query | Number of queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries .Type: float Unit: count |
QueryCacheHits | Number of times a query result has been found in the query cache (and query computation was avoided). Type: float Unit: count |
QueryCacheMisses | Number of times a query result has not been found in the query cache (and required query computation). Type: float Unit: count |
QueryMaskingRulesMatch | Number of times query masking rules was successfully matched. Type: float Unit: count |
QueryMemoryLimitExceeded | Number of times when memory limit exceeded for query. Type: float Unit: count |
QueryProfilerRuns | Number of times QueryProfiler had been run. Type: float Unit: count |
QueryProfilerSignalOverruns | Number of times we drop processing of a query profiler signal due to overrun plus the number of signals that OS has not delivered due to overrun. Type: float Unit: count |
QueryTimeMicroseconds | Total time of all queries. Type: float Unit: time,ms |
RWLockAcquiredReadLocks | Number of times a read lock was acquired (in a heavy RWLock). Type: float Unit: count |
RWLockAcquiredWriteLocks | Number of times a write lock was acquired (in a heavy RWLock). Type: float Unit: count |
RWLockReadersWaitMilliseconds | Total time spent waiting for a read lock to be acquired (in a heavy RWLock). Type: float Unit: time,ms |
RWLockWritersWaitMilliseconds | Total time spent waiting for a write lock to be acquired (in a heavy RWLock). Type: float Unit: time,ms |
ReadBackoff | Number of times the number of query processing threads was lowered due to slow reads. Type: float Unit: count |
ReadBufferAIORead | Read buffer AIO read Type: float Unit: count |
ReadBufferAIOReadBytes | Read buffer AIO read bytes Type: float Unit: digital,B |
ReadBufferFromFileDescriptorRead | Number of reads (read /pread ) from a file descriptor. Does not include sockets.Type: float Unit: count |
ReadBufferFromFileDescriptorReadBytes | Number of bytes read from file descriptors. If the file is compressed, this will show the compressed data size. Type: float Unit: digital,B |
ReadBufferFromFileDescriptorReadFailed | Number of times the read (read /pread ) from a file descriptor have failed.Type: float Unit: count |
ReadBufferFromS3Bytes | Bytes read from S3. Type: float Unit: digital,B |
ReadBufferFromS3InitMicroseconds | Time spent initializing connection to S3. Type: float Unit: time,ms |
ReadBufferFromS3Microseconds | Time spent on reading from S3. Type: float Unit: time,ms |
ReadBufferFromS3RequestsErrors | Number of exceptions while reading from S3. Type: float Unit: count |
ReadBufferSeekCancelConnection | Number of seeks which lead to new connection (s3, http) Type: float Unit: count |
ReadCompressedBytes | Number of bytes (the number of bytes before decompression) read from compressed sources (files, network). Type: float Unit: digital,B |
ReadTaskRequestsReceived | The number of callbacks requested from the remote server back to the initiator server to choose the read task (for s3Cluster table function and similar). Measured on the initiator server side. Type: float Unit: count |
ReadTaskRequestsSent | The number of callbacks requested from the remote server back to the initiator server to choose the read task (for s3Cluster table function and similar). Measured on the remote server side. Type: float Unit: count |
ReadTaskRequestsSentElapsedMicroseconds | Time spent in callbacks requested from the remote server back to the initiator server to choose the read task (for s3Cluster table function and similar). Measured on the remote server side. Type: float Unit: time,ms |
RealTimeMicroseconds | Total (wall clock) time spent in processing (queries and other tasks) threads (note that this is a sum). Type: float Unit: time,ms |
RegexpCreated | Compiled regular expressions. Identical regular expressions compiled just once and cached forever. Type: float Unit: count |
RejectedInserts | Number of times the INSERT of a block to a MergeTree table was rejected with 'Too many parts' exception due to high number of active data parts for partition. Type: float Unit: count |
RemoteFSBuffers | Number of buffers created for asynchronous reading from remote filesystem Type: float Unit: count |
RemoteFSCancelledPrefetches | Number of canceled prefecthes (because of seek)Type: float Unit: count |
RemoteFSLazySeeks | Number of lazy seeks Type: float Unit: count |
RemoteFSPrefetchedBytes | Number of bytes from prefetched buffer Type: float Unit: count |
RemoteFSPrefetchedReads | Number of reads from prefetched buffer Type: float Unit: count |
RemoteFSPrefetches | Number of prefetches made with asynchronous reading from remote filesystem Type: float Unit: count |
RemoteFSSeeks | Total number of seeks for async buffer Type: float Unit: count |
RemoteFSSeeksWithReset | Number of seeks which lead to a new connection Type: float Unit: count |
RemoteFSUnprefetchedBytes | Number of bytes from un prefetched buffer Type: float Unit: digital,B |
RemoteFSUnprefetchedReads | Number of reads from un prefetched buffer Type: float Unit: count |
RemoteFSUnusedPrefetches | Number of prefetches pending at buffer destruction Type: float Unit: count |
RemoteReadThrottlerBytes | Bytes passed through 'max_remote_read_network_bandwidth_for_server'/'max_remote_read_network_bandwidth' throttler. Type: float Unit: digital,B |
RemoteReadThrottlerSleepMicroseconds | Total time a query was sleeping to conform 'max_remote_read_network_bandwidth_for_server'/'max_remote_read_network_bandwidth' throttling. Type: float Unit: time,ms |
RemoteWriteThrottlerBytes | Bytes passed through 'max_remote_write_network_bandwidth_for_server'/'max_remote_write_network_bandwidth' throttler. Type: float Unit: digital,B |
RemoteWriteThrottlerSleepMicroseconds | Total time a query was sleeping to conform 'max_remote_write_network_bandwidth_for_server'/'max_remote_write_network_bandwidth' throttling. Type: float Unit: time,ms |
ReplicaPartialShutdown | How many times Replicated table has to deinitialize its state due to session expiration in ZooKeeper. The state is reinitialized every time when ZooKeeper is available again.Type: float Unit: count |
ReplicatedDataLoss | Number of times a data part that we wanted does not exist on any replica (even on replicas that are offline right now). That data parts are definitely lost. This is normal due to asynchronous replication (if quorum inserts were not enabled), when the replica on which the data part was written was failed and when it became online after fail it does not contain that data part. Type: float Unit: count |
ReplicatedPartChecks | Number of times we had to perform advanced search for a data part on replicas or to clarify the need of an existing data part. Type: float Unit: count |
ReplicatedPartChecksFailed | Number of times the advanced search for a data part on replicas did not give result or when unexpected part has been found and moved away. Type: float Unit: count |
ReplicatedPartFailedFetches | Number of times a data part was failed to download from replica of a ReplicatedMergeTree table. Type: float Unit: count |
ReplicatedPartFetches | Number of times a data part was downloaded from replica of a ReplicatedMergeTree table. Type: float Unit: count |
ReplicatedPartFetchesOfMerged | Number of times we prefer to download already merged part from replica of ReplicatedMergeTree table instead of performing a merge ourself (usually we prefer doing a merge ourself to save network traffic). This happens when we have not all source parts to perform a merge or when the data part is old enough. Type: float Unit: count |
ReplicatedPartMerges | Number of times data parts of ReplicatedMergeTree tables were successfully merged. Type: float Unit: count |
ReplicatedPartMutations | Number of times data parts of ReplicatedMergeTree tables were successfully mutated. Type: float Unit: count |
S3AbortMultipartUpload | Number of S3 API AbortMultipartUpload calls. Type: float Unit: count |
S3CompleteMultipartUpload | Number of S3 API CompleteMultipartUpload calls. Type: float Unit: count |
S3CopyObject | Number of S3 API CopyObject calls. Type: float Unit: count |
S3CreateMultipartUpload | Number of S3 API CreateMultipartUpload calls. Type: float Unit: count |
S3DeleteObjects | Number of S3 API DeleteObject(s) calls. Type: float Unit: count |
S3GetObject | Number of S3 API GetObject calls. Type: float Unit: count |
S3GetObjectAttributes | Number of S3 API GetObjectAttributes calls. Type: float Unit: count |
S3GetRequestThrottlerCount | Number of S3 GET and SELECT requests passed through throttler. Type: float Unit: count |
S3GetRequestThrottlerSleepMicroseconds | Total time a query was sleeping to conform S3 GET and SELECT request throttling. Type: float Unit: time,ms |
S3HeadObject | Number of S3 API HeadObject calls. Type: float Unit: count |
S3ListObjects | Number of S3 API ListObjects calls. Type: float Unit: count |
S3PutObject | Number of S3 API PutObject calls. Type: float Unit: count |
S3PutRequestThrottlerCount | Number of S3 PUT, COPY, POST and LIST requests passed through throttler. Type: float Unit: count |
S3PutRequestThrottlerSleepMicroseconds | Total time a query was sleeping to conform S3 PUT, COPY, POST and LIST request throttling. Type: float Unit: time,ms |
S3ReadBytes | Read bytes (incoming) in GET and HEAD requests to S3 storage. Type: float Unit: digital,B |
S3ReadMicroseconds | Time of GET and HEAD requests to S3 storage. Type: float Unit: time,ms |
S3ReadRequestsCount | Number of GET and HEAD requests to S3 storage. Type: float Unit: count |
S3ReadRequestsErrors | Number of non-throttling errors in GET and HEAD requests to S3 storage. Type: float Unit: count |
S3ReadRequestsRedirects | Number of redirects in GET and HEAD requests to S3 storage. Type: float Unit: count |
S3ReadRequestsThrottling | Number of 429 and 503 errors in GET and HEAD requests to S3 storage. Type: float Unit: count |
S3UploadPart | Number of S3 API UploadPart calls. Type: float Unit: count |
S3UploadPartCopy | Number of S3 API UploadPartCopy calls. Type: float Unit: count |
S3WriteBytes | Write bytes (outgoing) in POST, DELETE, PUT and PATCH requests to S3 storage. Type: float Unit: digital,B |
S3WriteMicroseconds | Time of POST, DELETE, PUT and PATCH requests to S3 storage. Type: float Unit: time,ms |
S3WriteRequestsCount | Number of POST, DELETE, PUT and PATCH requests to S3 storage. Type: float Unit: count |
S3WriteRequestsErrors | Number of non-throttling errors in POST, DELETE, PUT and PATCH requests to S3 storage. Type: float Unit: count |
S3WriteRequestsRedirects | Number of redirects in POST, DELETE, PUT and PATCH requests to S3 storage. Type: float Unit: count |
S3WriteRequestsThrottling | Number of 429 and 503 errors in POST, DELETE, PUT and PATCH requests to S3 storage. Type: float Unit: count |
ScalarSubqueriesCacheMiss | Number of times a read from a scalar sub query was not cached and had to be calculated completely Type: float Unit: count |
ScalarSubqueriesGlobalCacheHit | Number of times a read from a scalar sub query was done using the global cache Type: float Unit: count |
ScalarSubqueriesLocalCacheHit | Number of times a read from a scalar sub query was done using the local cache Type: float Unit: count |
SchemaInferenceCacheEvictions | Number of times a schema from cache was evicted due to overflow Type: float Unit: count |
SchemaInferenceCacheHits | Number of times a schema from cache was used for schema inference Type: float Unit: count |
SchemaInferenceCacheInvalidations | Number of times a schema in cache became invalid due to changes in data Type: float Unit: count |
SchemaInferenceCacheMisses | Number of times a schema is not in cache while schema inference Type: float Unit: count |
Seek | Number of times the 'lseek' function was called. Type: float Unit: count |
SelectQuery | Same as Query, but only for SELECT queries. Type: float Unit: count |
SelectQueryTimeMicroseconds | Total time of SELECT queries. Type: float Unit: time,ms |
SelectedBytes | Number of bytes (uncompressed; for columns as they stored in memory) SELECTed from all tables. Type: float Unit: digital,B |
SelectedMarks | Number of marks (index granules) selected to read from a MergeTree table. Type: float Unit: count |
SelectedParts | Number of data parts selected to read from a MergeTree table. Type: float Unit: count |
SelectedRanges | Number of (non-adjacent) ranges in all data parts selected to read from a MergeTree table. Type: float Unit: count |
SelectedRows | Number of rows SELECTed from all tables. Type: float Unit: count |
ServerStartupMilliseconds | Time elapsed from starting server to listening to sockets in milliseconds Type: float Unit: time,ms |
SleepFunctionCalls | Number of times a sleep function (sleep, sleepEachRow) has been called. Type: float Unit: count |
SleepFunctionMicroseconds | Time spent sleeping due to a sleep function call. Type: float Unit: time,ms |
SlowRead | Number of reads from a file that were slow. This indicate system overload. Thresholds are controlled by read_backoff_* settings. Type: float Unit: count |
SoftPageFaults | The number of soft page faults in query execution threads. Soft page fault usually means a miss in the memory allocator cache which required a new memory mapping from the OS and subsequent allocation of a page of physical memory. Type: float Unit: count |
StorageBufferErrorOnFlush | Number of times a buffer in the 'Buffer' table has not been able to flush due to error writing in the destination table. Type: float Unit: count |
StorageBufferFlush | Number of times a buffer in a 'Buffer' table was flushed. Type: float Unit: count |
StorageBufferLayerLockReadersWaitMilliseconds | Time for waiting for Buffer layer during reading. Type: float Unit: time,ms |
StorageBufferLayerLockWritersWaitMilliseconds | Time for waiting free Buffer layer to write to (can be used to tune Buffer layers). Type: float Unit: time,ms |
StorageBufferPassedAllMinThresholds | Number of times a criteria on min thresholds has been reached to flush a buffer in a 'Buffer' table. Type: float Unit: count |
StorageBufferPassedBytesFlushThreshold | Number of times background-only flush threshold on bytes has been reached to flush a buffer in a 'Buffer' table. This is expert-only metric. If you read this and you are not an expert, stop reading. Type: float Unit: count |
StorageBufferPassedBytesMaxThreshold | Number of times a criteria on max bytes threshold has been reached to flush a buffer in a 'Buffer' table. Type: float Unit: count |
StorageBufferPassedRowsFlushThreshold | Number of times background-only flush threshold on rows has been reached to flush a buffer in a 'Buffer' table. This is expert-only metric. If you read this and you are not an expert, stop reading. Type: float Unit: count |
StorageBufferPassedRowsMaxThreshold | Number of times a criteria on max rows threshold has been reached to flush a buffer in a 'Buffer' table. Type: float Unit: count |
StorageBufferPassedTimeFlushThreshold | Number of times background-only flush threshold on time has been reached to flush a buffer in a 'Buffer' table. This is expert-only metric. If you read this and you are not an expert, stop reading. Type: float Unit: count |
StorageBufferPassedTimeMaxThreshold | Number of times a criteria on max time threshold has been reached to flush a buffer in a 'Buffer' table. Type: float Unit: count |
SuspendSendingQueryToShard | Total count when sending query to shard was suspended when async_query_sending_for_remote is enabled. Type: float Unit: count |
SynchronousRemoteReadWaitMicroseconds | Time spent in waiting for synchronous remote reads. Type: float Unit: time,ms |
SystemTimeMicroseconds | Total time spent in processing (queries and other tasks) threads executing CPU instructions in OS kernel space. This include time CPU pipeline was stalled due to cache misses, branch mispredictions , hyper-threading, etc.Type: float Unit: time,ms |
TableFunctionExecute | Number of table function calls. Type: float Unit: count |
ThreadPoolReaderPageCacheHit | Number of times the read inside ThreadPoolReader was done from page cache. Type: float Unit: count |
ThreadPoolReaderPageCacheHitBytes | Number of bytes read inside ThreadPoolReader when it was done from page cache. Type: float Unit: digital,B |
ThreadPoolReaderPageCacheHitElapsedMicroseconds | Time spent reading data from page cache in ThreadPoolReader. Type: float Unit: time,ms |
ThreadPoolReaderPageCacheMiss | Number of times the read inside ThreadPoolReader was not done from page cache and was hand off to thread pool. Type: float Unit: count |
ThreadPoolReaderPageCacheMissBytes | Number of bytes read inside ThreadPoolReader when read was not done from page cache and was hand off to thread pool. Type: float Unit: digital,B |
ThreadPoolReaderPageCacheMissElapsedMicroseconds | Time spent reading data inside the asynchronous job in ThreadPoolReader - when read was not done from page cache. Type: float Unit: time,ms |
ThreadpoolReaderReadBytes | Bytes read from a threadpool task in asynchronous readingType: float Unit: digital,B |
ThreadpoolReaderSubmit | Bytes read from a threadpool task in asynchronous readingType: float Unit: digital,B |
ThreadpoolReaderTaskMicroseconds | Time spent getting the data in asynchronous reading Type: float Unit: time,ms |
ThrottlerSleepMicroseconds | Total time a query was sleeping to conform all throttling settings. Type: float Unit: time,ms |
UncompressedCacheHits | Number of times a block of data has been found in the uncompressed cache (and decompression was avoided). Type: float Unit: count |
UncompressedCacheMisses | Number of times a block of data has not been found in the uncompressed cache (and required decompression). Type: float Unit: count |
UncompressedCacheWeightLost | Number of bytes evicted from the uncompressed cache. Type: float Unit: digital,B |
UserTimeMicroseconds | Total time spent in processing (queries and other tasks) threads executing CPU instructions in user space. This include time CPU pipeline was stalled due to cache misses, branch mispredictions , hyper-threading, etc.Type: float Unit: time,ms |
VoluntaryContextSwitches | Voluntary context switches Type: float Unit: count |
WaitMarksLoadMicroseconds | Time spent loading marks Type: float Unit: time,ms |
WaitPrefetchTaskMicroseconds | Time spend waiting for prefetched reader Type: float Unit: time,ms |
WriteBufferAIOWrite | Write Buffer AIO Write Type: float Unit: count |
WriteBufferAIOWriteBytes | Write buffer AIO write bytes Type: float Unit: digital,B |
WriteBufferFromFileDescriptorWrite | Number of writes (write /pwrite ) to a file descriptor. Does not include sockets.Type: float Unit: count |
WriteBufferFromFileDescriptorWriteBytes | Number of bytes written to file descriptors. If the file is compressed, this will show compressed data size. Type: float Unit: digital,B |
WriteBufferFromFileDescriptorWriteFailed | Number of times the write (write /pwrite ) to a file descriptor have failed.Type: float Unit: count |
WriteBufferFromS3Bytes | Bytes written to S3. Type: float Unit: digital,B |
WriteBufferFromS3Microseconds | Time spent on writing to S3. Type: float Unit: time,ms |
WriteBufferFromS3RequestsErrors | Number of exceptions while writing to S3. Type: float Unit: count |
ZooKeeperBytesReceived | Number of bytes received over network while communicating with ZooKeeper. Type: float Unit: digital,B |
ZooKeeperBytesSent | Number of bytes send over network while communicating with ZooKeeper. Type: float Unit: digital,B |
ZooKeeperCheck | Number of 'check' requests to ZooKeeper. Usually they don't make sense in isolation, only as part of a complex transaction. Type: float Unit: count |
ZooKeeperClose | Number of times connection with ZooKeeper has been closed voluntary. Type: float Unit: count |
ZooKeeperCreate | Number of 'create' requests to ZooKeeper. Type: float Unit: count |
ZooKeeperExists | Number of 'exists' requests to ZooKeeper. Type: float Unit: count |
ZooKeeperGet | Number of 'get' requests to ZooKeeper. Type: float Unit: count |
ZooKeeperHardwareExceptions | Number of exceptions while working with ZooKeeper related to network (connection loss or similar). Type: float Unit: count |
ZooKeeperInit | Number of times connection with ZooKeeper has been established. Type: float Unit: count |
ZooKeeperList | Number of 'list' (getChildren) requests to ZooKeeper. Type: float Unit: count |
ZooKeeperMulti | Number of 'multi' requests to ZooKeeper (compound transactions). Type: float Unit: count |
ZooKeeperOtherExceptions | Number of exceptions while working with ZooKeeper other than ZooKeeperUserExceptions and ZooKeeperHardwareExceptions. Type: float Unit: count |
ZooKeeperRemove | Number of 'remove' requests to ZooKeeper. Type: float Unit: count |
ZooKeeperSet | Number of 'set' requests to ZooKeeper. Type: float Unit: count |
ZooKeeperSync | Number of 'sync' requests to ZooKeeper. These requests are rarely needed or usable. Type: float Unit: count |
ZooKeeperTransactions | Number of ZooKeeper operations, which include both read and write operations as well as multi-transactions. Type: float Unit: count |
ZooKeeperUserExceptions | Number of exceptions while working with ZooKeeper related to the data (no node, bad version or similar). Type: float Unit: count |
ZooKeeperWaitMicroseconds | Number of microseconds spent waiting for responses from ZooKeeper after creating a request, summed across all the requesting threads. Type: float Unit: time,ms |
ZooKeeperWatchResponse | Number of times watch notification has been received from ZooKeeper. Type: float Unit: count |
ClickHouseStatusInfo
¶
- Tags
Tag | Description |
---|---|
host | Host name |
instance | Instance endpoint |
- Metrics
Metric | Description |
---|---|
DictionaryStatus | Dictionary Status. Type: float Unit: - |
collector
¶
- Tags
Tag | Description |
---|---|
instance | Server addr of the instance |
job | Server name of the instance |
- Metrics
Metric | Description |
---|---|
up | Type: int Unit: - |