Ceph
Configuration¶
Prerequisites¶
- Ceph exposes Prometheus monitoring interface
The following components and port relationships:
| Component | Component Metrics Port |
|---|---|
| mgr-prometheus | 9283 |
| ceph-exporter | 9926 |
| nvmeof | 10008 |
| nfs | 9587 |
Obtain the corresponding monitoring configuration information via the following method, where 8765 is the management port:
Format http://192.168.0.37:8765/sd/prometheus/sd-config?service=<component name> to get the Prometheus configuration file
For example: http://192.168.0.37:8765/sd/prometheus/sd-config?service=mgr-prometheus gets the mgr-prometheus configuration file
[root@ceph001 ~]# curl http://192.168.0.37:8765/sd/prometheus/sd-config?service=mgr-prometheus
[{"targets": ["ceph001:9283"], "labels": {}}]
http://ceph001:9283/metrics. Since nodes may drift, it is necessary to configure multiple targets when collecting.
Enable Prom Collector in DataKit¶
Adjust configurations according to the actual number of nodes:
ceph-mgr.conf
Go to DataKit installation directory under conf.d/prom, copy prom.conf.sample as ceph-mgr.conf.
cp prom.conf.sample ceph-mgr.conf
Modify the content of ceph-mgr.conf as follows:
# {"version": "1.74.2", "desc": "do NOT edit this line"}
[[inputs.prom]]
## Exporter URLs.
urls=["http://ceph001:9283/metrics","http://ceph002:9283/metrics","http://ceph003:9283/metrics"]
## Collector alias.
source = "ceph-mgr"
## Customize tags.
[inputs.prom.tags]
cluster="ceph-cluster"
...
ceph-exporter.conf
Go to DataKit installation directory under conf.d/prom, copy prom.conf.sample as ceph-exporter.conf.
cp prom.conf.sample ceph-exporter.conf
Modify the content of ceph-exporter.conf as follows:
# {"version": "1.74.2", "desc": "do NOT edit this line"}
[[inputs.prom]]
## Exporter URLs.
urls=["http://ceph001:9926/metrics","http://ceph002:9926/metrics","http://ceph003:9926/metrics"]
## Collector alias.
source = "ceph-exporter"
## Customize tags.
[inputs.prom.tags]
cluster="ceph-cluster"
...
ceph-nfs.conf
Go to DataKit installation directory under conf.d/prom, copy prom.conf.sample as ceph-nfs.conf.
cp prom.conf.sample ceph-nfs.conf
Modify the content of ceph-nfs.conf as follows:
# {"version": "1.74.2", "desc": "do NOT edit this line"}
[[inputs.prom]]
## Exporter URLs.
urls=["http://ceph001:9587/metrics"]
## Collector alias.
source = "ceph-nfs"
## Customize tags.
[inputs.prom.tags]
cluster="ceph-cluster"
...
ceph-nvmeof.conf
Go to DataKit installation directory under conf.d/prom, copy prom.conf.sample as ceph-nvmeof.conf.
cp prom.conf.sample ceph-nvmeof.conf
Modify the content of ceph-nvmeof.conf as follows:
# {"version": "1.74.2", "desc": "do NOT edit this line"}
[[inputs.prom]]
## Exporter URLs.
urls=["http://ceph001:10008"]
## Collector alias.
source = "ceph-nvmeof"
## Customize tags.
[inputs.prom.tags]
cluster="ceph-cluster"
...
Restart DataKit¶
Metrics¶
| Metric Name | Description | Unit |
|---|---|---|
| AsyncMessenger_Worker_msgr_connection_idle_timeouts | Number of connections closed in the asynchronous messenger worker thread due to connection idle timeout | Times |
| AsyncMessenger_Worker_msgr_connection_ready_timeouts | Number of connections closed in the asynchronous messenger worker thread due to connection ready timeout | Times |
| bluefs_alloc_db_max_lat | Maximum delay for BlueFS database allocation | Milliseconds |
| bluefs_alloc_slow_fallback | Number of fallbacks when allocating on the slow path in BlueFS | Times |
| bluefs_alloc_slow_max_lat | Maximum delay for allocation on the slow path in BlueFS | Milliseconds |
| bluefs_alloc_slow_size_fallback | Number of fallbacks when allocating on the slow path in BlueFS due to insufficient size | Times |
| bluefs_alloc_unit_db | Size of the BlueFS database allocation unit | Bytes |
| bluefs_alloc_unit_slow | Size of the BlueFS slow path allocation unit | Bytes |
| bluefs_alloc_unit_wal | Size of the BlueFS Write-Ahead Log (WAL) allocation unit | Bytes |
| bluefs_alloc_wal_max_lat | Maximum delay for allocation in the BlueFS Write-Ahead Log (WAL) | Milliseconds |
| bluefs_bytes_written_slow | Number of bytes written to the BlueFS slow path | Bytes |
| bluefs_bytes_written_sst | Number of bytes written to the BlueFS SST file | Bytes |
| bluefs_bytes_written_wal | Number of bytes written to the BlueFS Write-Ahead Log (WAL) | Bytes |
| bluefs_compact_lat_count | Count of delays for BlueFS compaction operations | Times |
| bluefs_compact_lat_sum | Sum of delays for BlueFS compaction operations | Milliseconds |
| bluefs_compact_lock_lat_count | Count of delays for BlueFS compaction lock operations | Times |
| bluefs_compact_lock_lat_sum | Sum of delays for BlueFS compaction lock operations | Milliseconds |
| bluefs_db_alloc_lat_count | Count of delays for BlueFS database allocation operations | Times |
| bluefs_db_alloc_lat_sum | Sum of delays for BlueFS database allocation operations | Milliseconds |
| bluefs_db_total_bytes | Total number of bytes in the BlueFS database | Bytes |
| bluefs_db_used_bytes | Number of used bytes in the BlueFS database | Bytes |
| bluefs_log_bytes | Number of bytes in the BlueFS log | Bytes |
| bluefs_logged_bytes | Number of bytes logged into BlueFS | Bytes |
| bluefs_max_bytes_db | Maximum number of bytes in the BlueFS database | Bytes |
| bluefs_max_bytes_slow | Maximum number of bytes in the BlueFS slow path | Bytes |
| bluefs_max_bytes_wal | Maximum number of bytes in the BlueFS Write-Ahead Log (WAL) | Bytes |
| bluefs_num_files | Number of files in BlueFS | Files |
| bluefs_read_bytes | Number of bytes read from BlueFS | Bytes |
| bluefs_read_count | Number of reads from BlueFS | Times |
| bluefs_read_disk_bytes | Number of bytes read from the BlueFS disk | Bytes |
| bluefs_read_disk_bytes_db | Number of bytes read from the BlueFS database disk | Bytes |
| bluefs_read_disk_bytes_slow | Number of bytes read from the BlueFS slow path disk | Bytes |
| bluefs_read_disk_bytes_wal | Number of bytes read from the BlueFS Write-Ahead Log (WAL) disk | Bytes |
| bluefs_read_disk_count | Number of reads from the BlueFS disk | Times |
| bluefs_read_prefetch_bytes | Number of bytes prefetched read from BlueFS | Bytes |
| bluefs_read_prefetch_count | Number of prefetch reads from BlueFS | Times |
| bluefs_read_random_buffer_bytes | Number of bytes read randomly from the BlueFS buffer | Bytes |
| bluefs_read_random_buffer_count | Number of random buffer reads from BlueFS | Times |
| bluefs_read_random_bytes | Number of bytes read randomly from BlueFS | Bytes |
| bluefs_read_random_count | Number of random reads from BlueFS | Times |
| bluefs_read_random_disk_bytes | Number of bytes read randomly from the BlueFS disk | Bytes |
| bluefs_read_random_disk_bytes_db | Number of bytes read randomly from the BlueFS database disk | Bytes |
| bluefs_read_random_disk_bytes_slow | Number of bytes read randomly from the BlueFS slow path disk | Bytes |
| bluefs_read_random_disk_bytes_wal | Number of bytes read randomly from the BlueFS Write-Ahead Log (WAL) disk | Bytes |
| bluefs_read_random_disk_count | Number of random disk reads from BlueFS | Times |
| bluefs_slow_alloc_lat_count | Count of delays for allocation operations on the BlueFS slow path | Times |
| bluefs_slow_alloc_lat_sum | Sum of delays for allocation operations on the BlueFS slow path | Milliseconds |
| bluefs_slow_total_bytes | Total number of bytes in the BlueFS slow path | Bytes |
| bluefs_slow_used_bytes | Number of used bytes in the BlueFS slow path | Bytes |
| bluefs_wal_alloc_lat_count | Count of delays for allocation operations in the BlueFS Write-Ahead Log (WAL) | Times |
| bluefs_wal_alloc_lat_sum | Sum of delays for allocation operations in the BlueFS Write-Ahead Log (WAL) | Milliseconds |
| bluefs_wal_total_bytes | Total number of bytes in the BlueFS Write-Ahead Log (WAL) | Bytes |
| bluefs_wal_used_bytes | Number of used bytes in the BlueFS Write-Ahead Log (WAL) | Bytes |
| bluefs_write_bytes | Number of bytes written to BlueFS | Bytes |
| bluestore_alloc_unit | Size of the BlueStore allocation unit | Bytes |
| bluestore_allocated | Number of allocated bytes in BlueStore | Bytes |
| bluestore_allocator_lat_count | Count of delays for BlueStore allocator | Times |
| bluestore_allocator_lat_sum | Sum of delays for BlueStore allocator | Milliseconds |
| bluestore_clist_lat_count | Count of delays for BlueStore CList operations | Times |
| bluestore_clist_lat_sum | Sum of delays for BlueStore CList operations | Milliseconds |
| bluestore_compress_lat_count | Count of delays for BlueStore compression operations | Times |
| bluestore_compress_lat_sum | Sum of delays for BlueStore compression operations | Milliseconds |
| bluestore_compressed | Number of compressed bytes in BlueStore | Bytes |
| bluestore_compressed_allocated | Number of allocated compressed bytes in BlueStore | Bytes |
| bluestore_compressed_original | Number of original uncompressed bytes in BlueStore | Bytes |
| bluestore_csum_lat_count | Count of delays for BlueStore checksum operations | Times |
| bluestore_csum_lat_sum | Sum of delays for BlueStore checksum operations | Milliseconds |
| bluestore_decompress_lat_count | Count of delays for BlueStore decompression operations | Times |
| bluestore_decompress_lat_sum | Sum of delays for BlueStore decompression operations | Milliseconds |
| bluestore_fragmentation_micros | Time spent on fragmentation in BlueStore | Microseconds |
| bluestore_kv_commit_lat_count | Count of delays for BlueStore KV commit operations | Times |
| bluestore_kv_commit_lat_sum | Sum of delays for BlueStore KV commit operations | Milliseconds |
| bluestore_kv_final_lat_count | Count of delays for BlueStore KV final operations | Times |
| bluestore_kv_final_lat_sum | Sum of delays for BlueStore KV final operations | Milliseconds |
| bluestore_kv_flush_lat_count | Count of delays for BlueStore KV flush operations | Times |
| bluestore_kv_flush_lat_sum | Sum of delays for BlueStore KV flush operations | Milliseconds |
| bluestore_kv_sync_lat_count | Count of delays for BlueStore KV sync operations | Times |
| bluestore_kv_sync_lat_sum | Sum of delays for BlueStore KV sync operations | Milliseconds |
| bluestore_omap_get_keys_lat_count | Count of delays for BlueStore OMAP key-getting operations | Times |
| bluestore_omap_get_keys_lat_sum | Sum of delays for BlueStore OMAP key-getting operations | Milliseconds |
| bluestore_omap_get_values_lat_count | Count of delays for BlueStore OMAP value-getting operations | Times |
| bluestore_omap_get_values_lat_sum | Sum of delays for BlueStore OMAP value-getting operations | Milliseconds |
| bluestore_omap_lower_bound_lat_count | Count of delays for BlueStore OMAP lower-bound operations | Times |
| bluestore_omap_lower_bound_lat_sum | Sum of delays for BlueStore OMAP lower-bound operations | Milliseconds |
| bluestore_omap_next_lat_count | Count of delays for BlueStore OMAP next operations | Times |
| bluestore_omap_next_lat_sum | Sum of delays for BlueStore OMAP next operations | Milliseconds |
| bluestore_omap_seek_to_first_lat_count | Count of delays for BlueStore OMAP first-seek operations | Times |
| bluestore_omap_seek_to_first_lat_sum | Sum of delays for BlueStore OMAP first-seek operations | Milliseconds |
| bluestore_omap_upper_bound_lat_count | Count of delays for BlueStore OMAP upper-bound operations | Times |
| bluestore_omap_upper_bound_lat_sum | Sum of delays for BlueStore OMAP upper-bound operations | Milliseconds |
| bluestore_onode_hits | Number of hits in the BlueStore ONode cache | Times |
| bluestore_onode_misses | Number of misses in the BlueStore ONode cache | Times |
| bluestore_pricache:data_committed_bytes | Number of committed data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_pri0_bytes | Number of priority level 0 data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_pri10_bytes | Number of priority level 10 data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_pri11_bytes | Number of priority level 11 data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_pri1_bytes | Number of priority level 1 data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_pri2_bytes | Number of priority level 2 data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_pri3_bytes | Number of priority level 3 data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_pri4_bytes | Number of priority level 4 data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_pri5_bytes | Number of priority level 5 data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_pri6_bytes | Number of priority level 6 data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_pri7_bytes | Number of priority level 7 data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_pri8_bytes | Number of priority level 8 data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_pri9_bytes | Number of priority level 9 data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:data_reserved_bytes | Number of reserved data bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_committed_bytes | Number of committed KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_committed_bytes | Number of committed ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_pri0_bytes | Number of priority level 0 ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_pri10_bytes | Number of priority level 10 ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_pri11_bytes | Number of priority level 11 ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_pri1_bytes | Number of priority level 1 ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_pri2_bytes | Number of priority level 2 ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_pri3_bytes | Number of priority level 3 ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_pri4_bytes | Number of priority level 4 ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_pri5_bytes | Number of priority level 5 ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_pri6_bytes | Number of priority level 6 ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_pri7_bytes | Number of priority level 7 ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_pri8_bytes | Number of priority level 8 ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_pri9_bytes | Number of priority level 9 ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_onode_reserved_bytes | Number of reserved ONode KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_pri0_bytes | Number of priority level 0 KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_pri10_bytes | Number of priority level 10 KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_pri11_bytes | Number of priority level 11 KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_pri1_bytes | Number of priority level 1 KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_pri2_bytes | Number of priority level 2 KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_pri3_bytes | Number of priority level 3 KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_pri4_bytes | Number of priority level 4 KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_pri5_bytes | Number of priority level 5 KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_pri6_bytes | Number of priority level 6 KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_pri7_bytes | Number of priority level 7 KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_pri8_bytes | Number of priority level 8 KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_pri9_bytes | Number of priority level 9 KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:kv_reserved_bytes | Number of reserved KV bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_committed_bytes | Number of committed metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_pri0_bytes | Number of priority level 0 metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_pri10_bytes | Number of priority level 10 metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_pri11_bytes | Number of priority level 11 metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_pri1_bytes | Number of priority level 1 metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_pri2_bytes | Number of priority level 2 metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_pri3_bytes | Number of priority level 3 metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_pri4_bytes | Number of priority level 4 metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_pri5_bytes | Number of priority level 5 metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_pri6_bytes | Number of priority level 6 metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_pri7_bytes | Number of priority level 7 metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_pri8_bytes | Number of priority level 8 metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_pri9_bytes | Number of priority level 9 metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache:meta_reserved_bytes | Number of reserved metadata bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache_cache_bytes | Total number of bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache_heap_bytes | Heap memory bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache_mapped_bytes | Mapped bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache_target_bytes | Target bytes in the BlueStore priority cache | Bytes |
| bluestore_pricache_unmapped_bytes | Unmapped bytes in the BlueStore priority cache | Bytes |
| bluestore_read_lat_count | Count of delays for BlueStore read operations | Times |
| bluestore_read_lat_sum | Sum of delays for BlueStore read operations | Milliseconds |
| bluestore_read_onode_meta_lat_count | Count of delays for reading ONode metadata in BlueStore | Times |
| bluestore_read_onode_meta_lat_sum | Sum of delays for reading ONode metadata in BlueStore | Milliseconds |
| bluestore_read_wait_aio_lat_count | Count of delays for waiting asynchronous I/O in BlueStore read operations | Times |
| bluestore_read_wait_aio_lat_sum | Sum of delays for waiting asynchronous I/O in BlueStore read operations | Milliseconds |
| bluestore_reads_with_retries | Number of retries needed in BlueStore read operations | Times |
| bluestore_remove_lat_count | Count of delays for BlueStore remove operations | Times |
| bluestore_remove_lat_sum | Sum of delays for BlueStore remove operations | Milliseconds |
| bluestore_slow_aio_wait_count | Number of times waiting for asynchronous I/O on the slow path in BlueStore | Times |
| bluestore_slow_committed_kv_count | Number of committed KV operations on the slow path in BlueStore | Times |
| bluestore_slow_read_onode_meta_count | Number of ONode metadata reads on the slow path in BlueStore | Times |
| bluestore_slow_read_wait_aio_count | Number of waits for asynchronous I/O reads on the slow path in BlueStore | Times |
| bluestore_state_aio_wait_lat_count | Count of delays for AIO wait operations in the BlueStore state | Times |
| bluestore_state_aio_wait_lat_sum | Sum of delays for AIO wait operations in the BlueStore state | Milliseconds |
| bluestore_state_deferred_aio_wait_lat_count | Count of delays for deferred AIO wait operations in the BlueStore state | Times |
| bluestore_state_deferred_aio_wait_lat_sum | Sum of delays for deferred AIO wait operations in the BlueStore state | Milliseconds |
| bluestore_state_deferred_cleanup_lat_count | Count of delays for deferred cleanup operations in the BlueStore state | Times |
| bluestore_state_deferred_cleanup_lat_sum | Sum of delays for deferred cleanup operations in the BlueStore state | Milliseconds |
| bluestore_state_deferred_queued_lat_count | Count of delays for deferred queued operations in the BlueStore state | Times |
| bluestore_state_deferred_queued_lat_sum | Sum of delays for deferred queued operations in the BlueStore state | Milliseconds |
| bluestore_state_done_lat_count | Count of delays for done operations in the BlueStore state | Times |
| bluestore_state_done_lat_sum | Sum of delays for done operations in the BlueStore state | Milliseconds |
| bluestore_state_finishing_lat_count | Count of delays for finishing operations in the BlueStore state | Times |
| bluestore_state_finishing_lat_sum | Sum of delays for finishing operations in the BlueStore state | Milliseconds |
| bluestore_state_io_done_lat_count | Count of delays for IO done operations in the BlueStore state | Times |
| bluestore_state_io_done_lat_sum | Sum of delays for IO done operations in the BlueStore state | Milliseconds |
| bluestore_state_kv_commiting_lat_count | Count of delays for KV committing operations in the BlueStore state | Times |
| bluestore_state_kv_commiting_lat_sum | Sum of delays for KV committing operations in the BlueStore state | Milliseconds |
| bluestore_state_kv_done_lat_count | Count of delays for KV done operations in the BlueStore state | Times |
| bluestore_state_kv_done_lat_sum | Sum of delays for KV done operations in the BlueStore state | Milliseconds |
| bluestore_state_kv_queued_lat_count | Count of delays for KV queued operations in the BlueStore state | Times |
| bluestore_state_kv_queued_lat_sum | Sum of delays for KV queued operations in the BlueStore state | Milliseconds |
| bluestore_state_prepare_lat_count | Count of delays for prepare operations in the BlueStore state | Times |
| bluestore_state_prepare_lat_sum | Sum of delays for prepare operations in the BlueStore state | Milliseconds |
| bluestore_stored | Number of stored bytes in BlueStore | Bytes |
| bluestore_truncate_lat_count | Count of delays for truncate operations in BlueStore | Times |
| bluestore_truncate_lat_sum | Sum of delays for truncate operations in BlueStore | Milliseconds |
| bluestore_txc_commit_lat_count | Count of delays for transaction commit operations in BlueStore | Times |
| bluestore_txc_commit_lat_sum | Sum of delays for transaction commit operations in BlueStore | Milliseconds |
| bluestore_txc_submit_lat_count | Count of delays for transaction submit operations in BlueStore | Times |
| bluestore_txc_submit_lat_sum | Sum of delays for transaction submit operations in BlueStore | Milliseconds |
| bluestore_txc_throttle_lat_count | Count of delays for transaction throttle operations in BlueStore | Times |
| bluestore_txc_throttle_lat_sum | Sum of delays for transaction throttle operations in BlueStore | Milliseconds |
| cluster_by_class_total_bytes | Total bytes by class in the cluster | Bytes |
| cluster_by_class_total_used_bytes | Total used bytes by class in the cluster | Bytes |
| cluster_by_class_total_used_raw_bytes | Total raw used bytes by class in the cluster | Bytes |
| cluster_osd_blocklist_count | Number of blacklisted OSDs in the cluster | Items |
| cluster_total_bytes | Total bytes in the cluster | Bytes |
| cluster_total_used_bytes | Total used bytes in the cluster | Bytes |
| cluster_total_used_raw_bytes | Total raw used bytes in the cluster | Bytes |
| daemon_health_metrics | Daemon health metrics | No unit |
| disk_occupation | Disk occupation rate | Percentage |
| disk_occupation_human | Disk occupation rate (human-readable format) | Percentage |
| exporter_scrape_time | Exporter scrape time | Seconds |
| health_detail | Health details | No unit |
| health_status | Health status | No unit |
| healthcheck_slow_ops | Number of slow health check operations | Times |
| mgr_metadata | Manager metadata | No unit |
| mgr_module_can_run | Whether manager modules can run | Boolean |
| mgr_module_status | Manager module status | No unit |
| mgr_status | Manager status | No unit |
| mon_election_call | Monitor election call count | Times |
| mon_election_lose | Monitor election loss count | Times |
| mon_election_win | Monitor election win count | Times |
| mon_metadata | Monitor metadata | No unit |
| mon_num_elections | Monitor election count | Times |
| mon_num_sessions | Monitor session count | Sessions |
| mon_quorum_status | Monitor quorum status | No unit |
| mon_session_add | Monitor session add count | Times |
| mon_session_rm | Monitor session remove count | Times |
| mon_session_trim | Monitor session trim count | Times |
| num_objects_degraded | Number of degraded objects | Objects |
| num_objects_misplaced | Number of misplaced objects | Objects |
| num_objects_unfound | Number of unfound objects | Objects |
| objecter_op_active | Current active object operation count | Operations |
| objecter_op_r | Number of object read operations | Times |
| objecter_op_rmw | Number of object read-modify-write operations | Times |
| objecter_op_w | Number of object write operations | Times |
| osd_apply_latency_ms | OSD application latency | Milliseconds |
| osd_commit_latency_ms | OSD commit latency | Milliseconds |
| osd_flag_nobackfill | OSD no-backfill flag | Boolean |
| osd_flag_nodeep_scrub | OSD no-deep-scrub flag | Boolean |
| osd_flag_nodown | OSD no-down flag | Boolean |
| osd_flag_noin | OSD no-in flag | Boolean |
| osd_flag_noout | OSD no-out flag | Boolean |
| osd_flag_norebalance | OSD no-rebalance flag | Boolean |
| osd_flag_norecover | OSD no-recover flag | Boolean |
| osd_flag_noscrub | OSD no-scrub flag | Boolean |
| osd_flag_noup | OSD no-up flag | Boolean |
| osd_in | Whether OSD is online | Boolean |
| osd_metadata | OSD metadata | No unit |
| osd_numpg | Number of PGs on the OSD | PGs |
| osd_numpg_removing | Number of PGs being removed from the OSD | PGs |
| osd_op | Number of OSD operations | Times |
| osd_op_before_queue_op_lat_count | Count of delays for OSD operations before queuing | Times |
| osd_op_before_queue_op_lat_sum | Sum of delays for OSD operations before queuing | Milliseconds |
| osd_op_delayed_degraded | Number of delayed OSD operations due to degradation | Times |
| osd_op_delayed_unreadable | Number of delayed OSD operations due to unreadability | Times |
| osd_op_in_bytes | Number of input bytes for OSD operations | Bytes |
| osd_op_latency_count | Count of delays for OSD operations | Times |
| osd_op_latency_sum | Sum of delays for OSD operations | Milliseconds |
| osd_op_out_bytes | Number of output bytes for OSD operations | Bytes |
| osd_op_prepare_latency_count | Count of delays during preparation phase of OSD operations | Times |
| osd_op_prepare_latency_sum | Sum of delays during preparation phase of OSD operations | Milliseconds |
| osd_op_process_latency_count | Count of delays during processing phase of OSD operations | Times |
| osd_op_process_latency_sum | Sum of delays during processing phase of OSD operations | Milliseconds |
| osd_op_r | Number of OSD read operations | Times |
| osd_op_r_latency_count | Count of delays for OSD read operations | Times |
| osd_op_r_latency_sum | Sum of delays for OSD read operations | Milliseconds |
| osd_op_r_out_bytes | Number of output bytes for OSD read operations | Bytes |
| osd_op_r_prepare_latency_count | Count of delays during preparation phase of OSD read operations | Times |
| osd_op_r_prepare_latency_sum | Sum of delays during preparation phase of OSD read operations | Milliseconds |
| osd_op_r_process_latency_count | Count of delays during processing phase of OSD read operations | Times |
| osd_op_r_process_latency_sum | Sum of delays during processing phase of OSD read operations | Milliseconds |
| osd_op_rw | Number of OSD read-write operations | Times |
| osd_op_rw_in_bytes | Number of input bytes for OSD read-write operations | Bytes |
| osd_op_rw_latency_count | Count of delays for OSD read-write operations | Times |
| osd_op_rw_latency_sum | Sum of delays for OSD read-write operations | Milliseconds |
| osd_op_rw_out_bytes | Number of output bytes for OSD read-write operations | Bytes |
| osd_op_rw_prepare_latency_count | Count of delays during preparation phase of OSD read-write operations | Times |
| osd_op_rw_prepare_latency_sum | Sum of delays during preparation phase of OSD read-write operations | Milliseconds |
| osd_op_rw_process_latency_count | Count of delays during processing phase of OSD read-write operations | Times |
| osd_op_rw_process_latency_sum | Sum of delays during processing phase of OSD read-write operations | Milliseconds |
| osd_op_w | Number of OSD write operations | Times |
| osd_op_w_in_bytes | Number of input bytes for OSD write operations | Bytes |
| osd_op_w_latency_count | Count of delays for OSD write operations | Times |
| osd_op_w_latency_sum | Sum of delays for OSD write operations | Milliseconds |
| osd_op_w_prepare_latency_count | Count of delays during preparation phase of OSD write operations | Times |
| osd_op_w_prepare_latency_sum | Sum of delays during preparation phase of OSD write operations | Milliseconds |
| osd_op_w_process_latency_count | Count of delays during processing phase of OSD write operations | Times |
| osd_op_w_process_latency_sum | Sum of delays during processing phase of OSD write operations | Milliseconds |
| osd_op_wip | Number of ongoing OSD operations | Operations |
| osd_recovery_bytes | Number of recovered bytes in OSD | Bytes |
| osd_recovery_ops | Number of recovery operations in OSD | Times |
| osd_stat_bytes | Number of statistical bytes in OSD | Bytes |
| osd_stat_bytes_used | Number of used statistical bytes in OSD | Bytes |
| osd_up | Whether OSD is online | Boolean |
| osd_weight | OSD weight | No unit |
| paxos_accept_timeout | Number of Paxos accept timeouts | Times |
| paxos_begin | Number of Paxos begins | Times |
| paxos_begin_bytes_count | Number of bytes at Paxos begin | Bytes |
| paxos_begin_bytes_sum | Sum of bytes at Paxos begin | Bytes |
| paxos_begin_keys_count | Number of keys at Paxos begin | Keys |
| paxos_begin_keys_sum | Sum of keys at Paxos begin | Keys |
| paxos_begin_latency_count | Count of delays at Paxos begin | Times |
| paxos_begin_latency_sum | Sum of delays at Paxos begin | Milliseconds |
| paxos_collect | Number of Paxos collects | Times |
| paxos_collect_bytes_count | Number of bytes at Paxos collect | Bytes |
| paxos_collect_bytes_sum | Sum of bytes at Paxos collect | Bytes |
| paxos_collect_keys_count | Number of keys at Paxos collect | Keys |
| paxos_collect_keys_sum | Sum of keys at Paxos collect | Keys |
| paxos_collect_latency_count | Count of delays at Paxos collect | Times |
| paxos_collect_latency_sum | Sum of delays at Paxos collect | Milliseconds |
| paxos_collect_timeout | Number of Paxos collect timeouts | Times |
| paxos_collect_uncommitted | Number of uncommitted Paxos collects | Times |
| paxos_commit | Number of Paxos commits | Times |
| paxos_commit_bytes_count | Number of bytes at Paxos commit | Bytes |
| paxos_commit_bytes_sum | Sum of bytes at Paxos commit | Bytes |
| paxos_commit_keys_count | Number of keys at Paxos commit | Keys |
| paxos_commit_keys_sum | Sum of keys at Paxos commit | Keys |
| paxos_commit_latency_count | Count of delays at Paxos commit | Times |
| paxos_commit_latency_sum | Sum of delays at Paxos commit | Milliseconds |
| paxos_lease_ack_timeout | Number of Paxos lease acknowledgment timeouts | Times |
| paxos_lease_timeout | Number of Paxos lease timeouts | Times |
| paxos_new_pn | Number of new Paxos proposal numbers | Proposals |
| paxos_new_pn_latency_count | Count of delays for new Paxos proposal numbers | Times |
| paxos_new_pn_latency_sum | Sum of delays for new``` | |
| Paxos proposal numbers | Milliseconds | |
| paxos_refresh | Number of Paxos refreshes | Times |
| paxos_refresh_latency_count | Count of delays for Paxos refreshes | Times |
| paxos_refresh_latency_sum | Sum of delays for Paxos refreshes | Milliseconds |
| paxos_restart | Number of Paxos restarts | Times |
| paxos_share_state | Number of Paxos shared states | Times |
| paxos_share_state_bytes_count | Number of bytes at Paxos shared state | Bytes |
| paxos_share_state_bytes_sum | Sum of bytes at Paxos shared state | Bytes |
| paxos_share_state_keys_count | Number of keys at Paxos shared state | Keys |
| paxos_share_state_keys_sum | Sum of keys at Paxos shared state | Keys |
| paxos_start_leader | Number of Paxos started leaders | Times |
| paxos_start_peon | Number of Paxos started peons | Times |
| paxos_store_state | Number of Paxos stored states | Times |
| paxos_store_state_bytes_count | Number of bytes at Paxos stored state | Bytes |
| paxos_store_state_bytes_sum | Sum of bytes at Paxos stored state | Bytes |
| paxos_store_state_keys_count | Number of keys at Paxos stored state | Keys |
| paxos_store_state_keys_sum | Sum of keys at Paxos stored state | Keys |
| paxos_store_state_latency_count | Count of delays for Paxos stored state | Times |
| paxos_store_state_latency_sum | Sum of delays for Paxos stored state | Milliseconds |
| pg_activating | Number of activating PGs | PGs |
| pg_active | Number of active PGs | PGs |
| pg_backfill_toofull | Number of PGs where backfill failed (target OSD full) | PGs |
| pg_backfill_unfound | Number of PGs where backfill failed (object not found) | PGs |
| pg_backfill_wait | Number of PGs waiting for backfill | PGs |
| pg_backfilling | Number of PGs currently backfilling | PGs |
| pg_clean | Number of clean PGs | PGs |
| pg_creating | Number of PGs being created | PGs |
| pg_deep | Number of deep scrubbing PGs | PGs |
| pg_degraded | Number of degraded PGs | PGs |
| pg_down | Number of offline PGs | PGs |
| pg_failed_repair | Number of PGs where repair failed | PGs |
| pg_forced_backfill | Number of forced backfill PGs | PGs |
| pg_forced_recovery | Number of forced recovery PGs | PGs |
| pg_incomplete | Number of incomplete PGs | PGs |
| pg_inconsistent | Number of inconsistent PGs | PGs |
| pg_laggy | Number of lagging PGs | PGs |
| pg_peered | Number of peered PGs | PGs |
| pg_peering | Number of PGs currently peering | PGs |
| pg_premerge | Number of PGs pre-merging | PGs |
| pg_recovering | Number of PGs recovering | PGs |
| pg_recovery_toofull | Number of PGs where recovery failed (target OSD full) | PGs |
| pg_recovery_unfound | Number of PGs where recovery failed (object not found) | PGs |
| pg_recovery_wait | Number of PGs waiting for recovery | PGs |
| pg_remapped | Number of remapped PGs | PGs |
| pg_repair | Number of PGs repairing | PGs |
| pg_scrubbing | Number of scrubbing PGs | PGs |
| pg_snaptrim | Number of PGs trimming snapshots | PGs |
| pg_snaptrim_error | Number of PGs with snapshot trimming errors | PGs |
| pg_snaptrim_wait | Number of PGs waiting for snapshot trimming | PGs |
| pg_stale | Number of stale PGs | PGs |
| pg_total | Total number of PGs | PGs |
| pg_undersized | Number of undersized PGs | PGs |
| pg_unknown | Number of unknown-state PGs | PGs |
| pg_wait | Number of waiting PGs | PGs |
| pool_avail_raw | Pool available raw bytes | Bytes |
| pool_bytes_used | Pool used bytes | Bytes |
| pool_compress_bytes_used | Pool compressed used bytes | Bytes |
| pool_compress_under_bytes | Pool compression threshold bytes | Bytes |
| pool_dirty | Number of dirty objects in the pool | Objects |
| pool_max_avail | Pool maximum available bytes | Bytes |
| pool_metadata | Pool metadata | No unit |
| pool_num_bytes_recovered | Pool recovered bytes | Bytes |
| pool_num_objects_recovered | Pool recovered objects | Objects |
| pool_objects | Number of objects in the pool | Objects |
| pool_objects_repaired | Number of repaired objects in the pool | Objects |
| pool_percent_used | Pool usage percentage | Percentage |
| pool_quota_bytes | Pool quota bytes | Bytes |
| pool_quota_objects | Pool quota objects | Objects |
| pool_rd | Number of pool read operations | Times |
| pool_rd_bytes | Number of bytes read from the pool | Bytes |
| pool_recovering_bytes_per_sec | Pool recovery bytes per second | Bytes/second |
| pool_recovering_keys_per_sec | Pool recovery keys per second | Keys/second |
| pool_recovering_objects_per_sec | Pool recovery objects per second | Objects/second |
| pool_stored | Pool stored bytes | Bytes |
| pool_stored_raw | Pool stored raw bytes | Bytes |
| pool_wr | Number of pool write operations | Times |
| pool_wr_bytes | Number of bytes written to the pool | Bytes |
| prioritycache:full_committed_bytes | Number of fully committed bytes in the priority cache | Bytes |
| prioritycache:full_pri0_bytes | Number of fully committed bytes at priority level 0 in the priority cache | Bytes |
| prioritycache:full_pri10_bytes | Number of fully committed bytes at priority level 10 in the priority cache | Bytes |
| prioritycache:full_pri11_bytes | Number of fully committed bytes at priority level 11 in the priority cache | Bytes |
| prioritycache:full_pri1_bytes | Number of fully committed bytes at priority level 1 in the priority cache | Bytes |
| prioritycache:full_pri2_bytes | Number of fully committed bytes at priority level 2 in the priority cache | Bytes |
| prioritycache:full_pri3_bytes | Number of fully committed bytes at priority level 3 in the priority cache | Bytes |
| prioritycache:full_pri4_bytes | Number of fully committed bytes at priority level 4 in the priority cache | Bytes |
| prioritycache:full_pri5_bytes | Number of fully committed bytes at priority level 5 in the priority cache | Bytes |
| prioritycache:full_pri6_bytes | Number of fully committed bytes at priority level 6 in the priority cache | Bytes |
| prioritycache:full_pri7_bytes | Number of fully committed bytes at priority level 7 in the priority cache | Bytes |
| prioritycache:full_pri8_bytes | Number of fully committed bytes at priority level 8 in the priority cache | Bytes |
| prioritycache:full_pri9_bytes | Number of fully committed bytes at priority level 9 in the priority cache | Bytes |
| prioritycache:full_reserved_bytes | Number of reserved fully committed bytes in the priority cache | Bytes |
| prioritycache:inc_committed_bytes | Number of incrementally committed bytes in the priority cache | Bytes |
| prioritycache:inc_pri0_bytes | Number of incrementally committed bytes at priority level 0 in the priority cache | Bytes |
| prioritycache:inc_pri10_bytes | Number of incrementally committed bytes at priority level 10 in the priority cache | Bytes |
| prioritycache:inc_pri11_bytes | Number of incrementally committed bytes at priority level 11 in the priority cache | Bytes |
| prioritycache:inc_pri1_bytes | Number of incrementally committed bytes at priority level 1 in the priority cache | Bytes |
| prioritycache:inc_pri2_bytes | Number of incrementally committed bytes at priority level 2 in the priority cache | Bytes |
| prioritycache:inc_pri3_bytes | Number of incrementally committed bytes at priority level 3 in the priority cache | Bytes |
| prioritycache:inc_pri4_bytes | Number of incrementally committed bytes at priority level 4 in the priority cache | Bytes |
| prioritycache:inc_pri5_bytes | Number of incrementally committed bytes at priority level 5 in the priority cache | Bytes |
| prioritycache:inc_pri6_bytes | Number of incrementally committed bytes at priority level 6 in the priority cache | Bytes |
| prioritycache:inc_pri7_bytes | Number of incrementally committed bytes at priority level 7 in the priority cache | Bytes |
| prioritycache:inc_pri8_bytes | Number of incrementally committed bytes at priority level 8 in the priority cache | Bytes |
| prioritycache:inc_pri9_bytes | Number of incrementally committed bytes at priority level 9 in the priority cache | Bytes |
| prioritycache:inc_reserved_bytes | Number of reserved incrementally committed bytes in the priority cache | Bytes |
| prioritycache:kv_committed_bytes | Number of committed KV bytes in the priority cache | Bytes |
| prioritycache:kv_pri0_bytes | Number of KV bytes at priority level 0 in the priority cache | Bytes |
| prioritycache:kv_pri10_bytes | Number of KV bytes at priority level 10 in the priority cache | Bytes |
| prioritycache:kv_pri11_bytes | Number of KV bytes at priority level 11 in the priority cache | Bytes |
| prioritycache:kv_pri1_bytes | Number of KV bytes at priority level 1 in the priority cache | Bytes |
| prioritycache:kv_pri2_bytes | Number of KV bytes at priority level 2 in the priority cache | Bytes |
| prioritycache:kv_pri3_bytes | Number of KV bytes at priority level 3 in the priority cache | Bytes |
| prioritycache:kv_pri4_bytes | Number of KV bytes at priority level 4 in the priority cache | Bytes |
| prioritycache:kv_pri5_bytes | Number of KV bytes at priority level 5 in the priority cache | Bytes |
| prioritycache:kv_pri6_bytes | Number of KV bytes at priority level 6 in the priority cache | Bytes |
| prioritycache:kv_pri7_bytes | Number of KV bytes at priority level 7 in the priority cache | Bytes |
| prioritycache:kv_pri8_bytes | Number of KV bytes at priority level 8 in the priority cache | Bytes |
| prioritycache:kv_pri9_bytes | Number of KV bytes at priority level 9 in the priority cache | Bytes |
| prioritycache:kv_reserved_bytes | Number of reserved KV bytes in the priority cache | Bytes |
| prioritycache_cache_bytes | Total bytes in the priority cache | Bytes |
| prioritycache_heap_bytes | Heap memory bytes in the priority cache | Bytes |
| prioritycache_mapped_bytes | Mapped bytes in the priority cache | Bytes |
| prioritycache_target_bytes | Target bytes in the priority cache | Bytes |
| prioritycache_unmapped_bytes | Unmapped bytes in the priority cache | Bytes |
| prometheus_collect_duration_seconds_count | Count of Prometheus collection durations | Times |
| prometheus_collect_duration_seconds_sum | Sum of Prometheus collection durations | Seconds |
| rocksdb_compact | Number of RocksDB compactions | Times |
| rocksdb_compact_completed | Number of completed RocksDB compactions | Times |
| rocksdb_compact_lasted | Duration of the last RocksDB compaction | Seconds |
| rocksdb_compact_queue_len | Length of the RocksDB compaction queue | Items |
| rocksdb_compact_queue_merge | Number of RocksDB compaction queue merges | Times |
| rocksdb_compact_running | Number of running RocksDB compactions | Times |
| rocksdb_get_latency_count | Count of delays for RocksDB get operations | Times |
| rocksdb_get_latency_sum | Sum of delays for RocksDB get operations | Milliseconds |
| rocksdb_rocksdb_write_delay_time_count | Count of delays during RocksDB writes | Times |
| rocksdb_rocksdb_write_delay_time_sum | Sum of delays during RocksDB writes | Milliseconds |
| rocksdb_rocksdb_write_memtable_time_count | Count of delays writing to RocksDB memtables | Times |
| rocksdb_rocksdb_write_memtable_time_sum | Sum of delays writing to RocksDB memtables | Milliseconds |
| rocksdb_rocksdb_write_pre_and_post_time_count | Count of delays during pre-and-post processing in RocksDB writes | Times |
| rocksdb_rocksdb_write_pre_and_post_time_sum | Sum of delays during pre-and-post processing in RocksDB writes | Milliseconds |
| rocksdb_rocksdb_write_wal_time_count | Count of delays writing to RocksDB WAL | Times |
| rocksdb_rocksdb_write_wal_time_sum | Sum of delays writing to RocksDB WAL | Milliseconds |
| rocksdb_submit_latency_count | Count of delays for RocksDB submit operations | Times |
| rocksdb_submit_latency_sum | Sum of delays for RocksDB submit operations | Milliseconds |
| rocksdb_submit_sync_latency_count | Count of delays for synchronous RocksDB submit operations | Times |
| rocksdb_submit_sync_latency_sum | Sum of delays for synchronous RocksDB submit operations | Milliseconds |
| trackedop_slow_ops_count | Number of tracked slow operations | Times |
| nvmeof_gateway_info | Gateway information | No unit |
| nvmeof_bdev_capacity_bytes | BDEV capacity (bytes) | Bytes |
| nvmeof_bdev_metadata | BDEV metadata | No unit |
| nvmeof_bdev_reads_completed_total | Total completed read operations | Times |
| nvmeof_bdev_writes_completed_total | Total completed write operations | Times |
| nvmeof_bdev_read_bytes_total | Total successfully read bytes | Bytes |
| nvmeof_bdev_written_bytes_total | Total successfully written bytes | Bytes |
| nvmeof_bdev_read_seconds_total | Total time spent servicing READ I/O | Seconds |
| nvmeof_bdev_write_seconds_total | Total time spent servicing WRITE I/O | Seconds |
| nvmeof_reactor_seconds_total | Active time for Reactor threads (seconds) | Seconds |
| nvmeof_subsystem_metadata | Subsystem configuration metadata | No unit |
| nvmeof_subsystem_listener_count | Number of listener addresses used by the subsystem | Items |
| nvmeof_subsystem_host_count | Number of hosts defined for the subsystem | Items |
| nvmeof_subsystem_namespace_limit | Maximum number of supported namespaces | Items |
| nvmeof_subsystem_namespace_metadata | Namespace information for the subsystem | No unit |
| nvmeof_host_connection_state | Host connection state (0=disconnected, 1=connected) | No unit |
| nvmeof_subsystem_listener_iface_info | Interface information | No unit |
| nvmeof_subsystem_listener_iface_speed_bytes | Link speed of the listener interface (bytes) | Bytes |
| nvmeof_subsystem_listener_iface_nqn_info | Usage of NIC devices by the subsystem | No unit |
| nvmeof_rpc_method_seconds | Running time of RPC method calls | Seconds |
```