Huawei Cloud Data Warehouse Service DWS
Collect Huawei Cloud DWS monitoring metrics
Configuration¶
Install Func¶
It is recommended to activate Guance Integration - Extensions - DataFlux Func (Automata): All prerequisites are automatically installed, please proceed with the script installation.
If you need to deploy Func manually, refer to Manual Deployment of Func
Install Script¶
Note: Please prepare the Huawei Cloud AK in advance (for simplicity, you can grant the global read-only permission
Tenant Guest)
Automata Version Activation Script¶
- Log in to the Guance console
- Click the 【Integration】 menu, select 【Cloud Account Management】
- Click 【Add Cloud Account】, select 【Huawei Cloud】, and fill in the required information on the interface. If the cloud account information has been configured before, ignore this step
- Click 【Test】, if the test is successful, click 【Save】. If the test fails, please check if the relevant configuration information is correct and test again
- Click 【Cloud Account Management】 list, you can see the added cloud account, click the corresponding cloud account to enter the details page
- Click the 【Integration】 button on the cloud account details page, find
Huawei Cloud Data Warehouse Service DWSunder theNot Installedlist, click the 【Install】 button, and the installation interface will pop up for installation.
Manual Activation Script¶
-
Log in to the Func console, click 【Script Market】, enter the Guance script market, search for
integration_huaweicloud_dws -
Click 【Install】, then enter the corresponding parameters: Huawei Cloud AK, SK, and account name
-
Click 【Deploy Startup Script】, the system will automatically create the
Startupscript set and configure the corresponding startup script -
After activation, you can see the corresponding automatic trigger configuration in 「Management / Automatic Trigger Configuration」. Click 【Execute】 to immediately execute it once without waiting for the scheduled time. After a while, you can view the execution task records and corresponding logs
Verification¶
- In 「Management / Automatic Trigger Configuration」, confirm whether the corresponding task has the automatic trigger configuration, and you can also check the corresponding task records and logs for any anomalies
- In Guance, check if there is asset information in 「Infrastructure - Resource Catalog」
- In Guance, check if there is corresponding monitoring data in 「Metrics」
Metrics¶
Collect Huawei Cloud Data Warehouse Service DWS metrics, more metrics can be collected through configuration Huawei Cloud DWS Metrics Details
Huawei Cloud DWS metrics are under the huaweicloud_SYS.DWS set, below are the DWS related metrics descriptions
| Metric ID | Metric Name | Metric Meaning | Value Range | Unit | Measurement Object (Dimension) | Monitoring Period (Original Metric) |
|---|---|---|---|---|---|---|
dws001_shared_buffer_hit_ratio |
Cache Hit Ratio | This metric refers to the ratio of data requests that are already in the cache to the total data requests in the system. The higher the cache hit ratio, the higher the cache utilization of the system, and the fewer times data needs to be read from disk or network, the faster the system response speed. | 0%~100% | % | Data Warehouse Cluster | 4 minutes |
dws002_in_memory_sort_ratio |
Cache Hit Ratio | This metric is used to represent the ratio of additional memory space used by the sorting algorithm to the memory space occupied by the sorted data. | 0%~100% | % | Data Warehouse Cluster | 4 minutes |
dws003_physical_reads |
File Read Count | This metric is used to represent the cumulative number of times database files are read. | > 0 | count | Data Warehouse Cluster | 4 minutes |
dws004_physical_writes |
File Write Count | This metric is used to represent the cumulative number of times database files are written. | > 0 | count | Data Warehouse Cluster | 4 minutes |
dws005_physical_reads_per_second |
File Reads Per Second | This metric is used to represent the number of times database files are read per second. | ≥ 0 | count/s | Data Warehouse Cluster | 4 minutes |
dws006_physical_writes_per_second |
File Writes Per Second | This metric is used to represent the number of times database files are written per second. | ≥ 0 | count/s | Data Warehouse Cluster | 4 minutes |
dws007_db_size |
Data Size | This metric is used to represent the total size of all data in the database. | ≥0MB | MB | Data Warehouse Cluster | 4 minutes |
dws008_active_sql_count |
Active SQL Count | This metric is used to represent the number of currently active SQLs in the database. | ≥ 0 | count | Data Warehouse Cluster | 4 minutes |
dws009_session_count |
Session Count | This metric is used to represent the number of current connections accessing the database. | ≥ 0 | count | Data Warehouse Cluster | 4 minutes |
dws009_session_count |
Session Count | This metric is used to represent the number of current connections accessing the database. | ≥ 0 | count | Data Warehouse Cluster | 4 minutes |
dws010_cpu_usage |
CPU Usage | This metric is used to count the CPU usage of each cluster node. | 0%~100% | % | Data Warehouse Node | 1 minute |
dws011_mem_usage |
Memory Usage | This metric is used to count the memory usage of each cluster node. | 0%~100% | % | Data Warehouse Node | 1 minute |
dws012_iops |
IOPS | This metric is used to count the number of I/O requests processed per second by each cluster node. | ≥ 0 | count/s | Data Warehouse Node | 1 minute |
dws013_bytes_in |
Network Input Throughput | This metric is used to count the total amount of data input from the network to each cluster node per second. | ≥ 0 bytes/second | byte/s | Data Warehouse Node | 1 minute |
dws014_bytes_out |
Network Output Throughput | This metric is used to count the total amount of data sent from each node to the network per second. | ≥ 0 bytes/second | byte/s | Data Warehouse Node | 1 minute |
dws015_disk_usage |
Disk Utilization | This metric is used to count the disk usage of each cluster node. | 0%~100% | % | Data Warehouse Node | 1 minute |
dws016_disk_total_size |
Disk Total Size | This metric is used to count the total size of the data disk of each cluster node. | 100GB~2000GB | GB | Data Warehouse Node | 1 minute |
dws017_disk_used_size |
Disk Used Size | This metric is used to count the used size of the data disk of each cluster node. | 0GB~3600GB | GB | Data Warehouse Node | 1 minute |
dws018_disk_read_throughput |
Disk Read Throughput | This metric is used to count the amount of data read from the data disk of each cluster node per second. | ≥ 0 bytes/second | byte/s | Data Warehouse Node | 1 minute |
dws019_disk_write_throughput |
Disk Write Throughput | This metric is used to count the amount of data written to the data disk of each cluster node per second. | ≥ 0 bytes/second | byte/s | Data Warehouse Node | 1 minute |
dws020_avg_disk_sec_per_read |
Disk Read Latency | This metric is used to count the average latency of each data read from the data disk of each cluster node. | > 0 seconds | Second | Data Warehouse Node | 1 minute |
dws021_avg_disk_sec_per_write |
Disk Write Latency | This metric is used to count the average latency of each data write to the data disk of each cluster node. | > 0 seconds | Second | Data Warehouse Node | 1 minute |
dws022_avg_disk_queue_length |
Disk Average Queue Length | This metric is used to count the average I/O queue length of the data disk of each cluster node. | ≥ 0 | count | Data Warehouse Node | 1 minute |
Object¶
After the data is synchronized normally, you can view the data in Guance's 「Infrastructure - Resource Catalog」.
{
"measurement": "huaweicloud_dws",
"tags": {
"RegionId" : "cn-south-4",
"project_id" : "756ada1aa17e4049b2a16ea41912e52d",
"enterprise_project_id" : "0824ss-xxxx-xxxx-xxxx-12334fedffg",
"datastore_id" : "xxxxxx",
"datastore_name" : "test"
},
"fields": {
"status" : "AVAILABLE",
"version" : "8.0.x",
"port" : "8000",
"user_name" : "xxxxx",
"availability_zone" : "cn-south-4a",
"node_type" : "dwsk2-xxx-xxx",
"sub_status" : "xxxxxxxxx",
"create_time" : "2024-11-11T08:15:58Z",
"update_time" : "2024-11-11T08:15:58Z",
"endpoints" : "xxxxx",
"nodes" : "[]",
"number_of_node" : "3",
"recent_event" : "2",
"public_endpoints" : "[]",
"action_progress" : "{}",
"public_ip" : "xxxxxxxx"
}
}
Note: The fields in
tagsandfieldsmay change with subsequent updates