Huawei Cloud Data Migration CDM
Collect Huawei Cloud CDM monitoring metrics
Configuration¶
Install Func¶
It is recommended to activate Guance Integration - Extensions - DataFlux Func (Automata): All prerequisites are automatically installed, please proceed with the script installation
If you want to deploy Func by yourself, refer to Deploy Func by Yourself
Install Script¶
Note: Please prepare the Huawei Cloud AK that meets the requirements in advance (for simplicity, you can directly grant the global read-only permission
ReadOnlyAccess
)
-
Log in to the Func console, click 【Script Market】, enter the official script market, and search for
guance_huaweicloud_cdm
-
Click 【Install】, then enter the corresponding parameters: Huawei Cloud AK, SK, and account name
-
Click 【Deploy Startup Script】, the system will automatically create the
Startup
script set and configure the corresponding startup script -
After enabling, you can see the corresponding automatic trigger configuration in 「Manage / Automatic Trigger Configuration」. Click 【Execute】 to execute it immediately without waiting for the scheduled time. Wait for a moment, you can check the execution task records and corresponding logs
Verification¶
- In 「Manage / Automatic Trigger Configuration」, confirm whether the corresponding task has the automatic trigger configuration, and check the corresponding task records and logs for any exceptions
- In Guance, check if the asset information exists in 「Infrastructure - Resource Catalog」
- In Guance, check if there are corresponding monitoring data in 「Metrics」
Metrics¶
Collect Huawei Cloud Data Migration CDM metrics, more metrics can be collected through configuration Huawei Cloud CDM Metrics Details
Huawei Cloud CDM metrics are under the huaweicloud_SYS.CDM
Measurement, below are the CDM related metrics descriptions
Metric ID | Metric Name | Metric Meaning | Value Range | Unit | Measurement Object (Dimension) | Monitoring Period (Original Metric) |
---|---|---|---|---|---|---|
bytes_in |
Network Inbound Rate | This metric is used to count the network traffic per second flowing into the measurement object. | ≥ 0 bytes/s | bytes/s | CDM Cluster Instance | 1 minute |
bytes_out |
Network Outbound Rate | This metric is used to count the network traffic per second flowing out of the measurement object. | ≥ 0 bytes/s | bytes/s | CDM Cluster Instance | 1 minute |
cpu_usage |
CPU Usage | This metric is used to count the CPU usage of the measurement object. | 0%~100% | % | CDM Cluster Instance | 1 minute |
mem_usage |
Memory Usage | This metric is used to count the memory usage of the measurement object. | 0%~100% | % | CDM Cluster Instance | 1 minute |
pg_pending_job |
Pending Jobs | This metric is used to count the number of jobs in PENDING state in this CDM instance. | >=0 | Count | CDM Cluster Instance | 1 minute |
pending_threads |
Pending Extraction Concurrency | This metric is used to count the number of extraction concurrency threads in Waiting state in this CDM instance. | ≥ 0 | Count | CDM Cluster Instance | 1 minute |
disk_usage |
Disk Utilization | This metric is collected from the physical machine level, and the data accuracy is lower than that collected from the inside of the Elastic Cloud Server. | 0.001%~90% | % | CDM Cluster Instance | 1 minute |
disk_io |
Disk IO | This metric is collected from the physical machine level, and the data accuracy is lower than that collected from the inside of the Elastic Cloud Server. | ≥ 0 byte/s | byte/s | CDM Cluster Instance | 1 minute |
tomcat_heap_usage |
Heap Memory Usage | This metric is collected from the physical machine level, and the data accuracy is lower than that collected from the inside of the Elastic Cloud Server. | 0.001%~90% | % | CDM Cluster Instance | 1 minute |
tomcat_connect |
Tomcat Concurrent Connections | This metric is collected from the physical machine level, and the data accuracy is lower than that collected from the inside of the Elastic Cloud Server. | 0~2147483647 | Count | CDM Cluster Instance | 1 minute |
tomcat_thread_count |
Tomcat Threads | This metric is collected from the physical machine level, and the data accuracy is lower than that collected from the inside of the Elastic Cloud Server. | 0~2147483647 | Count | CDM Cluster Instance | 1 minute |
pg_connect |
Database Connections | This metric is collected from the physical machine level, and the data accuracy is lower than that collected from the inside of the Elastic Cloud Server. | 0~2147483647 | Count | CDM Cluster Instance | 1 minute |
pg_submission_row |
History Table Rows | This metric is collected from the physical machine level, and the data accuracy is lower than that collected from the inside of the Elastic Cloud Server. | 0~2147483647 | Count | CDM Cluster Instance | 1 minute |
pg_failed_job_rate |
Failed Job Rate | This metric is collected from the sqoop process at the physical machine level, and the data accuracy is lower than that collected from the inside of the Elastic Cloud Server. | 0.001%~100% | % | CDM Cluster Instance | 1 minute |
inodes_usage |
Inodes Utilization | This metric is collected from the physical machine level, and the data accuracy is lower than that collected from the inside of the Elastic Cloud Server. | 0.001%~100% | % | CDM Cluster Instance | 1 minute |
Object¶
After the data is synchronized normally, you can view the data in Guance's 「Infrastructure - Resource Catalog」.
{
"measurement": "huaweicloud_cdm",
"tags": {
"RegionId" : "cn-south-4",
"project_id" : "756ada1aa17e4049b2a16ea41912e52d",
"enterprise_project_id" : "0824ss-xxxx-xxxx-xxxx-12334fedffg",
"cluster_id" : "xxxxxx",
"cluster_name" : "test",
"instance_name" : "test-xxxx",
"instance_id" : "xxxxxxxxxx",
"status" : "AVAILABLE"
},
"fields": {
"azName" : "xszxxxxxx",
"dbuser" : "root",
"flavorName" : "xxxx",
"clusterMode" : "xxxxx",
"publicEndpoint" : "cn-south-4a",
"statusDetail" : "cdmk2-xxx-xxx",
"create_time" : "2024-11-11T08:15:58Z",
"update_time" : "2024-11-11T08:15:58Z",
"recentEvent" : "xxxxx",
"datastore" : "xxx",
"isAutoOff" : "xxx",
"task" : "xxx",
"actionProgress" : "xxx"
}
}
Note: The fields in
tags
andfields
may change with subsequent updates ```