Huawei Cloud Data Migration CDM
Huawei Cloud CDM monitoring metrics collection
Configuration¶
Install Func¶
It is recommended to enable Guance Integration - Extensions - Managed Func: all prerequisites will be installed automatically, so please continue with the script installation
If you deploy Func yourself, refer to Deploy Func yourself
Install script¶
Note: Please prepare a Huawei Cloud AK that meets the requirements in advance (for simplicity, you can directly grant the global read-only permission
Tenant Guest)
Managed version activation script¶
- Log in to the Guance console
- Click the [Integration] menu and select [Cloud Account Management]
- Click [Add Cloud Account], select [Huawei Cloud], and fill in the required information. If cloud account information has already been configured, ignore this step
- Click [Test]. After the test succeeds, click [Save]. If the test fails, check whether the relevant configuration information is correct and test again
- In the [Cloud Account Management] list, you can see the added cloud account. Click the corresponding cloud account to enter the details page
- Click the [Integration] button on the cloud account details page. In the
Not Installedlist, findHuawei Cloud APIGand click [Install] to open the installation interface and complete the installation
Manual activation script¶
-
Log in to the Func console, click [Script Market], enter the Guance script market, and search for
integration_huaweicloud_cdm -
After clicking [Install], enter the corresponding parameters: Huawei Cloud AK, SK, and account name
-
Click [Deploy and Start Script]. The system will automatically create a
Startupscript set and configure the corresponding startup script -
After enabling it, you can see the corresponding automatic trigger configuration under "Management / Automatic Trigger Configuration". Click [Execute] to run it immediately without waiting for the scheduled time. After a short while, you can view the execution task records and the corresponding logs
Verification¶
- In "Management / Automatic Trigger Configuration", confirm whether the corresponding task has an automatic trigger configuration, and check the corresponding task records and logs for abnormalities
- In Guance, check whether asset information exists in "Infrastructure - Resource Directory"
- In Guance, check whether there is corresponding monitoring data under "Metrics"
Metrics¶
To collect Huawei Cloud Data Migration CDM metrics, you can collect more metrics through configuration. See Huawei Cloud CDM metric details
Huawei Cloud CDM metrics are in the huaweicloud_SYS.CDM group. The following describes the CDM-related metrics
| Metric ID | Metric Name | Metric Description | Value Range | Unit | Measured Object (Dimension) | Monitoring Period (Raw Metric) |
|---|---|---|---|---|---|---|
bytes_in |
Network Ingress Rate | This metric is used to count the network traffic flowing into the measured object per second. | ≥ 0 bytes/s | bytes/s | CDM cluster instance | 1 minute |
bytes_out |
Network Egress Rate | This metric is used to count the network traffic flowing out of the measured object per second. | ≥ 0 bytes/s | bytes/s | CDM cluster instance | 1 minute |
cpu_usage |
CPU Utilization | This metric is used to count the CPU utilization of the measured object. | 0%~100% | % | CDM cluster instance | 1 minute |
mem_usage |
Memory Utilization | This metric is used to count the memory utilization of the measured object. | 0%~100% | % | CDM cluster instance | 1 minute |
pg_pending_job |
Pending Jobs | This metric is used to count the number of jobs in the PENDING state in this CDM instance. | >=0 | Count | CDM cluster instance | 1 minute |
pending_threads |
Pending Extraction Concurrency | This metric is used to count the number of extraction concurrent threads in the Waiting state in this CDM instance. | ≥ 0 | Count | CDM cluster instance | 1 minute |
disk_usage |
Disk Utilization | This metric is the disk usage collected at the physical machine level, and its data accuracy is lower than data collected from inside the ECS. | 0.001%~90% | % | CDM cluster instance | 1 minute |
disk_io |
Disk IO | This metric is the number of bytes read and written per second for disks collected at the physical machine level, and its data accuracy is lower than data collected from inside the ECS. | ≥ 0 byte/s | byte/s | CDM cluster instance | 1 minute |
tomcat_heap_usage |
Heap Memory Utilization | This metric is the heap memory utilization collected at the physical machine level, and its data accuracy is lower than data collected from inside the ECS. | 0.001%~90% | % | CDM cluster instance | 1 minute |
tomcat_connect |
Tomcat Concurrent Connections | This metric is the number of Tomcat concurrent connections collected at the physical machine level. | 0~2147483647 | Count | CDM cluster instance | 1 minute |
tomcat_thread_count |
Tomcat Threads | This metric is the number of threads occupied by Tomcat collected at the physical machine level. | 0~2147483647 | Count | CDM cluster instance | 1 minute |
pg_connect |
Database Connections | This metric is the number of PostgreSQL database connections collected at the physical machine level. | 0~2147483647 | Count | CDM cluster instance | 1 minute |
pg_submission_row |
Historical Table Row Count | This metric is the number of rows in the PostgreSQL database submission table collected at the physical machine level. |
0~2147483647 | Count | CDM cluster instance | 1 minute |
pg_failed_job_rate |
Failed Job Rate | This metric is the failed job rate collected from the sqoop process at the physical machine level. | 0.001%~100% | % | CDM cluster instance | 1 minute |
inodes_usage |
Inodes Utilization | This metric is the disk inodes usage collected at the physical machine level, and its data accuracy is lower than data collected from inside the ECS. | 0.001%~100% | % | CDM cluster instance | 1 minute |
Objects¶
After data is synchronized normally, you can view the data in Guance under "Infrastructure - Resource Directory".
{
"measurement": "huaweicloud_cdm",
"tags": {
"RegionId" : "cn-south-4",
"project_id" : "756ada1aa17e4049b2a16ea41912e52d",
"enterprise_project_id" : "0824ss-xxxx-xxxx-xxxx-12334fedffg",
"cluster_id" : "xxxxxx",
"cluster_name" : "test",
"instance_name" : "test-xxxx",
"instance_id" : "xxxxxxxxxx",
"status" : "AVAILABLE"
},
"fields": {
"azName" : "xszxxxxxx",
"dbuser" : "root",
"flavorName" : "xxxx",
"clusterMode" : "xxxxx",
"publicEndpoint" : "cn-south-4a",
"statusDetail" : "cdmk2-xxx-xxx",
"create_time" : "2024-11-11T08:15:58Z",
"update_time" : "2024-11-11T08:15:58Z",
"recentEvent" : "xxxxx",
"datastore" : "xxx",
"isAutoOff" : "xxx",
"task" : "xxx",
"actionProgress" : "xxx"
}
}
Note: The fields in
tagsandfieldsmay change with subsequent updates