Huawei Cloud DIS
Collect Huawei Cloud DIS Metrics data
Configuration¶
Install Func¶
It is recommended to enable Guance Integration - Extensions - DataFlux Func (Automata): All preconditions are automatically installed, please continue with the script installation.
If you deploy Func by yourself, refer to Deploy Func by yourself
Install Script¶
Note: Please prepare the Huawei Cloud AK that meets the requirements in advance (for simplicity, you can directly grant the global read-only permission
Tenant Guest)
Automata Version Script¶
- Log in to the Guance console
- Click the 【Integration】 menu, select 【Cloud Account Management】
- Click 【Add Cloud Account】, select 【Huawei Cloud】, fill in the required information on the interface, if the cloud account information has been configured before, ignore this step
- Click 【Test】, after the test is successful, click 【Save】, if the test fails, please check whether the relevant configuration information is correct, and test again
- Click 【Cloud Account Management】 list, you can see the added cloud account, click the corresponding cloud account, enter the details page
- Click the 【Integration】 button on the cloud account details page, find
Huawei Cloud DISin theNot Installedlist, click the 【Install】 button, and the installation interface will pop up for installation.
Manual Script¶
-
Log in to the Func console, click 【Script Market】, enter the Guance script market, search for
integration_huaweicloud_dis -
Click 【Install】, then enter the corresponding parameters: Huawei Cloud AK, SK, and account name
-
Click 【Deploy Startup Script】, the system will automatically create the
Startupscript set, and automatically configure the corresponding startup script -
After opening, you can see the corresponding automatic trigger configuration in 「Management / Automatic Trigger Configuration」. Click 【Execute】 to execute immediately without waiting for the regular time. After a while, you can check the execution task records and corresponding logs
Verification¶
- In 「Management / Automatic Trigger Configuration」, confirm whether the corresponding task has the corresponding automatic trigger configuration, and you can also check the corresponding task records and logs to check for any exceptions
- In Guance, 「Infrastructure - Resource Catalog」, check whether there is asset information
- In Guance, 「Metrics」, check whether there is corresponding monitoring data
Metrics¶
Configure Huawei Cloud DIS Metrics, you can collect more Metrics through configuration Huawei Cloud DIS Metrics Details
| Metric ID | Metric Name | Metric Description | Value Range | Measurement Object (Dimension) | Monitoring Period (Original Metric) |
|---|---|---|---|---|---|
| dis01_stream_put_bytes_rate | Total Input Traffic | This metric is used to count the amount of data uploaded to the channel within a specified time range. Unit: byte/s. | ≥ 0 bytes/s | Channel | 1 minute |
| dis02_stream_get_bytes_rate | Total Output Traffic | This metric is used to count the amount of data downloaded from the channel within a specified time range. Unit: byte/s. | ≥ 0 bytes/s | Channel | 1 minute |
| dis03_stream_put_records | Total Input Records | This metric is used to count the number of records uploaded to the channel within a specified time range. Unit: Count/s. | ≥ 0 Count/s | Channel | 1 minute |
| dis04_stream_get_records | Total Output Records | This metric is used to count the number of records downloaded from the channel within a specified time range. Unit: Count/s. | ≥ 0 Count/s | Channel | 1 minute |
| dis05_stream_put_requests_succeed | Upload Request Success Count | This metric is used to count the number of successful upload requests to the channel within a specified time range. Unit: Count/s. | ≥ 0 Count/s | Channel | 1 minute |
| dis06_stream_get_requests_succeed | Download Request Success Count | This metric is used to count the number of successful download requests from the channel within a specified time range. Unit: Count/s. | ≥ 0 Count/s | Channel | 1 minute |
| dis07_stream_put_req_average_latency | Upload Request Average Processing Time | This metric is used to count the average delay of upload requests to the channel within a specified time range. Unit: ms. | 0~50ms | Channel | 1 minute |
| dis08_stream_get_req_average_latency | Download Request Average Processing Time | This metric is used to count the average delay of download requests from the channel within a specified time range. Unit: ms. | 0~50ms | Channel | 1 minute |
| dis09_stream_traffic_control_put_records | Upload Requests Rejected Due to Traffic Control | This metric is used to count the number of upload requests rejected by the channel due to traffic control within a specified time range. Unit: Count/s. | 0~1Count/s | Channel | 1 minute |
| dis10_stream_traffic_control_get_records | Download Requests Rejected Due to Traffic Control | This metric is used to count the number of download requests rejected by the channel due to traffic control within a specified time range. Unit: Count/s. | 0~1Count/s | Channel | 1 minute |
Object¶
The collected Huawei Cloud DIS object data structure can be seen in 「Infrastructure -」.
{
"measurement": "huaweicloud_dis",
"tags": {
"RegionId" : "cn-north-4",
"data_type" : "BLOB",
"name" : "dis-YoME",
"project_id" : "c631f04625xxxxexxxxxx253c62d48585",
"status" : "RUNNING",
"stream_name" : "dis-YoME",
"stream_type" : "COMMON"
},
"fields": {
"partition_count" : 1,
"retention_period" : 24,
"auto_scale_enabled" : false,
"create_time" : 1691484876645,
"message" : "{Instance JSON Data}"
}
}
Some parameter descriptions are as follows:
| Field | Type | Description |
|---|---|---|
create_time |
integer | The creation time of the channel, 13-digit timestamp. |
retention_period |
integer | Data retention period, unit is hour. |
status |
str | The current status of the channel. CREATING:Creating RUNNING:Running TERMINATING:Deleting TERMINATED:Deleted |
stream_type |
str | Channel type. COMMON:Common channel, representing 1MB bandwidth. ADVANCED:Advanced channel, representing 5MB bandwidth. |
data_type |
str | Source data type. BLOB:A set of binary data stored in the database management system. JSON:An open file format, based on easy-to-read text, used to transmit data objects composed of attribute values or serialized values. CSV:Table data stored in plain text, the separator defaults to comma. Default value:BLOB. |
auto_scale_enabled |
bool | Whether to enable automatic scaling. true:Enable automatic scaling. false:Disable automatic scaling. Default is not enabled. Default value:false |
Note: The fields in
tags,fieldsmay change with subsequent updatesTip 1: The value of
tags.nameis the instance ID, as a unique identifierTip 2: The following fields are all JSON serialized strings
- fields.message