Collector "Tencent Cloud - Cloud Monitor" Configuration Manual¶

Before reading this document, please read:

Cloud Integration

Tip

Before using this collector, you must install "Integration Core" and its accompanying third-party dependencies.

Tip

Tencent Cloud Cloud Monitor Metrics collection depends on the resource attributes of related products. You must first configure the custom object collector for that product. Deployment via the script market will automatically configure this.

Tip

This collector supports multi-threading by default (five threads are enabled by default). If you need to change the thread pool size, you can set the environment variable COLLECTOR_THREAD_POOL_SIZE.

1. Configuration Structure¶

The configuration structure of this collector is as follows:

Field	Type	Required	Description
`Regions`	list	Required	List of regions for Cloud Monitor collection
`regions[#]`	str	Required	Region ID, e.g., `ap-shanghai` See appendix for full list
`targets`	list	Required	List of Cloud Monitor collection object configurations Multiple configurations under the same namespace have a logical "AND" relationship
`targets[#].namespace`	str	Required	Namespace for Cloud Monitor collection. e.g., `QCE/CVM`
`targets[#].metrics`	list	Required	List of Cloud Monitor metric names to collect See appendix for full list
`targets[#].metrics[#]`	str	Required	Metric name pattern, supports `"NOT"`, wildcard matching Normally, multiple metrics have a logical "OR" relationship. When `"NOT"` is included, multiple metrics have a logical "AND" relationship. See below for details

2. Configuration Examples¶

Specify Specific Metrics¶

Collect two metrics named WanOuttraffic and WanOutpkg from QCE/CVM.

tencentcloud_monitor_configs = {
    'regions': ['ap-shanghai'],
    'targets': [
        {
            'namespace': 'QCE/CVM',
            'metrics'  : ['WanOuttraffic', 'WanOutpkg'],
        }
    ]
}

Wildcard Matching Metrics¶

Metric names can use * wildcard for matching.

In this example, the following metrics will be collected:

Metrics named WanOutpkg
Metrics starting with Wan
Metrics ending with Outpkg
Metrics containing Out

tencentcloud_monitor_configs = {
    'regions': ['ap-shanghai'],
    'targets': [
        {
            'namespace': 'QCE/CVM',
            'metrics'  : ['WanOutpkg', 'Wan*', '*Outpkg', '*Out*']
        }
    ]
}

Exclude Specific Metrics¶

Adding "NOT" at the beginning indicates excluding the following metrics.

In this example, the following metrics will [not] be collected:

Metrics named WanOutpkg
Metrics starting with Wan
Metrics ending with Outpkg
Metrics containing Out

tencentcloud_monitor_configs = {
    'regions': ['ap-shanghai'],
    'targets': [
        {
            'namespace': 'QCE/CVM',
            'metrics'  : ['NOT', 'WanOutpkg', 'Wan*', '*Outpkg', '*Out*']
        }
    ]
}

Multi-level Filtering for Specific Metrics¶

The same namespace can be specified multiple times, with metric names filtered sequentially from top to bottom.

In this example, the metric names are filtered as follows:

Select all metrics containing Out
From the previous result, exclude metrics named WanOutpkg

tencentcloud_monitor_configs = {
    'regions': ['ap-shanghai'],
    'targets': [
        {
            'namespace': 'QCE/CVM',
            'metrics'  : ['*Out*']
        },
        {
            'namespace': 'QCE/CVM',
            'metrics'  : ['NOT', 'WanOutpkg']
        }
    ]
}

3. Data Reporting Format¶

After data is synchronized normally, you can view the data in the "Metrics" section of Guance.

Take the following collector configuration as an example:

tencentcloud_monitor_configs = {
    'regions': ['ap-shanghai'],
    'targets': [
        {
            'namespace': 'QCE/CVM',
            'metrics'  : ['WanOutpkg']
        }
    ]
}

The reported data example is as follows:

{
  "measurement": "tencentcloud_QCE/CVM",
  "tags": {
    "InstanceId": "i-xxx"
  },
  "fields": {
    "WanOutpkg_max": 0.005
  }
}

Tip

All metric values will be reported as float type.

Tip

This collector collects the WanOutpkg metric data under the QCE/CVM namespace (Namespace). See the Data Collection Description table for details.

4. Interaction with Custom Object Collectors¶

When you enable a custom object collector (e.g., CVM), this collector will supplement fields based on the dimension information in the Data Collection Description. For example, CVM attempts to match the tags.name field in the custom object based on the InstanceId field returned by Cloud Monitor data.

Since custom object information needs to be obtained first for interaction in the Cloud Monitor collector, it is generally recommended to place the Cloud Monitor collector at the end of the list, such as:

# Create collectors
collectors = [
    tencentcloud_cvm.DataCollector(account, common_tencentcloud_configs), # CVM custom object collector
    tencentcloud_monitor.DataCollector(account, tencentcloud_monitor_configs) # Cloud Monitor collector is generally placed last
]

When a successful match is made, additional fields from the custom object tags will be added to the Cloud Monitor data tags, enabling effects such as filtering Cloud Monitor metric data using instance names. The specific effect is as follows:

Assume the original data collected by Cloud Monitor is as follows:

{
  "measurement": "tencentcloud_QCE/CVM",
  "tags": {
    "InstanceId": "i-xxx"
  },
  "fields": { "..." }
}

At the same time, the custom object data collected by the Tencent Cloud CVM collector is as follows:

{
  "measurement": "tencentcloud_cvm",
  "tags": {
    "name"           : "i-xxx",
    "InstanceType"   : "c6g.xxx",
    "PlatformDetails": "xxx",
    "{...}"
  },
  "fields": { "..." }
}

Then, the final reported Cloud Monitor data is as follows:

{
  "measurement": "tencentcloud_QCE/CVM",
  "tags": {
    "name"            : "i-xxx",
    "InstanceId"          : "i-xxx",   // Original field from Cloud Monitor
    "InstanceType"    : "c6g.xxx", // Field from custom object CVM
    "PlatformDetails" : "xxx",     // Field from custom object CVM
    "{Other fields omitted}"
  },
  "fields": { "Content omitted" }
}

5. Cloud Monitor API Call Count Explanation¶

Tencent Cloud Cloud Monitor has a free quota limit for some API calls (this collector uses the GetMonitorData API to request monitoring data. It belongs to the free quota API. The free request quota for each primary account is 1 million times/month. Excess charges are 0.25 yuan/10,000 times. Additionally, after exceeding the free quota, you will not be able to continue using it. To continue calling the interface, you need to manually enable "Pay-as-you-go for API Requests". For billing rules, see API Billing Document.) The following provides a detailed explanation of the script set call count:

View real call counts by checking task execution logs

The collector counts the number of API calls for each task execution result, which can be viewed in the logs, for example:

[2023-04-24 19:02:02.359] [+1156ms] The [1st] account collection is complete, executed in [1155 milliseconds], during which [2] API calls were made
[2023-04-24 19:02:02.360] [+0ms] Detailed calls are as follows
[2023-04-24 19:02:02.360] [+0ms] -> monitor.tencentcloudapi.com/?Action=DescribeBaseMetrics: 1 call
[2023-04-24 19:02:02.565] [+0ms] -> monitor.tencentcloudapi.com/?Action=GetMonitorData: 1 call

Warning

Given the free quota for Cloud Monitor calls, it is recommended that users configure monitoring items as needed to avoid additional costs caused by wildcards.

Precautions¶

Task Error Conditions and Solutions¶

HTTPClientError: An HTTP Client raised an unhandled exception: SoftTimeLimitExceeded()

Reason: Task execution time is too long (timeout).

Solution:

Appropriately increase the task timeout setting (e.g., @DFF.API('Execute Collection', timeout=120, fixed_crontab="* * * * *"), which sets the task timeout to 120 seconds).
[TencentCloudSDKException] code:InvalidParameterValue message:cannot find metricName=xxx configure

Reason: Tencent Cloud does not support the collection of this metric (there are cases where the metric is documented by Tencent Cloud but not actually supported).

Solution:

It is recommended to refer to the Monitoring Metric Configuration Information in this document to configure valid metric names.
[TencentCloudSDKException] code:InvalidParameterValue message: xxxxx does not belong to the developer ....

Reason: When collecting Cloud Monitor data for a product under an account, the product has been released, causing the interface to throw an error. This can be ignored.

X. Appendix¶

Tencent Cloud Cloud Monitor¶

Please refer to the official Tencent Cloud documentation: