Cloud Integration Notes¶

Please read the following notes and instructions before using this series of script packages.

1. Fixed Cloud Provider Tags¶

Data reported to Guance using this series of script packages will automatically include the cloud_provider tag based on the corresponding cloud provider to distinguish between them.

Therefore, do not add a tag with the Key cloud_provider in the extra_tags of the account configuration.

The specific cloud platforms and their corresponding fixed tags are as follows:

2. Supplementary Cloud Monitoring Data¶

Generally, cloud monitoring products from various cloud providers only provide the monitored object's ID and its monitoring metrics, such as the instance ID and CPU usage of an ECS. They do not return additional information like the instance name.

Therefore, when only cloud monitoring collectors are enabled, although monitoring data can be viewed and alerts can be set, distinguishing different objects solely by ID can be very unintuitive.

Hence, when cloud monitoring collectors are enabled alongside corresponding custom object collectors, the system will automatically supplement the cloud monitoring data with more instance-related information based on IDs and other data.

Since custom object information must be obtained first for the cloud monitoring collectors to perform the linkage, it is generally recommended to place cloud monitoring collectors at the end of the list, such as:

# Create collectors
collectors = [
    aliyun_ecs.DataCollector(account, common_aliyun_configs),
    aliyun_rds.DataCollector(account, common_aliyun_configs),
    aliyun_slb.DataCollector(account, common_aliyun_configs),
    aliyun_oss.DataCollector(account, common_aliyun_configs),

    # Cloud monitoring collectors are usually placed at the end
    aliyun_monitor.DataCollector(account, monitor_collector_configs),
]

3. Collection Frequency Limits for Different Collectors¶

Even if a shorter trigger interval (e.g., 1 minute) is configured in the "Scheduled Tasks (Legacy: Auto Trigger Configuration)", each collector has its own minimum internal collection interval based on its business characteristics.

For most custom object collectors, the minimum internal collection interval is 15 minutes. If triggered continuously within 15 minutes, the program will automatically skip the actual collection process to avoid excessive API calls.

For cloud monitoring collectors, the minimum internal collection interval is 1 minute, but it is recommended to set a reasonable trigger interval (e.g., 5 minutes) based on actual needs to avoid excessive API calls.

4. Data Collection Latency¶

This series of scripts will have some data latency during the collection process.

For custom object collectors, due to the minimum internal collection interval, when new instances are purchased, released, or instance names are modified, it may take until the next actual collection execution for these changes to be reflected in Guance.

For cloud monitoring collectors, when "Supplementary Cloud Monitoring Data" is involved, since the supplementary tags come from the collection processing of custom object collectors, they are also subject to the minimum internal collection interval limitation of custom object collectors mentioned above.

5. Accuracy of Cloud Monitoring Data¶

For cloud monitoring collectors, since the data is obtained through cloud provider APIs rather than raw data, the collected cloud monitoring data is essentially "second-hand data" and may differ from the data seen in the cloud provider's console.

6. Potential Massive API Calls and Data¶

Some collectors, if configured to fetch full data, can generate massive API queries, easily triggering cloud platform rate limits or even getting the AK blocked.

Therefore, when writing collector configurations, full consideration should be given based on understanding the specific product API call rules and the data volume generated in Guance.

The script author and related parties are not responsible for any financial losses incurred as a result.

7. Time Consumption When Collecting Multiple Products Simultaneously¶

When configuring multiple collectors in a single function, due to the large number of API requests to the cloud provider, each execution may take a long time.

Taking Alibaba Cloud as an example, the following collection scheme:

A single run takes approximately 1 to 2 minutes.

Therefore, generally, when collecting multiple products simultaneously in a single function, it is necessary to specify a longer timeout timeout (in seconds) for the function, such as:

# Due to the large amount of data collected, a larger timeout (in seconds) is required here
@DFF.API('Execute Cloud Asset Sync', timeout=300)
def run():
    # Specific code omitted