Collector "GoogleCloud-Cloud Monitoring" Configuration Manual¶

Before reading this document, please read:

• Cloud Integration
• GoogleCloud (GCP) Client Authorization Configuration

1. Configuration Structure¶

The configuration structure of this collector is as follows:

1.1 Metric Types¶

Before understanding the collector configuration structure, we first need to understand GCP's metric types (Metric Type). A complete metric type consists of {namespace}/{metric}, for example:

Compute Engine CPU utilization: compute.googleapis.com/instance/cpu/utilization

• namespace: compute.googleapis.com

• metric: instance/cpu/utilization

Image source: Google Cloud Metrics

1.2 Namespace (namespace)¶

The general structure of a namespace is: {service_name}.googleapis.com. However, not all namespaces follow this structure. There are other cases, such as kubernetes.io, istio.io, etc.

Here, {service_name} is the service name. The measurement reported by the collector is composed of the service name: gcp_{service_name}.

1.3 Cloud Monitoring Resource Type (monitor_resource_type)¶

Cloud monitoring metrics can be described as Google Cloud x service x resource x metric, where x resource is the resource type (monitor_resource_type).

The tags of the reported metrics refer to monitor_resource_type.

2. Configuration Examples¶

Specify Specific Metrics¶

Collect the following 2 metrics from Compute Engine: instance/cpu/utilization, guest/disk/io_time

collector_configs = {
    'targets': [
        {
            'namespace': 'compute.googleapis.com',
            'metrics'  : [
              'instance/cpu/utilization',
              'guest/disk/bytes_used'
            ]
        }
    ]
}

Wildcard Matching Metrics¶

Metric names can be matched using the * wildcard.

In this example, the following metrics will be collected:

• Metric named instance/cpu/utilization
• Metrics starting with instance
• Metrics ending with cpu/utilization
• Metrics containing cpu

collector_configs = {
    'targets': [
        {
            'namespace': 'compute.googleapis.com',
            'metrics'  : [
              'instance/cpu/utilization',
              'instance*',
              '*cpu/utilization',
              '*cpu*'
            ]
        }
    ]
}

Exclude Specific Metrics¶

Adding "NOT" at the beginning indicates the exclusion of the following metrics.

In this example, the following metrics will [not] be collected:

• Metric named instance/cpu/utilization
• Metrics starting with instance
• Metrics ending with cpu/utilization
• Metrics containing cpu

collector_configs = {
    'targets': [
        {
            'namespace': 'compute.googleapis.com',
            'metrics'  : [
              'NOT',
              'instance/cpu/utilization',
              'instance*',
              '*cpu/utilization',
              '*cpu*'
            ]
        }
    ]
}

Multiple Filters to Specify Required Metrics¶

The same namespace can be specified multiple times, and the metric names are filtered sequentially from top to bottom.

In this example, the metric names are filtered as follows:

1. Select all metrics containing cpu
2. From the previous result, exclude metrics named cpu/utilization

collector_configs = {
    'targets': [
        {
            'namespace': 'compute.googleapis.com',
            'metrics'  : ['*cpu*'],
        },
        {
            'namespace': 'compute.googleapis.com',
            'metrics'  : ['NOT', '*cpu/utilization'],
        }
    ]
}

Configure Filters (Optional)¶

This collector script supports custom filters, allowing users to filter the data to be reported based on the tags of the reported data. The filter function returns True or False.

• True: The target resource needs to be collected.

• False: The target resource does not need to be collected.

def filter_instance(instance, namespace='compute.googleapis.com'):
    '''
    Collect metrics for instance_id i-xxxxxa, i-xxxxxb
    '''
    # print(instance) # For the first use, consider printing instance to view the data structure to be filtered.
    instance_id = instance['tags'].get('instance_id')
    if instance_id in ['i-xxxxxa', 'i-xxxxxb']:
        return True
    return False

from integration_core__runner import Runner
import integration_alibabacloud_monitor__main as main

@DFF.API('GoogleCloud-Monitor Collection', timeout=300, fixed_crontab="* * * * *")
def run():
    Runner(main.DataCollector(account, collector_configs, filters=[filter_instance])).run()

3. Data Reporting Format¶

After data is successfully synchronized, you can view the data in the "Metrics" section of Guance.

Take the following collector configuration as an example:

collector_configs = {
    'targets': [
        {
            'namespace': 'compute.googleapis.com',
            'metrics'  : ['instance/cpu/utilization'],
        }
    ]
}

The reported data example is as follows:

{
  "measurement": "gcp_gce_instance",
  "tags": {
    "instance_id": "i-xxxxx",
    "userId"    : "xxxxx"
  },
  "fields": {
    "instance_cpu_utilization": 1.23
  }
}

• measurement: The definition of this field refers to the "1.2 Namespace" section.
• tags: Dimensions corresponding to the metrics, and additional tags.
• fields: The metrics in this field are the metrics configured in metrics (converted to lowercase and concatenated with underscores before reporting).

Since the metrics are all real-time data, the latest values are reported, so the timestamp is omitted (default is the timestamp of the reporting time).

4. Cloud Monitoring API Call Count Explanation¶

Google Cloud Monitoring has a free tier limit for some API calls. Before October 1, 2025: The free tier for query APIs is 1 million calls per billing account per month. Exceeding this limit incurs a charge of $0.01 per thousand calls. The /v3/project/{project_id}/timeSeries API used by this collector is also within this limit:

Find the Actual Call Count by Viewing Task Execution Logs¶

The collector counts the number of API calls for each task execution result, which can be viewed in the logs. Example:

[2023-04-21 15:32:13.194] [+0ms] The [1st] account collection is complete, took [274 ms], and called API [2 times]
[2023-04-21 15:32:13.194] [+0ms] Detailed calls are as follows:
[2023-04-21 15:32:13.194] [+0ms] -> monitoring.googleapis.com/v3/projects/{project_id}/metricDescriptors: 1 time
[2023-04-21 15:32:13.194] [+0ms] -> monitoring.googleapis.com/v3/project/{project_id}/timeSeries: 1 time

X. Appendix¶

Reference official documents:

• Google Cloud Observability Pricing
• Google Cloud Metrics
• Get Google Cloud Metrics API Method:projects.timeSeries.list