Skip to content

AWS SageMaker

Collect endpoint assets, invocations, latency, and instance resource metrics from AWS SageMaker. An endpoint is the entry point to a SageMaker inference service, while a variant defines the model deployment, instance type, instance count, and traffic distribution.

Configuration

Install Func

We recommend enabling Guance Integration - Extensions - Hosted Func. All prerequisites are installed automatically, after which you can continue with script installation.

For a self-hosted Func deployment, see Self-hosted Func.

Enable the Script

Note: Prepare an AWS access key that meets the requirements in advance. For simplicity, you can grant the global read-only ReadOnlyAccess permission.

  1. Sign in to the Func console, click Script Market, open the Guance Script Market, and search for integration_aws_sagemaker.
  2. Click Install, then enter the AWS AK ID, AK Secret, and account name.
  3. Click Deploy Startup Script. The system automatically creates a Startup script set and configures the corresponding startup script.
  4. After enabling it, find the corresponding automatic trigger under Management / Automatic Trigger Configuration. Click Run to execute it immediately without waiting for the scheduled time. After a moment, review the task execution records and logs.

Verification

  1. Under Management / Automatic Trigger Configuration, confirm that the task has an automatic trigger, then review its task records and logs for errors.
  2. In Guance, go to Infrastructure / Custom and verify that asset information is available.
  3. In Guance, go to Metrics and verify that monitoring data is available.

Metrics

aws_sagemaker

AWS namespace: AWS/SageMaker.

MetricName MetricDescribe Dimensions Unit
invocation_4xx_errors_average Average ratio of HTTP 4xx errors during a CloudWatch period; raw values range from 0 to 1. account_name, region_id, endpoint_name, variant_name, dimensions ratio
invocation_4xx_errors_sum Number of HTTP 4xx errors during a CloudWatch period. account_name, region_id, endpoint_name, variant_name, dimensions count
invocation_5xx_errors_average Average ratio of HTTP 5xx errors during a CloudWatch period; raw values range from 0 to 1. account_name, region_id, endpoint_name, variant_name, dimensions ratio
invocation_5xx_errors_sum Number of HTTP 5xx errors during a CloudWatch period. account_name, region_id, endpoint_name, variant_name, dimensions count
invocation_model_errors_average Average ratio of model invocation errors during a CloudWatch period; raw values range from 0 to 1. account_name, region_id, endpoint_name, variant_name, dimensions ratio
invocation_model_errors_sum Number of model invocation errors during a CloudWatch period, including connection failures, timeouts, and abnormal responses; these may overlap with HTTP errors. account_name, region_id, endpoint_name, variant_name, dimensions count
invocations_per_instance_sum Number of invocations during a CloudWatch period normalized by the instance count at that time; multiplying it by the current instance count cannot reconstruct the historical total. account_name, region_id, endpoint_name, variant_name, dimensions count
invocations_sum Number of invocations during a CloudWatch period, not a cumulative counter or QPS. account_name, region_id, endpoint_name, variant_name, dimensions count
model_cache_hit_average Ratio of requests for which the model is already in memory; raw values range from 0 to 1. account_name, region_id, endpoint_name, variant_name, dimensions ratio
model_downloading_time_average Average time to download a model from S3. account_name, region_id, endpoint_name, variant_name, dimensions μs
model_latency_average Average time for the model to process a request. account_name, region_id, endpoint_name, variant_name, dimensions μs
model_loading_time_average Average time for the inference container to load a model. account_name, region_id, endpoint_name, variant_name, dimensions μs
model_loading_wait_time_average Average time a request waits for a model to be downloaded or loaded. account_name, region_id, endpoint_name, variant_name, dimensions μs
overhead_latency_average Average additional processing time on the SageMaker platform, excluding client network round-trip time. account_name, region_id, endpoint_name, variant_name, dimensions μs

aws_sagemaker_endpoints

AWS namespace: /aws/sagemaker/Endpoints.

MetricName MetricDescribe Dimensions Unit
cpuutilization_average Average sum of utilization across all CPU cores on an instance; it can exceed 100% on multi-core instances. account_name, region_id, endpoint_name, variant_name, dimensions percent
disk_utilization_average Average utilization of instance disk space, not disk I/O utilization. account_name, region_id, endpoint_name, variant_name, dimensions percent
loaded_model_count_average Average number of loaded models per instance, not the total number of unique models across instances. account_name, region_id, endpoint_name, variant_name, dimensions count
memory_utilization_average Average instance memory utilization; multi-model caching can legitimately consume memory. account_name, region_id, endpoint_name, variant_name, dimensions percent

Objects

Endpoint objects are stored in aws_sagemaker_endpoint. Only verified scalar fields are shown below:

{
  "measurement": "aws_sagemaker_endpoint",
  "tags": {
    "account_name": "pdsa-tw",
    "region_id": "ap-northeast-2",
    "endpoint_name": "guance-sagemaker-mme-20260909"
  },
  "fields": {
    "endpoint_config_name": "guance-sagemaker-mme-20260909",
    "endpoint_status": "InService",
    "last_modified_time": "2026-09-09T15:56:35.141000+08:00",
    "resource_creation_time": "2026-09-09T15:53:14.838000+08:00"
  }
}

Note: Fields in tags and fields may change in future updates.

Feedback

Is this page helpful?