AWS S3¶

Use the「Guance Synchronization」series script package in the script market to synchronize data from cloud monitoring cloud assets to the Guance.

config¶

Install Func¶

Recommend opening 「Integrations - Extension - DataFlux Func (Automata)」: All preconditions are installed automatically, Please continue with the script installation

If you deploy Func yourself,Refer to Self-Deployment of Func

Installation script¶

Tip：Please prepare AWS AK that meets the requirements in advance（For simplicity's sake,，You can directly grant the global read-only permissionReadOnlyAccess）

To synchronize the monitoring data of AWS S3, install the corresponding collection script:「Guance Integration（AWS-S3Collect）」(ID：guance_aws_s3)

Click 【Install】 and enter the corresponding parameters: AWS AK, AWS account name.

tap【Deploy startup Script】，The system automatically creates Startup script sets，And automatically configure the corresponding startup script。

After this function is enabled, you can view the automatic triggering configuration in「Management / Crontab Config」。Click【Run】，you can immediately execute once, without waiting for a regular time。After a while, you can view task execution records and corresponding logs。

We collected some configurations by default, as described in the Metrics column Configure custom cloud object metrics

Verify¶

1. In「Management / Crontab Config」check whether the automatic triggering configuration exists for the corresponding task,In addition, you can view task records and logs to check whether exceptions exist
2. On the Guance platform, click 「Infrastructure / Custom」 to check whether asset information exists
3. On the Guance platform, press 「Metrics」 to check whether monitoring data exists

Metrics¶

After configuring Amazon CloudWatch, the default metric set includes the following metrics. You can collect more metrics by configuring it Amazon S3 CloudWatch Metrics Details

Amazon S3 daily storage metrics for buckets in CloudWatch¶

The AWS/S3 namespace includes the following daily storage metrics for buckets.

Amazon S3 request metrics in CloudWatch¶

The AWS/S3 namespace includes the following request metrics. These metrics include non-billable requests (in the case of GET requests from COPY and Replication).

S3 Replication metrics in CloudWatch¶

You can monitor the progress of replication with S3 Replication metrics by tracking bytes pending, operations pending, and replication latency. For more information, see Monitoring progress with replication metrics.

Note

You can enable alarms for your replication metrics in Amazon CloudWatch. When you set up alarms for your replication metrics, set the Missing data treatment field to Treat missing data as ignore (maintain the alarm state).

S3 Storage Lens metrics in CloudWatch¶

You can publish S3 Storage Lens usage and activity metrics to Amazon CloudWatch to create a unified view of your operational health in CloudWatch dashboards. S3 Storage Lens metrics are published to the AWS/S3/Storage-Lens namespace in CloudWatch. The CloudWatch publishing option is available for S3 Storage Lens dashboards that have been upgraded to advanced metrics and recommendations.

For a list of S3 Storage Lens metrics that are published to CloudWatch, see Amazon S3 Storage Lens metrics glossary. For a complete list of dimensions, see Dimensions.

S3 Object Lambda request metrics in CloudWatch¶

S3 Object Lambda includes the following request metrics.

Amazon S3 on Outposts metrics in CloudWatch¶

For a list of metrics in CloudWatch that are used for S3 on Outposts buckets, see CloudWatch metrics.

Amazon S3 dimensions in CloudWatch¶

The following dimensions are used to filter Amazon S3 metrics.

S3 Storage Lens dimensions in CloudWatch¶

For a list of dimensions that are used to filter S3 Storage Lens metrics in CloudWatch, see Dimensions.

S3 Object Lambda request dimensions in CloudWatch¶

The following dimensions are used to filter data from an Object Lambda Access Point.

Object¶

The collected AWS S3 object data structure can be viewed in "Infrastructure - Custom" under the object data.

{
  "measurement": "aws_s3",
  "tags": {
    "name"              : "dataxxxx",
    "RegionId"          : "cn-northwest-1",
    "LocationConstraint": "cn-northwest-1",
    "Name"              : "dataxxxx"
  },
  "fields": {
    "CreationDate": "2022-03-09T06:13:31Z",
    "Grants"      : "{JSON data}",
    "message"     : "{Instance JSON data}"
  }
}

Note: The fields in tags and fields may be subject to changes in subsequent updates.

Tip 1: The value of tags.name is the instance name and serves as a unique identifier.

Tip 2: fields.message is a JSON-serialized string.

Tip 3: fields.Grants represents the bucket access control list.