Skip to content

Alibaba Cloud KafKa

Alibaba Cloud KafKa includes instance disk usage, instance and topic message production volume, message production frequency, message consumption volume, message consumption frequency, etc. These metrics reflect the reliability guarantee of Kafka when handling large-scale message transmission and real-time data streams.

Configuration

Install Func

It is recommended to enable Guance integration - extension - hosted Func.

If you deploy Func yourself, refer to Self-deployed Func

Enable Script

Note: Please prepare the required Alibaba Cloud AK in advance (for simplicity, you can directly grant global read-only permission ReadOnlyAccess)

Hosted Version Enable Script

  1. Log in to the Guance console
  2. Click on the 【Manage】 menu, select 【Cloud Account Management】
  3. Click 【Add Cloud Account】, select 【Alibaba Cloud】, fill in the required information on the interface; if cloud account information has been configured before, skip this step
  4. Click 【Test】, after a successful test click 【Save】. If the test fails, check whether the related configuration information is correct and retest
  5. In the 【Cloud Account Management】 list, you can see the added cloud accounts. Click on the corresponding cloud account to enter the details page
  6. Click the 【Integration】 button on the cloud account details page. In the Not Installed list, find Alibaba Cloud KafKa, click the 【Install】 button, and follow the installation interface to complete the installation.

Manual Enable Script

  1. Log in to the Func console, click 【Script Market】, enter the official script market, search for guance_aliyun_alikafka

  2. After clicking 【Install】, input the corresponding parameters: Alibaba Cloud AK ID, AK Secret, and account name.

  3. Click 【Deploy Startup Script】, the system will automatically create a Startup script set and automatically configure the corresponding startup script.

  4. After enabling, you can see the corresponding automatic trigger configuration in 「Management / Automatic Trigger Configuration」. Click 【Execute】 to immediately execute once without waiting for the scheduled time. Wait a moment, then you can view the execution task records and corresponding logs.

We default collect some configurations, see the Metrics section for details.

Configure Custom Cloud Object Metrics

Verification

  1. In 「Management / Automatic Trigger Configuration」, confirm that the corresponding task has an automatic trigger configuration, and at the same time, check the corresponding task records and logs for any abnormalities.
  2. In Guance, under 「Infrastructure / Custom」, check if there is asset information.
  3. In Guance, under 「Metrics」, check if there are corresponding monitoring data.

Metrics

After configuring Alibaba Cloud - Cloud Monitoring, the default metric set is as follows. You can collect more metrics through configuration. Alibaba Cloud Cloud Monitoring Metric Details

Metric Id Metric Name Dimensions Statistics Unit
instance_disk_capacity_Maximum V2 Instance Disk Usage Rate userId,instanceId Maximum %
instance_message_input Instance Message Production Volume userId,instanceId Value bytes/s
instance_message_num_input Instance Message Production Count userId,instanceId Value countSecond
instance_message_output Instance Message Consumption Volume userId,instanceId Value bytes/s
instance_reqs_input Instance Message Sending Frequency userId,instanceId Value countSecond
instance_reqs_output Instance Message Consumption Frequency userId,instanceId Value countSecond
topic_message_input Topic Message Production Volume userId,instanceId Value bytes/s
topic_message_num_input Topic Message Production Count userId,instanceId Value countSecond
topic_message_output Topic Message Consumption Volume userId,instanceId Value bytes/s
topic_reqs_input Topic Message Sending Frequency userId,instanceId Value countSecond
topic_reqs_output Topic Message Consumption Frequency userId,instanceId Value countSecond

Objects

The object data structure of Alibaba Cloud KafKa collected can be seen in 「Infrastructure - Custom」

{
  "measurement": "aliyun_kafka",
  "tags": {
    "InstanceId"        : "alikafka_post-cn-zsk3cluq100d",
    "InstanceName"      : "alikafka_post-cn-zsk3cluq100d",
    "RegionId"          : "cn-hangzhou",
    "ResourceGroupId"   : "rg-acfmv3ro3xnfwaa",
    "SpecType"          : "normal",
    "cloud_provider"    : "aliyun",
    "name"              : "alikafka_post-cn-zsk3cluq100d"
  },
  "fields": {
    "AllConfig"         : "{}",
    "CreateTime"        : 1692080715000,
    "DeployType"        : 5,
    "EipMax"            : 0,
    "ExpiredTime"       : 2007699914000,
    "IoMax"             : 20,
    "IoMaxSpec"         : "alikafka.hw.2xlarge",
    "PaidType"          : 1,
    "ServiceStatus"     : 0,
    "TopicNumLimit"     : 1000,
    "UsedGroupCount"    : 0,
    "UsedTopicCount"    : 0,
  }
}

Feedback

Is this page helpful? ×