Alibaba Cloud KafKa¶

Alibaba Cloud KafKa includes instance disk usage, instance and topic message production volume, message production frequency, message consumption volume, message consumption frequency, etc. These metrics reflect the reliability of Kafka in handling large-scale message transmission and real-time data streams.

Configuration¶

Install Func¶

It is recommended to activate the Guance integration - extension - hosted Func.

If you deploy Func yourself, refer to Self-deployed Func

Activation Script¶

Note: Please prepare the required Alibaba Cloud AK in advance (for simplicity, you can directly grant global read-only permission ReadOnlyAccess)

Hosted Activation Script¶

1. Log in to Guance console
2. Click on the 【Integration】 menu, select 【Cloud Account Management】
3. Click 【Add Cloud Account】, choose 【Alibaba Cloud】, fill in the required information on the interface; if cloud account information has been configured before, ignore this step
4. Click 【Test】, after a successful test click 【Save】. If the test fails, check whether the relevant configuration information is correct and retest
5. In the 【Cloud Account Management】 list, you can see the added cloud account. Click on the corresponding cloud account to enter the details page
6. Click the 【Integration】 button on the cloud account details page. Under the Not Installed list, find Alibaba Cloud KafKa, click the 【Install】 button, and follow the installation interface instructions to install it.

Manual Activation Script¶

1. Log in to the Func console, click 【Script Market】, enter the official script market, search for guance_aliyun_alikafka

2. After clicking 【Install】, input the corresponding parameters: Alibaba Cloud AK ID, AK Secret, and account name.

3. Click 【Deploy Startup Script】, the system will automatically create a Startup script set and configure the corresponding startup script automatically.

4. After activation, you can see the corresponding automatic trigger configuration in 「Management / Automatic Trigger Configuration」. Click 【Execute】 to run it immediately without waiting for the scheduled time. Wait a moment, then you can view the execution task records and corresponding logs.

We default collect some configurations, see the Metrics section for details.

Customize Cloud Object Metrics Configuration

Verification¶

1. In 「Management / Automatic Trigger Configuration」, confirm whether the corresponding tasks have the corresponding automatic trigger configurations. You can also view the corresponding task records and logs to check for any anomalies.
2. In Guance, under 「Infrastructure / Custom」, check if asset information exists.
3. In Guance, under 「Metrics」, check if there are corresponding monitoring data.

Metrics¶

After configuring Alibaba Cloud - Cloud Monitor, the default metric set is as follows. You can collect more metrics through configuration Alibaba Cloud Cloud Monitor Metric Details

Objects¶

The object data structure collected from Alibaba Cloud KafKa can be viewed in 「Infrastructure - Custom」

{
  "measurement": "aliyun_kafka",
  "tags": {
    "InstanceId"        : "alikafka_post-cn-zsk3cluq100d",
    "InstanceName"      : "alikafka_post-cn-zsk3cluq100d",
    "RegionId"          : "cn-hangzhou",
    "ResourceGroupId"   : "rg-acfmv3ro3xnfwaa",
    "SpecType"          : "normal",
    "cloud_provider"    : "aliyun",
    "name"              : "alikafka_post-cn-zsk3cluq100d"
  },
  "fields": {
    "AllConfig"         : "{}",
    "CreateTime"        : 1692080715000,
    "DeployType"        : 5,
    "EipMax"            : 0,
    "ExpiredTime"       : 2007699914000,
    "IoMax"             : 20,
    "IoMaxSpec"         : "alikafka.hw.2xlarge",
    "PaidType"          : 1,
    "ServiceStatus"     : 0,
    "TopicNumLimit"     : 1000,
    "UsedGroupCount"    : 0,
    "UsedTopicCount"    : 0,
  }
}