Tencent Cloud Elasticsearch Service
Collect Tencent Cloud Elasticsearch Service Metrics
Configuration¶
Install Func¶
It is recommended to enable Guance Integration - Extensions - DataFlux Func (Automata): All prerequisites are automatically installed, please proceed with the script installation.
If you need to deploy Func manually, refer to Deploy Func Manually
Enable Script¶
Note: Please prepare the required Tencent Cloud AK in advance (for simplicity, you can directly grant the global read-only permission
ReadOnlyAccess)
Enable Script for DataFlux Func (Automata)¶
- Log in to the Guance console
- Click the 【Integration】 menu, select 【Cloud Account Management】
- Click 【Add Cloud Account】, select 【Tencent Cloud】, and fill in the required information on the interface. If you have already configured the cloud account information before, you can skip this step
- Click 【Test】, if the test is successful, click 【Save】. If the test fails, please check if the relevant configuration information is correct and retest
- Click 【Cloud Account Management】, you can see the added cloud account in the list. Click the corresponding cloud account to enter the details page
- Click the 【Integration】 button on the cloud account details page, find
Tencent Cloud Elasticsearch Serviceunder theNot Installedlist, and click the 【Install】 button. The installation interface will pop up for installation.
Enable Script Manually¶
-
Log in to the Func console, click 【Script Market】, enter the Guance script market, search for
integration_tencentcloud_es -
Click 【Install】, then enter the corresponding parameters: Tencent Cloud AK, SK, and account name
-
Click 【Deploy Startup Script】, the system will automatically create the
Startupscript set and configure the corresponding startup script -
After enabling, you can see the corresponding automatic trigger configuration in 「Management / Automatic Trigger Configuration」. Click 【Execute】 to execute it immediately without waiting for the scheduled time. After a while, you can check the execution task records and corresponding logs
Verification¶
- In 「Management / Automatic Trigger Configuration」, confirm whether the corresponding task has the corresponding automatic trigger configuration, and you can also check the corresponding task records and logs to check for any exceptions
- In Guance, check if the asset information exists in 「Infrastructure - Resource Catalog」
- In Guance, check if there is corresponding monitoring data in 「Metrics」
Metrics¶
After configuring Tencent Cloud Cloud Monitor, the default Measurement is as follows. You can collect more Metrics through configuration Tencent Cloud Cloud Monitor Metrics Details
Tencent Cloud ES Metrics are under the tencentcloud_QCE/CES Measurement. The following are the relevant Metrics descriptions:
| Metric Name | Metric Chinese Name | Metric Description | Unit | Statistical Granularity |
|---|---|---|---|---|
IsReadOnly |
Cluster Read-Only Status | Whether the cluster is read-only | None | 5s, 10s, 60s, 300s, 3600s, 86400s |
NodeBulkSpeed |
Node Single Cycle Bulk Speed | Node Single Cycle Bulk Speed | Count/s | 5s, 10s, 60s, 300s, 3600s, 86400s |
NodeCpuLoad1min |
Node CPU 1 Minute Load | Node CPU 1 Minute Load | None | 5s, 10s, 60s, 300s, 3600s, 86400s |
NodeIndexLatency |
Node Single Cycle Write Average Latency | Node Single Cycle Write Average Latency | ms | 5s, 10s, 60s, 300s, 3600s, 86400s |
NodeIndexSpeed |
Node Write Speed | Node Write Speed | Count/s | 5s, 10s, 60s, 300s, 3600s, 86400s |
NodeJvmMemUsage |
Node JVM Memory Usage | Node JVM Memory Usage | % | 5s, 10s, 60s, 300s, 3600s, 86400s |
NodeJvmOldMemUsage |
JVM_Old Area Memory Usage | JVM_Old Area Memory Usage | % | 5s, 10s, 60s, 300s, 3600s, 86400s |
NodeMemUsage |
Node Memory Usage | Node Memory Usage | % | 5s, 10s, 60s, 300s, 3600s, 86400s |
NodeOldGcDif |
Node Single Cycle OldGC Count | Node Single Cycle OldGC Count | Count | 5s, 10s, 60s, 300s, 3600s, 86400s |
NodeSearchLatency |
Node Single Cycle Query Average Latency | Node Single Cycle Query Average Latency | ms | 5s, 10s, 60s, 300s, 3600s, 86400s |
NodeStatus |
Node Health Status | Node Health Status | None | 5s, 10s, 60s, 300s, 3600s, 86400s |
ShardNum |
Cluster Shard Count | Cluster Shard Count | Count | 5s, 10s, 60s, 300s, 3600s, 86400s |
Status |
Cluster Health Status | Cluster Health Status: 0: Green 1: Yellow 2: Red | % | 5s, 10s, 60s, 300s, 3600s, 86400s |
ClusterSlowSearchCount1min |
Cluster 1min Slow Query Count | Cluster 1min Slow Query Count | Count | 5s, 10s, 60s, 300s, 3600s, 86400s |
ForceMergeQueue |
forceMerge Thread Queue Count | forceMerge Thread Queue Count | Count | 5s, 10s, 60s, 300s, 3600s, 86400s |
NodeCpuLoad1minRate |
Node CPU 1 Minute Load Rate | Node CPU 1 Minute Load Rate | % | 5s, 10s, 60s, 300s, 3600s, 86400s |
QueryCacheHitRate |
Query Cache Hit Rate | Query Cache Hit Rate | % | 5s, 10s, 60s, 300s, 3600s, 86400s |
ShardNumLimitPercen |
Cluster Shard Usage Rate | Cluster Shard Usage Rate | % | 5s, 10s, 60s, 300s, 3600s, 86400s |
DiskAwait |
Operation Wait Time | Operation Wait Time | ms | 5s, 10s, 60s, 300s, 3600s, 86400s |
DiskIoutil |
Disk IO Util | Disk IO Util | % | 5s, 10s, 60s, 300s, 3600s, 86400s |
DiskIps |
Writes Per Second | Writes Per Second | None | 5s, 10s, 60s, 300s, 3600s, 86400s |
DiskOps |
Reads Per Second | Reads Per Second | None | 5s, 10s, 60s, 300s, 3600s, 86400s |
DiskReadTraffic |
Disk Read Traffic | Disk Read Traffic | KBytes/s | 5s, 10s, 60s, 300s, 3600s, 86400s |
DiskWriteTraffic |
Disk Write Traffic | Disk Write Traffic | KBytes/s | 5s, 10s, 60s, 300s, 3600s, 86400s |
MaxP90SearchTaskMillisMax |
Query Task Duration 90th Percentile | Query Task Duration 90th Percentile | ms | 5s, 10s, 60s, 300s, 3600s, 86400s |
MaxP95SearchTaskMillis |
Query Task Duration 95th Percentile | Query Task Duration 95th Percentile | ms | 5s, 10s, 60s, 300s, 3600s, 86400s |
MaxP99SearchTaskMillis |
Query Task Duration 99th Percentile | Query Task Duration 99th Percentile | ms | 5s, 10s, 60s, 300s, 3600s, 86400s |
NodeDiskPathMaxUsage |
Node Disk Maximum Usage Rate | Node Disk Maximum Usage Rate | % | 5s, 10s, 60s, 300s, 3600s, 86400s |
ClusterHttpCurrentOpen |
Cluster HTTP Connection Count | Cluster HTTP Connection Count | ms | 5s, 10s, 60s, 300s, 3600s, 86400s |
Objects¶
The collected Tencent Cloud ES object data structure can be seen in 「Infrastructure - Resource Catalog」
{
"measurement": "tencentcloud_es",
"tags": {
"RegionId" : "ap-shanghai",
"InstanceId" : "ap-shanghai-1",
"InstanceName" : "xxxx",
"InstanceType" : "BIND",
"AppId" : "119.45.172.xxx",
"EsVip" : "xxxxx ",
"EsPort" : "xxxx",
"VpcUid" : "xxxxxxxxxx"
},
"fields": {
"EsPrivateUrl" : "xxxxxxxxxxx",
"EsPublicUrl" : "xxxxxxxxxxx",
"Status" : "xxxxxxxxxxx",
"NodeInfoList" : "mmxxxx",
"CpuNum" : "8xxxxx",
"MemSize" : "xxxx",
"MasterNodeInfo" : "xxxxxxxxxxx",
"KibanaNodeInfo" : false,
"OptionalWebServiceInfos" : false,
"ChargeType" : 1,
"CreateTime" : "2024-06-11T08:10:57Z",
"TagList" : "{Instance JSON Data}"
}
}
Note: The fields in
tagsandfieldsmay change with subsequent updates