Volcengine ESCloud Cloud Search Service
Collect metrics data from the Volcengine ESCloud cloud search service
Configuration¶
Install Func¶
It is recommended to enable Guance Integration - Extension - Managed Func: all prerequisites will be automatically installed, please continue with script installation.
If you deploy Func yourself, refer to Self-deployed Func
Install Script¶
Note: Please prepare a qualified Volcengine AK in advance (for simplicity, you can directly grant global read-only permission
ReadOnlyAccess
).
To synchronize monitoring data of ESCloud cloud resources, we install the corresponding collection script: 「Guance Integration (Volcengine - Cloud Search Service Collection)」(ID: guance_volcengine_es_cloud
)
After clicking 【Install】, enter the corresponding parameters: Volcengine AK, Volcengine account name, regions.
Click 【Deploy Startup Script】, and the system will automatically create a Startup
script set and configure the corresponding startup script automatically.
After enabling, you can see the corresponding automatic trigger configuration in "Manage / Automatic Trigger Configuration". Click 【Execute】 to run immediately without waiting for the scheduled time. Wait a moment, and you can view the execution task records and corresponding logs.
Verification¶
- In "Manage / Automatic Trigger Configuration", confirm whether the corresponding task has an automatic trigger configuration, and check the corresponding task records and logs for any anomalies.
- In Guance, under "Infrastructure - Resource Catalog", check if there are asset information entries.
- In Guance, under "Metrics", check if there are corresponding monitoring data.
Metrics¶
Configure Volcengine ESCloud monitoring metrics, more metrics can be collected via configuration Volcengine Cloud Search Service Metrics Details
Volcengine ESCloud Cloud Search Service Monitoring Metrics¶
Volcengine ESCloud metrics are located under the volcengine_VCM_ESCloud measurement set. Here is an introduction to related metric descriptions.
MetricName |
Subnamespace |
Metric Name | MetricUnit | Dimension |
---|---|---|---|---|
ActivePrimaryShardNumber |
cluster |
Primary Shard Number | Count | ResourceID |
ActiveShardNumber |
cluster |
Active Shard Number | Count | ResourceID |
AvgDiskUsagePercent |
total |
Average Cluster Disk Usage | Percent | ResourceID |
AvgJvmUsage |
total |
Average Cluster JVM Usage | Percent | ResourceID |
DeletedDocumentRate |
operation |
Delete Document Frequency | None | ResourceID |
DiskUsagePercent |
resource |
Node Disk Usage | Percent | ResourceID,Node |
FielddataMemorySize |
cluster |
Fielddata Memory Usage | Bytes | ResourceID,Node |
FsIoTotalReadBytes |
resource |
Node Disk Read Bandwidth | Bytes/Second | ResourceID,Node |
FsIoTotalWriteBytes |
resource |
Node Disk Write Bandwidth | Bytes/Second | ResourceID,Node |
GreenIndexNumber |
cluster |
Green Status Index Number | Count | ResourceID |
IndexOps |
operation |
Index Operation OPS | None | ResourceID |
InitializingShardNumber |
cluster |
Initializing Shard Number | Count | ResourceID |
InstanceCapacityUsage |
total |
Instance Used Capacity | Percent | ResourceID |
InstanceDataNodeNumber |
total |
Instance Data Node Number | Count | ResourceID |
InstanceHealthState |
total |
Cluster State | None | ResourceID |
InstanceIngestNodeNumber |
total |
Instance Ingest Role Node Number | Count | ResourceID |
InstanceMasterNodeNumber |
total |
Instance Master Role Node Number | Count | ResourceID |
InstanceMasterOnlyNodeNumber |
total |
Instance Dedicated Master Node Number | Count | ResourceID |
InstanceStorageUsedBytes |
total |
Instance Storage Usage Size | Bytes | ResourceID |
JvmUsage |
resource |
Node JVM Usage | Percent | ResourceID,Node |
MaxCpuUsage |
total |
Maximum CPU Usage | Percent | ResourceID |
MaxDiskUsagePercent |
total |
Maximum Cluster Disk Usage | Percent | ResourceID |
MaxJvmUsage |
total |
Maximum Cluster JVM Usage | Percent | ResourceID |
MeanCpuUsage |
total |
Average CPU Usage | Percent | ResourceID |
MergeOps |
operation |
Merge Operation OPS | None | ResourceID |
NodeCpuUsage |
resource |
Node CPU Usage | Percent | ResourceID,Node |
NodeDiskReadIOPS |
resource |
Node Disk Read IOPS | Count/Second | ResourceID,Node |
NodeFsIops |
resource |
Node Data Directory IOPS | None | ResourceID,Node |
NodeHttpConnections |
resource |
Node HTTP Connections | Count | ResourceID,Node |
NodeWriteIOPS |
resource |
Node Disk Write IOPS | Count/Second | ResourceID,Node |
OldGcCountPerMin |
resource |
Old Generation GC Count Per Minute | None | ResourceID,Node |
QueryOps |
operation |
Query Operation OPS | None | ResourceID |
ReadOnlyIndexNumber |
cluster |
Read-Only Index Number | Count | ResourceID |
RedIndexNumber |
total |
Red Status Index Number | None | ResourceID |
RelocatingShardNumber |
cluster |
Relocating Shard Number | None | ResourceID |
ReplicaShardNumber |
cluster |
Replica Shard Number | Count | ResourceID |
SearchLatency |
operation |
Search Latency | Second | ResourceID |
SearchSuccessRatio |
operation |
Search Success Rate | Percent | ResourceID |
BulkSuccessRatio |
operation |
Bulk Success Rate | Percent | ResourceID |
ThreadPoolFlushRejected |
cluster |
Rejected Flush Task Count | Count | ResourceID,Node |
ThreadPoolForceMergeQueue |
cluster |
ForceMerge Task Queue Count | Count | ResourceID,Node |
ThreadPoolForceMergeRejected |
cluster |
Rejected ForceMerge Task Count | Count | ResourceID,Node |
TotalDocumentNumber |
total |
Total Document Count | Count | ResourceID |
TotalIndexNumber |
total |
Total Index Count | Count | ResourceID |
TotalPendingTasks |
cluster |
Total Pending Task Count | Count | ResourceID |
TotalShardNumber |
total |
Total Shard Count | Count | ResourceID |
UnsignedShardNumber |
cluster |
Unassigned Shard Count | Count | ResourceID |
YellowIndexNumber |
cluster |
Yellow Status Index Count | Count | ResourceID |
YoungGcCountPerMin |
resource |
Young Generation GC Count Per Minute | None | ResourceID,Node |
Objects¶
Collected Volcengine ESCloud object data structure, which can be seen in "Infrastructure - Resource Catalog"
{
"measurement": "volcengine_es_cloud",
"tags": {
"InstanceId" : "o-0089apbhxxx",
"RegionId" : "cn-guangzhou",
"Status" : "Running",
"ESEip" : "118.145.xxx.xxx",
"UserId" : "2102598885",
"ESEipId" : "eip-11vuua67wc3r440yrhczl0xxx",
"KibanaEipId" : "'\'",
"KibanaPublicDomain" : "'\'",
"CerebroPublicDomain" : "'\'",
"KibanaPrivateDomain" : "https://visual-o-xxxxxapbhpvmd.escloud.ivolces.com:5601",
"CerebroPrivateDomain" : "'\'",
"CerebroEnabled" : "True",
"EnableESPublicNetwork" : "True",
"EnableESPrivateNetwork" : "vpc-13fpvlahzboxsxxxxxxxxxx",
"EnableKibanaPublicNetwork" : "True",
"EnableKibanaPrivateNetwork" : "True",
"EnableESPrivateDomainPublic" : "False",
"EnableKibanaPrivateDomainPublic" : "False",
},
"fileds": {
"TotalNodes": "4",
"ChargeEnabled": "True",
"DeletionProtection": "True",
"CreateTime": "2025-05-06T09:39:50+08:00",
"ExpireDate": "'\'",
"KibanaConfig": "'\'",
"ResourceTags": "[]",
"SubInstances": "[]",
"TransferInfo": "{}",
"MaintenanceDay": "[\"MONDAY\", \"TUESDAY\", \"WEDNESDAY\", \"THURSDAY\", \"FRIDAY\", \"SATURDAY\", \"SUNDAY\"",
"ESPublicEndpoint": "https://elasticsearch-o-xxxxxxapbhpvmd.escloud.volces.com:9200",
"ESPrivateEndpoint": "https://elasticsearch-o-xxxxxapbhpvmd.escloud.ivolces.com:9200",
"ESPublicIpWhitelist": "127.0.0.1",
"ESPrivateIpWhitelist": "0.0.0.0/0",
"InstanceConfiguration": "{Json data}",
"KibanaPublicIpWhitelist": "'\'",
"KibanaPrivateIpWhitelist": "0.0.0.0/0"
}
}