Tencent Cloud MySQL¶
Use the script packages in the "Guance Cloud Sync" series in the Script Market to synchronize cloud monitoring and cloud asset data to Guance
Configuration¶
Install Func¶
It is recommended to activate the Guance Integration - Extensions - DataFlux Func (Automata): all prerequisites are automatically installed, please proceed with the script installation.
If you are deploying Func yourself, refer to Self-deploying Func
Install Script¶
Note: Please prepare the Tencent Cloud AK that meets the requirements in advance (for simplicity, you can directly grant the global read-only permission
ReadOnlyAccess)
To synchronize the monitoring data of Tencent Cloud MySQL, we install the corresponding collection script: (ID: integration_tencentcloud_mysql)
Click 【Install】, then enter the corresponding parameters: Tencent Cloud AK, Tencent Cloud account name.
Click 【Deploy Startup Script】, the system will automatically create the Startup script set and automatically configure the corresponding startup script.
In addition, you can see the corresponding automatic trigger configuration in "Management / Automatic Trigger Configuration". Click 【Execute】 to execute it immediately without waiting for the scheduled time. After a while, you can check the execution task records and corresponding logs.
Verification¶
- In "Management / Automatic Trigger Configuration", confirm whether the corresponding task has the corresponding automatic trigger configuration, and you can also check the corresponding task records and logs to see if there are any exceptions
- In Guance, check if the asset information exists in "Infrastructure / Custom"
- In Guance, check if there is corresponding monitoring data in "Metrics"
Metrics¶
After configuring Tencent Cloud Cloud Monitoring, the default measurement sets are as follows. More metrics can be collected through configuration Tencent Cloud Cloud Monitoring Metrics Details
Resource Monitoring¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
BytesReceived |
Internal Network Inbound Traffic | Bytes received per second | Bytes/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
BytesSent |
Internal Network Outbound Traffic | Bytes sent per second | Bytes/s | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
Capacity |
Disk Usage Space | Includes MySQL data directory and binlog、relaylog、undolog、errorlog、slowlog log space |
MB | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
CpuUseRate |
CPU Utilization | Allows overuse during idle time, CPU utilization may exceed 100% | % | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
IOPS |
IOPS | Input/output operations per second (or read/write operations) | Ops/s | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
MemoryUse |
Memory Usage | Allows overuse during idle time, actual memory usage may exceed the purchased specifications | MB | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
MemoryUseRate |
Memory Utilization | Allows overuse during idle time, memory utilization may exceed 100% | % | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
RealCapacity |
Data Usage Space | Only includes MySQL data directory, excluding binlog、relaylog、undolog、errorlog、slowlog log space |
MB | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
VolumeRate |
Disk Utilization | Disk usage space/instance purchased space | % | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、3600s、86400s |
Engine Monitoring (General)- MyISAM¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
KeyCacheHitRate |
myisam Cache Hit Rate | myisam engine cache hit rate | % | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
KeyCacheUseRate |
myisam Cache Usage Rate | myisam engine cache usage rate | % | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (General)- InnoDB¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
InnodbCacheHitRate |
innodb Cache Hit Rate | Innodb engine cache hit rate | % | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
InnodbCacheUseRate |
innodb Cache Usage Rate | Innodb engine cache usage rate | % | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbNumOpenFiles |
Current InnoDB Open Table Count | Innodb engine current open table count | Count | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbOsFileReads |
innodb Disk Read Count | Innodb engine disk file read count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbOsFileWrites |
innodb Disk Write Count | Innodb engine disk file write count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbOsFsyncs |
innodbfsyn Count | Innodb engine fsync function call count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
Engine Monitoring (General)- Connection¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
ConnectionUseRate |
Connection Usage Rate | Current open connections/maximum connections | % | InstanceId、InstanceType（Optional） | 5s、10s、60s、300s、3600s |
MaxConnections |
Maximum Connections | Maximum connections | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Qps |
Queries Per Second | Database SQL executions per second (including insert、select、update、delete、replace），QPS metric mainly reflects the actual processing capability of TencentDB instances | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
ThreadsConnected |
Current Connections | Current open connections count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Tps |
Transactions Per Second | Database transactions processed per second | Tps/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (General)- Access¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
ComDelete |
Delete Count | Delete count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
ComInsert |
Insert Count | Insert count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
ComReplace |
Replace Count | Replace count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、6400s |
ComUpdate |
Update Count | Update count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Queries |
Total Queries | All executed SQL statements, including set, show, etc. | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
QueryRate |
Query Rate | Queries per second QPS/recommended queries per second | % | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
SelectCount |
Select Count | Select count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
SelectScan |
Full Table Scan Count | Full table scan count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
SlowQueries |
Slow Query Count | Queries exceeding long_query_time seconds count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (General)- Table¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
CreatedTmpTables |
Memory Temporary Table Count | Memory temporary table count per second | Count/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
TableLocksWaited |
Table Lock Wait Count | Table lock wait count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
OpenedTables |
Opened Table Count | Engine opened table count per second | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
TableLocksImmediate |
Immediate Table Lock Release Count | Engine immediate table lock release count per second | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (Extended)- Tmp¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
CreatedTmpDiskTables |
Disk Temporary Table Count | Disk temporary table count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
CreatedTmpFiles |
Temporary File Count | Temporary file count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (Extended)- Key¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
KeyBlocksUnused |
Unused Key Cache Block Count | myisam engine unused key cache block count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
KeyBlocksUsed |
Used Key Cache Block Count | myisam engine used key cache block count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
KeyReadRequests |
Key Cache Read Count | myisam engine key cache read count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
KeyReads |
Disk Read Count | myisam engine disk read count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
KeyWriteRequests |
Key Cache Write Count | myisam engine key cache write count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
KeyWrites |
Disk Write Count | myisam engine disk write count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (Extended)- InnoDB Row¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
InnodbRowLockTimeAvg |
InnoDB Average Row Lock Time (ms) | Innodb engine row lock average duration | ms | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbRowLockWaits |
InnoDB Row Lock Wait Count | Innodb engine row lock wait count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbRowsDeleted |
InnoDB Row Delete Count | Innodb engine row delete count per second | Rows/s | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbRowsInserted |
InnoDB Row Insert Count | Innodb engine row insert count per second | Rows/s | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbRowsRead |
InnoDB Row Read Count | Innodb engine row read count per second | Rows/s | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbRowsUpdated |
InnoDB Row Update Count | Innodb engine row update count per second | Rows/s | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、86400s |
Engine Monitoring (Extended)- InnoDB Data¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
InnodbDataRead |
InnoDB Read Count | Innodb engine read data bytes per second | Bytes/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
InnodbDataReads |
InnoDB Total Read Count | Innodb engine read data count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
InnodbDataWrites |
InnoDB Total Write Count | Innodb engine write data count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
InnodbDataWritten |
InnoDB Write Count | Innodb engine write data bytes per second | Bytes/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (Extended)- Handler¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
HandlerCommit |
Internal Commit Count | Internal commit count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
HandlerReadRndNext |
Read Next Row Request Count | Read next row request count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
HandlerRollback |
Internal Rollback Count | Internal rollback count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (Extended)- Buff¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
InnodbBufferPoolPagesFree |
InnoDB Free Page Count | Innodb engine memory free page count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s |
InnodbBufferPoolPagesTotal |
InnoDB Total Page Count | Innodb engine memory total page count | Count | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbBufferPoolReadRequests |
Innodb Buffer Pool Read Request Count | Innodb engine logical read request count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
InnodbBufferPoolReads |
Innodb Disk Read Count | Innodb engine physical read request count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、 60s、 300s、 3600s、 86400s |
Engine Monitoring (Extended)- Other¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
LogCapacity |
Log Usage | Engine log usage count | MB | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s |
OpenFiles |
Open File Count | Engine open file count per second | Count/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600、86400s |
Engine Monitoring (Extended)- Connection¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
ThreadsCreated |
Created Thread Count | Thread count created to handle connections | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
ThreadsRunning |
Running Thread Count | Active (non-sleeping) thread count | Count | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Engine Monitoring (Extended)- Access¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
ComCommit |
Commit Count | Commit count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
ComRollback |
Rollback Count | Rollback count per second | Ops/s | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Database Proxy (Proxy)¶
| Metric Name | Metric Chinese Name | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|
ProxyCpuUseRate |
CPU Utilization | % | InstanceId、InstanceType | 5s、60s、300s、3600s、86400s |
ProxyCurrentConnections |
Current Connection Count | Count | InstanceId、InstanceType | 5s、 60s、 300s、 3600s、 86400s |
ProxyMemoryUseRate |
Memory Utilization | % | InstanceId、InstanceType | 5s、 60s、 300s、 3600s、 86400s |
ProxyQueries |
Request Count | Count | InstanceId、InstanceType | 5s、 60s、 300s、 3600s、 86400s |
ProxyRouteMaster |
Write Request Count | Count | InstanceId、InstanceType | 5s、 60s、 300s、 3600s、 86400s |
ProxyRouteSlave |
Read Request Count | Count | InstanceId、InstanceType | 5s、 60s、 300s、 3600s、 86400s |
ProxyMemoryUse |
Memory Usage | MB | InstanceId、InstanceType | 5s、 60s、 300s、 3600s、 86400s |
Deployment Monitoring (Slave)¶
| Metric Name | Metric Chinese Name | Metric Description | Unit | Dimensions | Statistical Granularity |
|---|---|---|---|---|---|
MasterSlaveSyncDistance |
Master-Slave Sync Distance | Master-slave binlog gap | MB | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
SecondsBehindMaster |
Master-Slave Sync Time | Master-slave sync time | Seconds | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
SlaveIoRunning |
IO Thread Status | IO thread running status | Status value（0-Yes，1-No，2-Connecting） | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
SlaveSqlRunning |
SQL Thread Status | SQL thread running status | Status value（0-Yes，1-No） | InstanceId、InstanceType（Optional） | 5s、60s、300s、3600s、86400s |
Objects¶
The object data structure of Tencent Cloud MySQL collected can be seen in "Infrastructure - Custom"
{
"measurement": "tencentcloud_cdb",
"tags": {
"name" : "cdb-xxxxxxx",
"RegionId" : "ap-shanghai",
"Region" : "ap-shanghai",
"InstanceId" : "cdb-imxxxx",
"InstanceName" : "smart_check_test",
"InstanceType" : "1",
"Zone" : "ap-shanghai-3",
"ZoneName" : "",
"DeviceType" : "UNIVERSAL",
"EngineVersion": "8.0",
"Vip" : "172.xx.x.9",
"Status" : "1",
"ProtectMode" : "0",
"ProjectId" : "0",
"PayType" : "1",
"WanStatus" : "0"
},
"fields": {
"WanPort" : 0,
"Memory" : 1000,
"Volume" : 25,
"DeadlineTime": "0000-00-00 00:00:00",
"CreateTime" : "2022-04-27 15:18:06",
"message" : "{Instance JSON Data}"
}
}
Logs¶
Slow Query Statistics¶
Prerequisites¶
Note 1: Before using this collector, you must install the "Guance Integration Core Package" and its accompanying third-party dependencies
Note 2: The code execution of this script depends on the MySQL instance object collection. If the custom object collection of MySQL is not configured, the slow log script cannot collect slow log data
Install Script¶
On top of the previous installation, you need to install a corresponding MySQL Slow Query Statistics Log Collection Script
In "Management / Script Market", click and install the corresponding script package:
- ID:
integration_tencentcloud_mysql_slowlog
After the data is synchronized normally, you can view the data in the "Logs" section of Guance.
The reported data example is as follows:
{
"measurement": "tencentcloud_cdb_slow_log",
"tags": {
"name" : "cdb-llxxxxx",
"Database" : "test",
"UserHost" : "9.xxx.xxx.122",
"UserName" : "root",
"InstanceId" : "cdb-lxxxxtk8",
"DeviceType" : "UNIVERSAL",
"EngineVersion" : "8.0",
"InstanceName" : "cdbxxxxx",
"InstanceType" : "1",
"PayType" : "1",
"ProjectId" : "0",
"ProtectMode" : "0",
"Region" : "ap-shanghai",
"RegionId" : "ap-shanghai",
"Status" : "1",
"Vip" : "172.xx.xxx.15",
"WanStatus" : "0",
"Zone" : "ap-shanghai-3",
"account_name" : "Script Development Tencent Account",
"cloud_provider": "tencentcloud"
},
"fields": {
"QueryTime" : 3.000195,
"SqlText" : "select sleep(3)",
"Timestamp" : 1652933796,
"LockTime" : 0,
"RowsExamined": 1,
"RowsSent" : 1,
"SqlTemplate" : "select sleep(?);",
"Md5" : "26A15F1AE530F28F",
"message" : "{Instance JSON Data}"
}
}
Partial parameter descriptions are as follows
| Field | Type | Description |
|---|---|---|
QueryTime |
float | SQL execution duration (seconds) |
Timestamp |
integer | SQL execution time |
Md5 |
str | SQL statement MD5 |
LockTime |
float | Lock duration (seconds) |
RowsExamined |
integer | Scanned rows |
RowsSent |
integer | Result set rows |
Note: The fields in
tagsandfieldsmay change with subsequent updatesNote 1:
tags.namevalue is the instance ID, used as a unique identifierNote 2:
fields.messageis a JSON serialized string