跳转至

PostgreSQL

·


PostgreSQL 采集器可以从 PostgreSQL 实例中采集实例运行状态指标,并将指标采集到观测云,帮助监控分析 PostgreSQL 各种异常情况。

配置

前置条件

  • PostgreSQL 版本 >= 9.0
  • 创建监控帐号

    -- PostgreSQL >= 10
    create user datakit with password '<PASSWORD>';
    grant pg_monitor to datakit;
    grant SELECT ON pg_stat_database to datakit;
    
    -- PostgreSQL < 10
    create user datakit with password '<PASSWORD>';
    grant SELECT ON pg_stat_database to datakit;
    
  • 开启 pg_stat_statements 扩展(可选)

    PostgreSQL 对象 采集时,部分指标如 qps/tps/avg_query_time 等,需要开启 pg_stat_statements 扩展。具体步骤如下:

    • 修改配置文件

      找到并编辑 PostgreSQL 配置文件(通常位于 /var/lib/pgsql/data/postgresql.conf/etc/postgresql/<版本>/main/postgresql.conf):

      # 启用 pg_stat_statements 扩展
      shared_preload_libraries = 'pg_stat_statements'
      
      # 可选配置
      pg_stat_statements.track = 'all'  # 收集所有 SQL 语句
      pg_stat_statements.max = 10000  # 最多收集的 SQL 语句数量
      pg_stat_statements.track_utility = off # 忽略 utility 语句,只跟踪常规 SQL 查询如 SELECT、INSERT、UPDATE、DELETE
      
    • 重启 PostgreSQL 服务

      修改好配置文件后,需要重启 PostgreSQL 服务。

    • 数据库中创建扩展

      连接到目标数据库, 执行以下 SQL:

      CREATE EXTENSION pg_stat_statements;
      
    • 验证扩展是否成功开启

      SELECT * FROM pg_extension WHERE extname = 'pg_stat_statements';
      SELECT * FROM pg_stat_statements LIMIT 10;
      

采集器配置

进入 DataKit 安装目录下的 conf.d/samples 目录,复制 postgresql.conf.sample 并命名为 postgresql.conf。示例如下:

[[inputs.postgresql]]
  ## Server address
  # URI format
  # postgres://[datakit[:PASSWORD]]@localhost[/dbname]?sslmode=[disable|verify-ca|verify-full]
  # or simple string
  # host=localhost user=pqgotest password=... sslmode=... dbname=app_production

  address = "postgres://datakit:PASSWORD@localhost/postgres?sslmode=disable"

  ## Ignore databases which are gathered. Do not use with 'databases' option.
  #
  # ignored_databases = ["db1"]

  ## Specify the list of the databases to be gathered. Do not use with the 'ignored_databases' option.
  #
  # databases = ["db1"]

  ## Specify the name used as the "server" tag.
  #
  # outputaddress = "db01"

  ## Collect interval
  # Time unit: "ns", "us", "ms", "s", "m", "h"
  #
  interval = "10s"

  ## Set true to enable election
  #
  election = true

  ## Metric name in metric_exclude_list will not be collected.
  #
  metric_exclude_list = [""]

  ## Set dbm to true to collect database activity 
  dbm = false

  ## Config dbm metric 
  [inputs.postgresql.dbm_metric]
    enabled = true

  ## Config dbm sample 
  [inputs.postgresql.dbm_sample]
    enabled = true  

  ## Config dbm activity
  [inputs.postgresql.dbm_activity]
    enabled = true  

  ## collect object
  [inputs.postgresql.object]
    # Set true to enable collecting objects
    enabled = true

    # interval to collect postgresql object which will be greater than collection interval
    interval = "600s"

    [inputs.postgresql.object.collect_schemas]
      # Set true to enable collecting schemas
      enabled = true

      # Maximum number of tables to collect
      max_tables = 300

      # Set true to enable auto discovery database
      auto_discovery_database = false

      # Maximum number of databases to collect
      max_database = 100

  ## Relations config
  #
  # The list of relations/tables can be specified to track per-relation metrics. 
  # To collect relation metrics, you need to specify the databases or database_regex field to indicate which databases to collect.
  # relation_name refer to the name of a relation, either relation_name or relation_regex must be set.
  # relation_regex is a regex rule, only takes effect when relation_name is not set.
  # schemas used for filtering, ignore this field when it is empty
  # relkind can be a list of the following options:
  #   r(ordinary table), i(index), S(sequence), t(TOAST table), p(partitioned table),
  #   m(materialized view), c(composite type), f(foreign table)
  # Size metrics are collected only for ordinary tables. Index metrics are collected only for user indexes.
  # Lock metrics are collected for all relation types. The rest of the metrics are collected only for user tables.
  #
  # [[inputs.postgresql.relations]]
  # databases = ["postgres"]
  # database_regex = "<DATABASE_PATTERN>"
  # relation_name = "<TABLE_NAME>"
  # relation_regex = "<TABLE_PATTERN>"
  # schemas = ["public"]
  # relkind = ["r", "p"]

  ## Run a custom SQL query and collect corresponding metrics.
  #
  # [[inputs.postgresql.custom_queries]]
  #   sql = '''
  #     select datname,numbackends,blks_read
  #     from pg_stat_database
  #     limit 10
  #   '''
  #   metric = "postgresql_custom_stat"
  #   tags = ["datname" ]
  #   fields = ["numbackends", "blks_read"]
  #   interval = "10s"

  ## Log collection
  #
  # [inputs.postgresql.log]
  # files = []
  # pipeline = "postgresql.p"

  ## Custom tags
  #
  [inputs.postgresql.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

配置好后,重启 DataKit 即可。

目前可以通过 ConfigMap 方式注入采集器配置来开启采集器。

数据库性能指标采集

Version-1.84.0

数据库性能指标主要来源于 PostgreSQL 的内置系统视图和扩展插件,其中最核心的包括 pg_stat_activitypg_stat_statements。这些工具提供了在运行时获取数据库内部执行情况的方法:pg_stat_activity 实时展示当前会话的活动状态、执行的查询及等待事件等信息;pg_stat_statements 则记录历史 SQL 语句的执行统计数据,包括执行次数、耗时、IO 情况等。

通过这些视图和插件,DataKit 能够采集实时会话活动、历史查询的性能指标统计以及相关执行信息。采集的性能指标数据保存为日志,source 分别为 postgresql_dbm_metricpostgresql_dbm_samplepostgresql_dbm_activity

如需开启,需要执行以下步骤。

  • 修改配置文件,开启监控采集
[[inputs.postgresql]]

  ## Set dbm to true to collect database activity 
  dbm = false

  ## Config dbm metric 
  [inputs.postgresql.dbm_metric]
    enabled = true

  ## Config dbm sample 
  [inputs.postgresql.dbm_sample]
    enabled = true  

  ## Config dbm activity
  [inputs.postgresql.dbm_activity]
    enabled = true  

...
  • PostgreSQL 配置

修改配置文件(如 postgresql.conf),配置相关参数:

shared_preload_libraries = 'pg_stat_statements'
track_activity_query_size = 4096 a# Required for collection of larger queries.
  • 权限配置

账号授权

ALTER ROLE datakit INHERIT;

在每个数据库中执行以下 SQL:

CREATE SCHEMA datakit;
GRANT USAGE ON SCHEMA datakit TO datakit;
GRANT USAGE ON SCHEMA public TO datakit;
GRANT pg_monitor TO datakit;
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

在每个数据库中执行以下 SQL:

CREATE SCHEMA datakit;
GRANT USAGE ON SCHEMA datakit TO datakit;
GRANT USAGE ON SCHEMA public TO datakit;
GRANT pg_monitor TO datakit;
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

在每个数据库中执行以下 SQL:

CREATE SCHEMA datakit;
GRANT USAGE ON SCHEMA datakit TO datakit;
GRANT USAGE ON SCHEMA public TO datakit;
GRANT SELECT ON pg_stat_database TO datakit;
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

CREATE OR REPLACE FUNCTION datakit.pg_stat_activity() RETURNS SETOF pg_stat_activity AS
  $$ SELECT * FROM pg_catalog.pg_stat_activity; $$
LANGUAGE sql
SECURITY DEFINER;
CREATE OR REPLACE FUNCTION datakit.pg_stat_statements() RETURNS SETOF pg_stat_statements AS
    $$ SELECT * FROM pg_stat_statements; $$
LANGUAGE sql
SECURITY DEFINER;

指标

以下所有数据采集,默认会追加全局选举 tag,也可以在配置中通过 [inputs.postgresql.tags] 指定其它标签:

postgresql

Tags & Fields Description
db
(tag)
The database name
server
(tag)
The address of the server. The value is host:port
active_time Time spent executing SQL statements in this database, in milliseconds.
Type: float | (count)
Unit: count
blks_hit The number of times disk blocks were found in the buffer cache, preventing the need to read from the database.
Type: int | (gauge)
Unit: count
blks_read The number of disk blocks read in this database.
Type: int | (gauge)
Unit: count
database_size The disk space used by this database.
Type: int | (gauge)
Unit: count
deadlocks The number of deadlocks detected in this database.
Type: int | (gauge)
Unit: count
idle_in_transaction_time Time spent idling while in a transaction in this database, in milliseconds.
Type: float | (count)
Unit: count
numbackends The number of active connections to this database.
Type: int | (gauge)
Unit: count
session_time Time spent by database sessions in this database, in milliseconds.
Type: float | (count)
Unit: count
sessions Total number of sessions established to this database.
Type: int | (count)
Unit: count
sessions_abandoned Number of database sessions to this database that were terminated because connection to the client was lost.
Type: int | (count)
Unit: count
sessions_fatal Number of database sessions to this database that were terminated by fatal errors.
Type: int | (count)
Unit: count
sessions_killed Number of database sessions to this database that were terminated by operator intervention.
Type: int | (count)
Unit: count
temp_bytes The amount of data written to temporary files by queries in this database.
Type: int | (gauge)
Unit: count
temp_files The number of temporary files created by queries in this database.
Type: int | (gauge)
Unit: count
tup_deleted The number of rows deleted by queries in this database.
Type: int | (gauge)
Unit: count
tup_fetched The number of rows fetched by queries in this database.
Type: int | (gauge)
Unit: count
tup_inserted The number of rows inserted by queries in this database.
Type: int | (gauge)
Unit: count
tup_returned The number of rows returned by queries in this database.
Type: int | (gauge)
Unit: count
tup_updated The number of rows updated by queries in this database.
Type: int | (gauge)
Unit: count
wraparound The number of transactions that can occur until a transaction wraparound.
Type: float | (gauge)
Unit: count
xact_commit The number of transactions that have been committed in this database.
Type: int | (gauge)
Unit: count
xact_rollback The number of transactions that have been rolled back in this database.
Type: int | (gauge)
Unit: count

postgresql_lock

Tags & Fields Description
db
(tag)
The database name
locktype
(tag)
The lock type
mode
(tag)
The lock mode
schema
(tag)
The schema name
server
(tag)
The address of the server. The value is host:port
table
(tag)
The table name
lock_count The number of locks active for this database.
Type: int | (gauge)
Unit: count

postgresql_index

Tags & Fields Description
db
(tag)
The database name
pg_index
(tag)
The index name
schema
(tag)
The schema name
server
(tag)
The address of the server. The value is host:port
table
(tag)
The table name
idx_scan The number of index scans initiated on this table, tagged by index.
Type: int | (gauge)
Unit: count
idx_tup_fetch The number of live rows fetched by index scans.
Type: int | (gauge)
Unit: count
idx_tup_read The number of index entries returned by scans on this index.
Type: int | (gauge)
Unit: count

postgresql_replication

Tags & Fields Description
db
(tag)
The database name
server
(tag)
The address of the server. The value is host:port
replication_delay The current replication delay in seconds. Only available with postgresql 9.1 and newer.
Type: int | (gauge)
Unit: time,s
replication_delay_bytes The current replication delay in bytes. Only available with postgresql 9.2 and newer.
Type: int | (gauge)
Unit: digital,B

postgresql_replication_slot

Tags & Fields Description
db
(tag)
The database name
server
(tag)
The address of the server. The value is host:port
slot_name
(tag)
The replication slot name
slot_type
(tag)
The replication slot type
spill_bytes Amount of decoded transaction data spilled to disk while performing decoding of changes from WAL for this slot. This and other spill counters can be used to gauge the I/O which occurred during logical decoding and allow tuning logical_decoding_work_mem. Only available with PostgreSQL 14 and newer.
Type: int | (gauge)
Unit: digital,B
spill_count Number of times transactions were spilled to disk while decoding changes from WAL for this slot. This counter is incremented each time a transaction is spilled, and the same transaction may be spilled multiple times. Only available with PostgreSQL 14 and newer.
Type: int | (gauge)
Unit: count
spill_txns Number of transactions spilled to disk once the memory used by logical decoding to decode changes from WAL has exceeded logical_decoding_work_mem. The counter gets incremented for both top-level transactions and subtransactions. Only available with PostgreSQL 14 and newer.
Type: int | (gauge)
Unit: count
stream_bytes Amount of transaction data decoded for streaming in-progress transactions to the decoding output plugin while decoding changes from WAL for this slot. This and other streaming counters for this slot can be used to tune logical_decoding_work_mem. Only available with PostgreSQL 14 and newer.
Type: int | (gauge)
Unit: digital,B
stream_count Number of times in-progress transactions were streamed to the decoding output plugin while decoding changes from WAL for this slot. This counter is incremented each time a transaction is streamed, and the same transaction may be streamed multiple times. Only available with PostgreSQL 14 and newer.
Type: int | (gauge)
Unit: count
stream_txns Number of in-progress transactions streamed to the decoding output plugin after the memory used by logical decoding to decode changes from WAL for this slot has exceeded logical_decoding_work_mem. Streaming only works with top-level transactions (subtransactions can't be streamed independently), so the counter is not incremented for subtransactions. Only available with PostgreSQL 14 and newer.
Type: int | (gauge)
Unit: count
total_bytes Amount of transaction data decoded for sending transactions to the decoding output plugin while decoding changes from WAL for this slot. Note that this includes data that is streamed and/or spilled. Only available with PostgreSQL 14 and newer.
Type: int | (gauge)
Unit: digital,B
total_txns Number of decoded transactions sent to the decoding output plugin for this slot. This counts top-level transactions only, and is not incremented for subtransactions. Note that this includes the transactions that are streamed and/or spilled. Only available with PostgreSQL 14 and newer.
Type: int | (gauge)
Unit: count

postgresql_size

Tags & Fields Description
db
(tag)
The database name
schema
(tag)
The schema name
server
(tag)
The address of the server. The value is host:port
table
(tag)
The table name
index_size The total disk space used by indexes attached to the specified table.
Type: int | (gauge)
Unit: digital,B
table_size The total disk space used by the specified table with TOAST data. Free space map and visibility map are not included.
Type: int | (gauge)
Unit: digital,B
total_size The total disk space used by the table, including indexes and TOAST data.
Type: int | (gauge)
Unit: digital,B

postgresql_statio

Tags & Fields Description
db
(tag)
The database name
schema
(tag)
The schema name
server
(tag)
The address of the server. The value is host:port
table
(tag)
The table name
heap_blks_hit The number of buffer hits in this table.
Type: int | (gauge)
Unit: count
heap_blks_read The number of disk blocks read from this table.
Type: int | (gauge)
Unit: count
idx_blks_hit The number of buffer hits in all indexes on this table.
Type: int | (gauge)
Unit: count
idx_blks_read The number of disk blocks read from all indexes on this table.
Type: int | (gauge)
Unit: count
tidx_blks_hit The number of buffer hits in this table's TOAST table index.
Type: int | (gauge)
Unit: count
tidx_blks_read The number of disk blocks read from this table's TOAST table index.
Type: int | (gauge)
Unit: count
toast_blks_hit The number of buffer hits in this table's TOAST table.
Type: int | (gauge)
Unit: count
toast_blks_read The number of disk blocks read from this table's TOAST table.
Type: int | (gauge)
Unit: count

postgresql_stat

Tags & Fields Description
db
(tag)
The database name
schema
(tag)
The schema name
server
(tag)
The address of the server. The value is host:port
table
(tag)
The table name
analyze_count The number of times this table has been manually analyzed.
Type: int | (gauge)
Unit: count
autoanalyze_count The number of times this table has been analyzed by the autovacuum daemon.
Type: int | (gauge)
Unit: count
autovacuum_count The number of times this table has been vacuumed by the autovacuum daemon.
Type: int | (gauge)
Unit: count
idx_scan The number of index scans initiated on this table, tagged by index.
Type: int | (gauge)
Unit: count
idx_tup_fetch The number of live rows fetched by index scans.
Type: int | (gauge)
Unit: count
n_dead_tup The estimated number of dead rows.
Type: int | (gauge)
Unit: count
n_live_tup The estimated number of live rows.
Type: int | (gauge)
Unit: count
n_tup_del The number of rows deleted by queries in this database.
Type: int | (gauge)
Unit: count
n_tup_hot_upd The number of rows HOT updated, meaning no separate index update was needed.
Type: int | (gauge)
Unit: count
n_tup_ins The number of rows inserted by queries in this database.
Type: int | (gauge)
Unit: count
n_tup_upd The number of rows updated by queries in this database.
Type: int | (gauge)
Unit: count
seq_scan The number of sequential scans initiated on this table.
Type: int | (gauge)
Unit: count
seq_tup_read The number of live rows fetched by sequential scans.
Type: int | (gauge)
Unit: count
vacuum_count The number of times this table has been manually vacuumed.
Type: int | (gauge)
Unit: count

postgresql_slru

Tags & Fields Description
name
(tag)
The name of the SLRU
server
(tag)
The address of the server. The value is host:port
blks_exists Number of blocks checked for existence for this SLRU (simple least-recently-used) cache.
Type: int | (gauge)
Unit: count
blks_hit Number of times disk blocks were found already in the SLRU (simple least-recently-used.)
Type: int | (gauge)
Unit: count
blks_read Number of disk blocks read for this SLRU (simple least-recently-used) cache. SLRU caches are created with a fixed number of pages.
Type: int | (gauge)
Unit: count
blks_written Number of disk blocks written for this SLRU (simple least-recently-used) cache.
Type: int | (gauge)
Unit: count
blks_zeroed Number of blocks zeroed during initializations of SLRU (simple least-recently-used) cache.
Type: int | (gauge)
Unit: count
flushes Number of flush of dirty data for this SLRU (simple least-recently-used) cache.
Type: int | (gauge)
Unit: count
truncates Number of truncates for this SLRU (simple least-recently-used) cache.
Type: int | (gauge)
Unit: count

postgresql_bgwriter

Tags & Fields Description
server
(tag)
The address of the server. The value is host:port
buffers_alloc The number of buffers allocated
Type: int | (count)
Unit: count
buffers_backend The number of buffers written directly by a backend.
Type: int | (count)
Unit: count
buffers_backend_fsync The of times a backend had to execute its own fsync call instead of the background writer.
Type: int | (count)
Unit: count
buffers_checkpoint The number of buffers written during checkpoints.
Type: int | (count)
Unit: count
buffers_clean The number of buffers written by the background writer.
Type: int | (count)
Unit: count
checkpoint_sync_time The total amount of checkpoint processing time spent synchronizing files to disk.
Type: float | (count)
Unit: time,ms
checkpoint_write_time The total amount of checkpoint processing time spent writing files to disk.
Type: float | (count)
Unit: time,ms
checkpoints_req The number of requested checkpoints that were performed.
Type: int | (count)
Unit: count
checkpoints_timed The number of scheduled checkpoints that were performed.
Type: int | (count)
Unit: count
maxwritten_clean The number of times the background writer stopped a cleaning scan due to writing too many buffers.
Type: int | (count)
Unit: count

postgresql_connection

Tags & Fields Description
server
(tag)
The address of the server. The value is host:port
max_connections The maximum number of client connections allowed to this database.
Type: float | (gauge)
Unit: count
percent_usage_connections The number of connections to this database as a fraction of the maximum number of allowed connections.
Type: float | (gauge)
Unit: count

postgresql_conflict

Tags & Fields Description
db
(tag)
The database name
server
(tag)
The address of the server. The value is host:port
confl_bufferpin Number of queries in this database that have been canceled due to pinned buffers.
Type: int | (count)
Unit: count
confl_deadlock Number of queries in this database that have been canceled due to deadlocks.
Type: int | (count)
Unit: count
confl_lock Number of queries in this database that have been canceled due to dropped tablespaces. This will occur when a temp_tablespace is dropped while being used on a standby.
Type: int | (count)
Unit: count
confl_snapshot Number of queries in this database that have been canceled due to old snapshots.
Type: int | (count)
Unit: count
confl_tablespace Number of queries in this database that have been canceled due to dropped tablespaces. This will occur when a temp_tablespace is dropped while being used on a standby.
Type: int | (count)
Unit: count

postgresql_archiver

Tags & Fields Description
server
(tag)
The address of the server. The value is host:port
archived_count Number of WAL files that have been successfully archived.
Type: int | (count)
Unit: count
archived_failed_count Number of failed attempts for archiving WAL files.
Type: int | (count)
Unit: count

collector

Tags & Fields Description
instance
(tag)
Server addr of the instance
job
(tag)
Server name of the instance
up
Type: int | (gauge)
Unit: -

对象

database

PostgreSQL object metrics( Version-1.76.0)

Tags & Fields Description
database_type
(tag)
The type of the database. The value is PostgreSQL
host
(tag)
The hostname of the PostgreSQL server
name
(tag)
The name of the database. The value is host:port in default
port
(tag)
The port of the PostgreSQL server
server
(tag)
The server address of the PostgreSQL server. The value is host:port
version
(tag)
The version of the PostgreSQL server
avg_query_time The average time taken by a query to execute
Type: float
Unit: timeStamp,usec
message Summary of database information
Type: string
Unit: -
qps The number of queries executed by the database per second
Type: float
Unit: gauge
slow_queries The number of queries that have taken more than long_query_time seconds. This counter increments regardless of whether the slow query log is enabled.
Type: int
Unit: count
slow_query_log Whether the slow query log is enabled. The value can OFF to disable the log or ON to enable the log.
Type: string
Unit: -
tps The number of transactions executed by the database per second
Type: float
Unit: gauge
uptime The number of seconds that the server has been up
Type: int
Unit: time,s

message 指标字段结构

message 字段基本结构如下:

{
  "setting": {
    "DateStyle":"ISO, MDY",
    ...
  },

  "databases": [ # databases information
    {
      "name": "db1",
      "encoding": "utf8",
      "owner": "datakit",
      "schemas": [ # schemas information
        {
          "name": "schema1",
          "owner": "datakit",
          "tables": [ # tables information
            {
              "name": "table1",
              "columns": [], # columns information
              "indexes": [], # indexes information
              "foreign_keys": [], # foreign keys information
            }
            ...
          ]
        },
        ...
      ]
    }
    ...
  ]
}

setting

setting 字段中的数据来源于 pg_settings 系统视图,用于展示当前数据库系统的配置参数信息,详细信息可以参考 PostgreSQL 文档

databases

databases 字段保存了 PostgreSQL 服务器上所有数据库的信息,每个数据库的信息如下:

字段名 描述 类型
name 数据库名称 string
encoding 数据库编码 string
owner 角色名称 string
description 描述文本 string
schemas 包含 schema 信息的列表 list

schemas 字段包含了数据库中的所有 schema 信息,每个 schema 的信息如下:

字段名 描述 类型
name schema 名称 string
owner 角色名称 string
tables 包含 table 信息的列表 list

tables 字段包含了数据库中所有表的信息,每个表的信息如下:

字段名 描述 类型
name 表名称 string
owner 角色名称 string
has_indexes 是否有索引 bool
has_partitions 是否有分区 bool
toast_table toast 表名称 string
partition_key 分区键 string
num_partitions 分区数量 int64
foreign_keys 包含外键信息的列表 list
columns 包含列信息的列表 list
indexes 包含索引信息的列表 list
  • tables.columns

columns 字段包含了数据库中表的所有列的信息,每个列的信息如下:

字段名 描述 类型
name column 名称 string
data_type 数据类型 string
nullable 是否可以为空 bool
default 默认值 string
  • tables.indexes

indexes 字段包含了数据库中表的所有索引的信息,每个索引的信息如下:

字段名 描述 类型
name index 名称 string
columns 索引包含的列 list
index_type 索引类型 string
definition 索引定义 string
is_unique 是否唯一 bool
is_primary 是否主键 bool
is_exclusion 是否为排除约束索引 bool
is_immediate 每条语句执行结束后是否立即检查约束 bool
is_valid 是否有效 bool
is_clustered 是否为聚簇索引 bool
is_checkxmin 是否检查 xmin bool
is_ready 是否就绪 bool
is_live 是否活跃 bool
is_replident 是否是行标识索引 bool
is_partial 是否是部分索引 bool

索引列信息字段 indexes.columns 包含了索引中包含的列的信息,每个列的信息如下:

字段名 描述 类型
name 列名称 string
  • tables.foreign_keys

foreign_keys 字段包含了数据库中表的所有外键的信息,每个外键的信息如下:

字段名 描述 类型
name foreign_key 名称 string
definition 外键定义 string
constraint_schema 外键所属的数据库(通常与表所在数据库一致) string
column_names 外键列名称(多个列用逗号分隔,如 user_id, order_id) string
referenced_table_schema 引用表所在的数据库 string
referenced_table_name 引用表名称 string
referenced_column_names 引用列名称(多个列用逗号分隔) string
update_action 级联更新规则(如 CASCADE, RESTRICT) string
delete_action 级联删除规则(如 CASCADE, SET NULL) string

日志

postgresql_dbm_metric

PostgreSQL database statement execution performance metrics, collected from the pg_stat_statements extension. Provides detailed statistics on query performance and resource usage.( Version-1.84.0)

Tags & Fields Description
db
(tag)
The database name
message
(tag)
Text of a normalized statement.
query_signature
(tag)
The hash value computed from query
server
(tag)
The server address
status
(tag)
The status of the statement. The value is only info for now.
blk_read_time Total time the statement spent reading data file blocks, in milliseconds (if track_io_timing is enabled, otherwise zero). Postgres < 17
Type: float | (gauge)
Unit: timeStamp,msec
blk_write_time Total time the statement spent writing data file blocks, in milliseconds (if track_io_timing is enabled, otherwise zero). Postgres < 17
Type: float | (gauge)
Unit: timeStamp,msec
calls Number of times the statement was executed.
Type: int | (count)
Unit: count
local_blks_dirtied Total number of local blocks dirtied by the statement.
Type: int | (count)
Unit: count
local_blks_hit Total number of local block cache hits by the statement.
Type: int | (count)
Unit: count
local_blks_read Total number of local blocks read by the statement.
Type: int | (count)
Unit: count
local_blks_written Total number of local blocks written by the statement.
Type: int | (count)
Unit: count
max_plan_time Maximum time spent planning the statement, in milliseconds (if pg_stat_statements.track_planning is enabled, otherwise zero).
Type: int | (gauge)
Unit: timeStamp,msec
min_plan_time Minimum time spent planning the statement, in milliseconds (if pg_stat_statements.track_planning is enabled, otherwise zero).
Type: float | (gauge)
Unit: timeStamp,msec
rows Total number of rows retrieved or affected by the statement.
Type: int | (count)
Unit: count
shared_blk_read_time Total time the statement spent reading shared blocks, in milliseconds (if track_io_timing is enabled, otherwise zero). Postgres >= 17
Type: float | (gauge)
Unit: timeStamp,msec
shared_blk_write_time Total time the statement spent writing shared blocks, in milliseconds (if track_io_timing is enabled, otherwise zero). Postgres >= 17
Type: float | (gauge)
Unit: timeStamp,msec
shared_blks_dirtied Total number of shared blocks dirtied by the statement.
Type: int | (count)
Unit: count
shared_blks_hit Total number of shared block cache hits by the statement.
Type: int | (count)
Unit: count
shared_blks_read Total number of shared blocks read by the statement.
Type: int | (count)
Unit: count
shared_blks_written Total number of shared blocks written by the statement.
Type: int | (count)
Unit: count
temp_blks_read Total number of temp blocks read by the statement.
Type: int | (count)
Unit: count
temp_blks_written Total number of temp blocks written by the statement.
Type: int | (count)
Unit: count
total_exec_time Total time spent executing the statement, in milliseconds.
Type: float | (gauge)
Unit: timeStamp,msec
total_plan_time Total time spent planning the statement, in milliseconds (if pg_stat_statements.track_planning is enabled, otherwise zero).
Type: float | (gauge)
Unit: timeStamp,msec
wal_bytes Total amount of WAL generated by the statement in bytes.
Type: int | (gauge)
Unit: digital,B
wal_fpi Total number of WAL full page images generated by the statement.
Type: int | (count)
Unit: count
wal_records Total number of WAL records generated by the statement.
Type: int | (count)
Unit: count

postgresql_dbm_sample

PostgreSQL database statement sample information, including statement text, execution plans, used for detailed analysis of SQL execution and performance issues.( Version-1.84.0)

Tags & Fields Description
application_name
(tag)
Name of the application that is connected to this backend.
client_addr
(tag)
IP address of the client that is connected to this backend.
client_hostname
(tag)
Host name of the connected client, as reported by a reverse DNS lookup of client_addr. This field will only be non-null for IP connections, and only when log_hostname is enabled.
client_port
(tag)
TCP port number that the client is using for communication, or -1 if a Unix socket is used.
message
(tag)
Text of a normalized statement.
plan_definition
(tag)
The plan definition of the statement.
query_signature
(tag)
The hash value computed from query
server
(tag)
The server address
status
(tag)
The status of the statement. The value is only info for now.
usename
(tag)
Name of the user logged into this backend.

postgresql_dbm_activity

PostgreSQL database activity information collected from the pg_stat_activity view. Provides detailed information about the current activity of each backend, including query status, execution time, and resource usage.( Version-1.84.0)

Tags & Fields Description
application_name
(tag)
Name of the application that is connected to this backend.
backend_type
(tag)
Type of current backend.
client_addr
(tag)
IP address of the client that is connected to this backend. If this field is null, it indicates either that the client is connected via a Unix socket or that this is an internal process (such as autovacuum worker).
client_hostname
(tag)
Host name of the connected client, as reported by a reverse DNS lookup of client_addr. This field will only be non-null for IP connections, and only when log_hostname is enabled.
client_port
(tag)
TCP port number that the client is using for communication, or -1 if a Unix socket is used.
datname
(tag)
Name of the database this backend is connected to.
message
(tag)
Text of a normalized statement.
pid
(tag)
Process ID of this backend.
query_signature
(tag)
The hash value computed from query
server
(tag)
The server address
service
(tag)
The service name postgresql
state
(tag)
Current overall state of this backend.
status
(tag)
The status of the statement. The value is only info for now.
usename
(tag)
Name of the user logged into this backend.
wait_event
(tag)
Wait event name if backend is currently waiting.
wait_event_type
(tag)
Type of event for which the backend is waiting.
backend_start Time when this process was started. For client backends, this is the time the client connected to the server.
Type: int | (count)
Unit: timeStamp,usec
query_start Time when the currently active query was started, or if state is not active, when the last query was started.
Type: int | (count)
Unit: timeStamp,usec
state_change Time when the state was last changed.
Type: int | (count)
Unit: timeStamp,usec
xact_start Time when this process' current transaction was started, or null if no transaction is active. If the current query is the first of its transaction, this column is equal to the query_start column.
Type: int | (count)
Unit: timeStamp,usec

文件日志

日志采集

  • PostgreSQL 日志默认是输出至 stderr,如需开启文件日志,可在 PostgreSQL 的配置文件 /etc/postgresql/<VERSION>/main/postgresql.conf , 进行如下配置:
logging_collector = on    # 开启日志写入文件功能

log_directory = 'pg_log'  # 设置文件存放目录,绝对路径或相对路径(相对 PGDATA)

log_filename = 'pg.log'   # 日志文件名称
log_statement = 'all'     # 记录所有查询

#log_duration = on
log_line_prefix= '%m [%p] %d [%a] %u [%h] %c ' # 日志行前缀
log_file_mode = 0644

# For Windows
#log_destination = 'eventlog'

更多配置,请参考官方文档

  • PostgreSQL 采集器默认是未开启日志采集功能,可在 conf.d/samples/postgresql.conf 中 将 files 打开,并写入 PostgreSQL 日志文件的绝对路径。比如:
[[inputs.postgresql]]

  ...

  [inputs.postgresql.log]
  files = ["/tmp/pgsql/postgresql.log"]

开启日志采集后,默认会产生日志来源(source)为 PostgreSQL 的日志。

注意:日志采集仅支持已安装 DataKit 主机上的日志。

日志 Pipeline 切割

原始日志为

2021-05-31 15:23:45.110 CST [74305] test [pgAdmin 4 - DB:postgres] postgres [127.0.0.1] 60b48f01.12241 LOG:  statement:
        SELECT psd.*, 2^31 - age(datfrozenxid) as wraparound, pg_database_size(psd.datname) as pg_database_size
        FROM pg_stat_database psd
        JOIN pg_database pd ON psd.datname = pd.datname
        WHERE psd.datname not ilike 'template%'   AND psd.datname not ilike 'rdsadmin'
        AND psd.datname not ilike 'azure_maintenance'   AND psd.datname not ilike 'postgres'

切割后的字段说明:

字段名 字段值 说明
application_name pgAdmin 4 - DB:postgres 连接当前数据库的应用的名称
db_name test 访问的数据库
process_id 74305 当前连接的客户端进程 ID
remote_host 127.0.0.1 客户端的地址
session_id 60b48f01.12241 当前会话的 ID
user postgres 当前访问用户名
status LOG 当前日志的级别(LOG,ERROR,FATAL,PANIC,WARNING,NOTICE,INFO)
time 1622445825110000000 日志产生时间

FAQ

缺失指标

对于 postgresql_lock/postgresql_stat/postgresql_index/postgresql_size/postgresql_statio 这些指标,需要开启配置文件中的 relations 字段。如果这些指标存在部分缺失,可能是因为相关指标不存在数据导致的。

文档评价

文档内容是否对您有帮助? ×