Consul

·

Consul collector is used to collect metric data related to Consul, and currently it only supports data in Prometheus format.

Configuration¶

Preconditions¶

Installing consul-exporter

Download consul_exporter package

sudo wget https://github.com/prometheus/consul_exporter/releases/download/v0.7.1/consul_exporter-0.7.1.linux-amd64.tar.gz

Unzip consul_exporter package

sudo tar -zxvf consul_exporter-0.7.1.linux-amd64.tar.gz

Go to the consul_exporter-0.7.1.linux-amd64 directory and run the consul_exporter script

./consul_exporter

Collector Configuration¶

host installationKubernetes

Go to the conf.d/consul directory under the DataKit installation directory, copy consul.conf.sample and name it consul.conf. Examples are as follows:

[[inputs.prom]]
  url = "http://127.0.0.1:9107/metrics"
  source = "consul"
  metric_types = ["counter", "gauge"]
  metric_name_filter = ["consul_raft_leader", "consul_raft_peers", "consul_serf_lan_members", "consul_catalog_service", "consul_catalog_service_node_healthy", "consul_health_node_status", "consul_serf_lan_member_status"]
  measurement_prefix = ""
  tags_ignore = ["check"]
  interval = "10s"

[[inputs.prom.measurements]]
  prefix = "consul_"
  name = "consul"

After configuration, restart DataKit.

The collector can now be turned on by ConfigMap Injection Collector Configuration.

Metric¶

`consul`¶

Tags

Tag	Description
check	Check.
check_id	Check id.
check_name	Check name.
host	Host name.
instance	Instance endpoint.
key	Key.
member	Member name.
node	Node name.
service_id	Service id.
service_name	Service name.
status	Status: critical, maintenance, passing, warning.
tag	Tag.

Metrics

Metric	Description
catalog_kv	The values for selected keys in Consul's key/value catalog. Keys with non-numeric values are omitted. Type: float Unit: -
catalog_service_node_healthy	Is this service healthy on this node? Type: float Unit: bool
catalog_services	How many services are in the cluster. Type: float Unit: count
health_node_status	Status of health checks associated with a node. Type: float Unit: bool
health_service_status	Status of health checks associated with a service. Type: float Unit: bool
raft_leader	Does Raft cluster have a leader (according to this node). Type: float Unit: bool
raft_peers	How many peers (servers) are in the Raft cluster. Type: float Unit: count
serf_lan_member_status	Status of member in the cluster. 1=Alive, 2=Leaving, 3=Left, 4=Failed. Type: float Unit: -
serf_lan_members	How many members are in the cluster. Type: float Unit: count
serf_wan_member_status	SStatus of member in the wan cluster. 1=Alive, 2=Leaving, 3=Left, 4=Failed. Type: float Unit: -
service_checks	Link the service id and check name if available. Type: float Unit: bool
service_tag	Tags of a service. Type: float Unit: count
up	Was the last query of Consul successful. Type: float Unit: bool

Logs¶

If you need to collect the log of Consul, you need to use the-syslog parameter when opening Consul, for example:

consul agent -dev -syslog

To use the logging collector to collect logs, you need to configure the logging collector. Go to the conf.d/log directory under the DataKit installation directory, copy logging.conf.sample and name it logging.conf. The configuration is as follows:

[[inputs.logging]]
  ## required
  logfiles = [
    "/var/log/syslog",
  ]

  ## glob filteer
  ignore = [""]

  ## your logging source, if it's empty, use 'default'
  source = "consul"

  ## add service tag, if it's empty, use $source.
  service = ""

  ## grok pipeline script path
  pipeline = "consul.p"

  ## optional status:
  ##   "emerg","alert","critical","error","warning","info","debug","OK"
  ignore_status = []

  ## optional encodings:
  ##    "utf-8", "utf-16le", "utf-16le", "gbk", "gb18030" or ""
  character_encoding = ""

  ## The pattern should be a regexp. Note the use of '''this regexp'''
  ## regexp link: https://golang.org/pkg/regexp/syntax/#hdr-Syntax
  multiline_match = '''^\S'''

  [inputs.logging.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"

Original log:

Sep 18 19:30:23 derrick-ThinkPad-X230 consul[11803]: 2021-09-18T19:30:23.522+0800 [INFO]  agent.server.connect: initialized primary datacenter CA with provider: provider=consul

The list of cut fields is as follows:

Field name	Field value	Description
`date`	`2021-09-18T19:30:23.522+0800`	log date
`level`	`INFO`	log level
`character`	`agent.server.connect`	role
`msg`	`initialized primary datacenter CA with provider: provider=consul`	log content