Skip to content

NSQ

·

Collect NSQ operation data and report it to Guance in the form of indicators.

Configuration

Preconditions

  • NSQ installed（NSQ official website）

  • Recommend NSQ version >= 1.0.0, already tested version:

  • 1.2.1

  • 1.1.0
  • 0.3.8

Collector Configuration

Go to the conf.d/samples directory under the DataKit installation directory, copy nsq.conf.sample and name it nsq.conf. Examples are as follows:

[[inputs.nsq]]
  ## NSQ Lookupd HTTP API endpoint
  lookupd = "http://localhost:4161"

  ## NSQD HTTP API endpoint
  ## example:
  ##   ["http://localhost:4151"]
  nsqd = []

  ## time units are "ms", "s", "m", "h"
  interval = "10s"

  ## Set true to enable election
  election = true

  ## Optional TLS Config
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Use TLS but skip chain & host verification
  # insecure_skip_verify = false

  [inputs.nsq.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"

Once configured, restart DataKit.

The collector can now be turned on by ConfigMap Injection Collector Configuration.

Tow mode for NSQ collector

The NSQ collector is available in two configurations, lookupd and nsqd, as follows:

  • lookupd: Configure the lookupd address of the NSQ cluster, and the collector will automatically discover the NSQ Server and collect data, which is more scalable.
  • nsqd: Configure a fixed list of NSQ Daemon (nsqd) addresses for which the collector collects only NSQ Server data

The above two configuration methods are mutually exclusive, and lookupd has higher priority, so it is recommended to use lookupd configuration method.

Metric

For all of the following data collections, the global election tags will added automatically, we can add extra tags in [inputs.nsq.tags] if needed:

 [inputs.nsq.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

nsq_topics

Metrics of all topics in the NSQ cluster

Tags & Fields Description
channel
(tag)		 Channel name
topic
(tag)		 Topic name
backend_depth Total number of unconsumed messages exceeding the max-queue-size.
Type: int
Unit: count
deferred_count Number of messages that have been requeued and are not yet ready for re-sending.
Type: int
Unit: count
depth Total number of unconsumed messages in the current channel.
Type: int
Unit: count
in_flight_count Number of messages during the sending process or client processing that have not been sent FIN, REQ (requeued), or timed out.
Type: int
Unit: count
message_count Total number of messages processed in the current channel.
Type: int
Unit: count
requeue_count Number of messages that have timed out or have been sent REQ by the client.
Type: int
Unit: count
timeout_count Number of messages that have timed out and are still unprocessed.
Type: int
Unit: count

nsq_nodes

Metrics of all nodes in the NSQ cluster.

Tags & Fields Description
host
(tag)		 Hostname
server_host
(tag)		 Service address, that is host:ip.
backend_depth Total number of unconsumed messages exceeding the max-queue-size.
Type: int
Unit: count
depth Total number of unconsumed messages in the current node.
Type: int
Unit: count
message_count Total number of messages processed by the current node.
Type: int
Unit: count

collector

Shared collector availability metric emitted by multiple collectors to report whether the target was successfully scraped.

Tags & Fields Description
instance
(tag)		 Server addr of the instance
job
(tag)		 Server name of the instance
up Whether the collector successfully scraped the target during the last collection cycle: 1 means true and 0 means false.
Type: int | (gauge)
Unit: bool

Custom Object

mq

NSQ custom object metadata for monitored message queue instances, including uptime, display identity, version, and collector status.

Tags & Fields Description
col_co_status
(tag)		 Collector status for this instance, such as OK or NotOK.
host
(tag)		 Hostname or address of the server that runs this instance.
ip
(tag)		 Configured connection IP address for this instance.
name
(tag)		 Stable object identifier for this monitored instance.
reason
(tag)		 Reason reported when the collector status is not OK.
display_name Display name shown for this instance in the Datakit UI.
Type: string | (gauge)
Unit: N/A
uptime Time in seconds since this instance last started.
Type: int | (gauge)
Unit: time,s
version Server version reported by this instance.
Type: string | (gauge)
Unit: N/A

Feedback

Is this page helpful? ×