Skip to content

NSQ

·

Collect NSQ operation data and report it to Guance in the form of indicators.

Configuration

Preconditions

  • NSQ installed（NSQ official website）

  • Recommend NSQ version >= 1.0.0, already tested version:

  • 1.2.1

  • 1.1.0
  • 0.3.8

Collector Configuration

Go to the conf.d/nsq directory under the DataKit installation directory, copy nsq.conf.sample and name it nsq.conf. Examples are as follows:

[[inputs.nsq]]
  ## NSQ Lookupd HTTP API endpoint
  lookupd = "http://localhost:4161"

  ## NSQD HTTP API endpoint
  ## example:
  ##   ["http://localhost:4151"]
  nsqd = []

  ## time units are "ms", "s", "m", "h"
  interval = "10s"

  ## Set true to enable election
  election = true

  ## Optional TLS Config
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Use TLS but skip chain & host verification
  # insecure_skip_verify = false

  [inputs.nsq.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"

Once configured, restart DataKit.

The collector can now be turned on by ConfigMap Injection Collector Configuration.

Tow mode for NSQ collector

The NSQ collector is available in two configurations, lookupd and nsqd, as follows:

  • lookupd: Configure the lookupd address of the NSQ cluster, and the collector will automatically discover the NSQ Server and collect data, which is more scalable.
  • nsqd: Configure a fixed list of NSQ Daemon (nsqd) addresses for which the collector collects only NSQ Server data

The above two configuration methods are mutually exclusive, and lookupd has higher priority, so it is recommended to use lookupd configuration method.

Metric

For all of the following data collections, the global election tags will added automatically, we can add extra tags in [inputs.nsq.tags] if needed:

 [inputs.nsq.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

nsq_topics

Metrics of all topics in the NSQ cluster

  • Tags
Tag Description
channel Channel name
topic Topic name
  • Metrics
Metric Description
backend_depth Total number of unconsumed messages exceeding the max-queue-size.
Type: int
Unit: count
deferred_count Number of messages that have been requeued and are not yet ready for re-sending.
Type: int
Unit: count
depth Total number of unconsumed messages in the current channel.
Type: int
Unit: count
in_flight_count Number of messages during the sending process or client processing that have not been sent FIN, REQ (requeued), or timed out.
Type: int
Unit: count
message_count Total number of messages processed in the current channel.
Type: int
Unit: count
requeue_count Number of messages that have timed out or have been sent REQ by the client.
Type: int
Unit: count
timeout_count Number of messages that have timed out and are still unprocessed.
Type: int
Unit: count

nsq_nodes

Metrics of all nodes in the NSQ cluster.

  • Tags
Tag Description
host Hostname
server_host Service address, that is host:ip.
  • Metrics
Metric Description
backend_depth Total number of unconsumed messages exceeding the max-queue-size.
Type: int
Unit: count
depth Total number of unconsumed messages in the current node.
Type: int
Unit: count
message_count Total number of messages processed by the current node.
Type: int
Unit: count

collector

  • Tags
Tag Description
instance Server addr of the instance
job Server name of the instance
  • Metrics
Metric Description
up
Type: int
Unit: -

Custom Object

mq

  • Tags
Tag Description
col_co_status Current status of collector on instance(OK/NotOK)
host The server host address
ip Connection IP of the instance
name Object uniq ID
reason If status not ok, we'll get some reasons about the status
  • Metrics
Metric Description
display_name Displayed name in UI
Type: string
Unit: N/A
uptime Current instance uptime
Type: int
Unit: time,s
version Current version of the instance
Type: string
Unit: N/A

Feedback

Is this page helpful? ×