Skip to content

RabbitMQ

·


RabbitMQ collector monitors RabbitMQ by collecting data through the plug-in rabbitmq-management and can:

  • RabbitMQ overview, such as connections, queues, total messages, and so on.
  • Track RabbitMQ queue information, such as queue size, consumer count and so on.
  • Rack RabbitMQ node information, such as socket mem.
  • Tracking RabbitMQ exchange information such as message_publish_count.

Configuration

Preconditions

  • RabbitMQ version >= 3.6.0; Already tested version:

    • 3.11.x
    • 3.10.x
    • 3.9.x
    • 3.8.x
    • 3.7.x
    • 3.6.x
  • Install rabbitmq, take Ubuntu as an example

    sudo apt-get update
    sudo apt-get install rabbitmq-server
    sudo service rabbitmq-server start
    
  • Start REST API plug-ins

    sudo rabbitmq-plugins enable rabbitmq_management
    
  • Create user, for example:

    sudo rabbitmqctl add_user guance <SECRET>
    sudo rabbitmqctl set_permissions  -p / guance "^aliveness-test$" "^amq\.default$" ".*"
    sudo rabbitmqctl set_user_tags guance monitoring
    

Collector Configuration

Go to the conf.d/rabbitmq directory under the DataKit installation directory, copy rabbitmq.conf.sample and name it rabbitmq.conf. Examples are as follows:

[[inputs.rabbitmq]]
  # rabbitmq url ,required
  url = "http://localhost:15672"

  # rabbitmq user, required
  username = "guest"

  # rabbitmq password, required
  password = "guest"

  # ##(optional) collection interval, default is 30s
  # interval = "30s"

  ## Optional TLS Config
  # tls_ca = "/xxx/ca.pem"
  # tls_cert = "/xxx/cert.cer"
  # tls_key = "/xxx/key.key"
  ## Use TLS but skip chain & host verification
  insecure_skip_verify = false

  ## Set true to enable election
  election = true

  # [inputs.rabbitmq.log]
  # files = []
  # #grok pipeline script path
  # pipeline = "rabbitmq.p"

  [inputs.rabbitmq.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

After configuration, restart DataKit.

The collector can now be turned on by ConfigMap injection collector configuration.

Metric

For all of the following data collections, the global election tags will added automatically, we can add extra tags in [inputs.rabbitmq.tags] if needed:

 [inputs.rabbitmq.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"
  # ...

rabbitmq_overview

  • Tags
Tag Description
cluster_name RabbitMQ cluster name
host Hostname of RabbitMQ running on.
rabbitmq_version RabbitMQ version
url RabbitMQ url
  • Metrics
Metric Description
message_ack_count Number of messages delivered to clients and acknowledged
Type: int
Unit: count
message_ack_rate Rate of messages delivered to clients and acknowledged per second
Type: float
Unit: percent,percent
message_confirm_count Count of messages confirmed
Type: int
Unit: count
message_confirm_rate Rate of messages confirmed per second
Type: float
Unit: percent,percent
message_deliver_get_count Sum of messages delivered in acknowledgement mode to consumers, in no-acknowledgement mode to consumers, in acknowledgement mode in response to basic.get, and in no-acknowledgement mode in response to basic.get
Type: int
Unit: count
message_deliver_get_rate Rate per second of the sum of messages delivered in acknowledgement mode to consumers, in no-acknowledgement mode to consumers, in acknowledgement mode in response to basic.get, and in no-acknowledgement mode in response to basic.get
Type: float
Unit: percent,percent
message_publish_count Count of messages published
Type: int
Unit: count
message_publish_in_count Count of messages published from channels into this overview
Type: int
Unit: count
message_publish_in_rate Rate of messages published from channels into this overview per sec
Type: float
Unit: percent,percent
message_publish_out_count Count of messages published from this overview into queues
Type: int
Unit: count
message_publish_out_rate Rate of messages published from this overview into queues per second
Type: float
Unit: percent,percent
message_publish_rate Rate of messages published per second
Type: float
Unit: percent,percent
message_redeliver_count Count of subset of messages in deliver_get which had the redelivered flag set
Type: int
Unit: count
message_redeliver_rate Rate of subset of messages in deliver_get which had the redelivered flag set per second
Type: float
Unit: percent,percent
message_return_unroutable_count Count of messages returned to publisher as unroutable
Type: int
Unit: count
message_return_unroutable_count_rate Rate of messages returned to publisher as unroutable per second
Type: float
Unit: percent,percent
object_totals_channels Total number of channels
Type: int
Unit: count
object_totals_connections Total number of connections
Type: int
Unit: count
object_totals_consumers Total number of consumers
Type: int
Unit: count
object_totals_queues Total number of queues
Type: int
Unit: count
queue_totals_messages_count Total number of messages (ready plus unacknowledged)
Type: int
Unit: count
queue_totals_messages_rate Total rate of messages (ready plus unacknowledged)
Type: float
Unit: percent,percent
queue_totals_messages_ready_count Number of messages ready for delivery
Type: int
Unit: count
queue_totals_messages_ready_rate Rate of number of messages ready for delivery
Type: float
Unit: percent,percent
queue_totals_messages_unacknowledged_count Number of unacknowledged messages
Type: int
Unit: count
queue_totals_messages_unacknowledged_rate Rate of number of unacknowledged messages
Type: float
Unit: percent,percent

rabbitmq_queue

  • Tags
Tag Description
cluster_name RabbitMQ cluster name
host Hostname of RabbitMQ running on.
node_name RabbitMQ node name
queue_name RabbitMQ queue name
url RabbitMQ host URL
vhost RabbitMQ queue virtual hosts
  • Metrics
Metric Description
bindings_count Number of bindings for a specific queue
Type: int
Unit: count
consumer_utilization The ratio of time that a queue's consumers can take new messages
Type: float
Unit: percent,percent
consumers Number of consumers
Type: int
Unit: count
head_message_timestamp Timestamp of the head message of the queue. Shown as millisecond
Type: int
Unit: timeStamp,msec
memory Bytes of memory consumed by the Erlang process associated with the queue, including stack, heap and internal structures
Type: int
Unit: digital,B
message_ack_count Number of messages in queues delivered to clients and acknowledged
Type: int
Unit: count
message_ack_rate Number per second of messages delivered to clients and acknowledged
Type: float
Unit: percent,percent
message_deliver_count Count of messages delivered in acknowledgement mode to consumers
Type: int
Unit: count
message_deliver_get_count Sum of messages in queues delivered in acknowledgement mode to consumers, in no-acknowledgement mode to consumers, in acknowledgement mode in response to basic.get, and in no-acknowledgement mode in response to basic.get.
Type: int
Unit: count
message_deliver_get_rate Rate per second of the sum of messages in queues delivered in acknowledgement mode to consumers, in no-acknowledgement mode to consumers, in acknowledgement mode in response to basic.get, and in no-acknowledgement mode in response to basic.get.
Type: float
Unit: percent,percent
message_deliver_rate Rate of messages delivered in acknowledgement mode to consumers
Type: float
Unit: percent,percent
message_publish_count Count of messages in queues published
Type: int
Unit: count
message_publish_rate Rate per second of messages published
Type: float
Unit: percent,percent
message_redeliver_count Count of subset of messages in queues in deliver_get which had the redelivered flag set
Type: int
Unit: count
message_redeliver_rate Rate per second of subset of messages in deliver_get which had the redelivered flag set
Type: float
Unit: percent,percent
messages Count of the total messages in the queue
Type: int
Unit: count
messages_rate Count per second of the total messages in the queue
Type: float
Unit: percent,percent
messages_ready Number of messages ready to be delivered to clients
Type: int
Unit: count
messages_ready_rate Number per second of messages ready to be delivered to clients
Type: float
Unit: percent,percent
messages_unacknowledged Number of messages delivered to clients but not yet acknowledged
Type: int
Unit: count
messages_unacknowledged_rate Number per second of messages delivered to clients but not yet acknowledged
Type: float
Unit: percent,percent

rabbitmq_exchange

  • Tags
Tag Description
auto_delete If set, the exchange is deleted when all queues have finished using it
cluster_name RabbitMQ cluster name
durable If set when creating a new exchange, the exchange will be marked as durable. Durable exchanges remain active when a server restarts. Non-durable exchanges (transient exchanges) are purged if/when a server restarts.
exchange_name RabbitMQ exchange name
host Hostname of RabbitMQ running on.
internal If set, the exchange may not be used directly by publishers, but only when bound to other exchanges. Internal exchanges are used to construct wiring that is not visible to applications
type RabbitMQ exchange type
url RabbitMQ host URL
vhost RabbitMQ exchange virtual hosts
  • Metrics
Metric Description
message_ack_count Number of messages in exchanges delivered to clients and acknowledged
Type: int
Unit: count
message_ack_rate Rate of messages in exchanges delivered to clients and acknowledged per second
Type: float
Unit: percent,percent
message_confirm_count Count of messages in exchanges confirmed
Type: int
Unit: count
message_confirm_rate Rate of messages in exchanges confirmed per second
Type: float
Unit: percent,percent
message_deliver_get_count Sum of messages in exchanges delivered in acknowledgement mode to consumers, in no-acknowledgement mode to consumers, in acknowledgement mode in response to basic.get, and in no-acknowledgement mode in response to basic.get
Type: int
Unit: count
message_deliver_get_rate Rate per second of the sum of exchange messages delivered in acknowledgement mode to consumers, in no-acknowledgement mode to consumers, in acknowledgement mode in response to basic.get, and in no-acknowledgement mode in response to basic.get
Type: float
Unit: percent,percent
message_publish_count Count of messages in exchanges published
Type: int
Unit: count
message_publish_in_count Count of messages published from channels into this exchange
Type: int
Unit: count
message_publish_in_rate Rate of messages published from channels into this exchange per sec
Type: float
Unit: percent,percent
message_publish_out_count Count of messages published from this exchange into queues
Type: int
Unit: count
message_publish_out_rate Rate of messages published from this exchange into queues per second
Type: float
Unit: percent,percent
message_publish_rate Rate of messages in exchanges published per second
Type: float
Unit: percent,percent
message_redeliver_count Count of subset of messages in exchanges in deliver_get which had the redelivered flag set
Type: int
Unit: count
message_redeliver_rate Rate of subset of messages in exchanges in deliver_get which had the redelivered flag set per second
Type: float
Unit: percent,percent
message_return_unroutable_count Count of messages in exchanges returned to publisher as un-routable
Type: int
Unit: count
message_return_unroutable_count_rate Rate of messages in exchanges returned to publisher as un-routable per second
Type: float
Unit: percent,percent

rabbitmq_node

  • Tags
Tag Description
cluster_name RabbitMQ cluster name
host Hostname of RabbitMQ running on.
node_name RabbitMQ node name
url RabbitMQ url
  • Metrics
Metric Description
disk_free Current free disk space
Type: int
Unit: digital,B
disk_free_alarm Does the node have disk alarm
Type: bool
Unit: N/A
fd_used Used file descriptors
Type: int
Unit: count
io_read_avg_time Average wall time (milliseconds) for each disk read operation in the last statistics interval
Type: float
Unit: time,ms
io_seek_avg_time Average wall time (milliseconds) for each seek operation in the last statistics interval
Type: float
Unit: time,ms
io_sync_avg_time Average wall time (milliseconds) for each fsync() operation in the last statistics interval
Type: float
Unit: time,ms
io_write_avg_time Average wall time (milliseconds) for each disk write operation in the last statistics interval
Type: float
Unit: time,ms
mem_alarm Does the node have mem alarm
Type: bool
Unit: N/A
mem_limit Memory usage high watermark in bytes
Type: int
Unit: digital,B
mem_used Memory used in bytes
Type: int
Unit: digital,B
run_queue Average number of Erlang processes waiting to run
Type: int
Unit: count
running Is the node running or not
Type: bool
Unit: N/A
sockets_used Number of file descriptors used as sockets
Type: int
Unit: count

collector

  • Tags
Tag Description
instance Server addr of the instance
job Server name of the instance
  • Metrics
Metric Description
up
Type: int
Unit: -

Custom Object

mq

  • tag
Tag Description
col_co_status Current status of collector on instance(OK/NotOK)
host The server host address
ip Connection IP of the instance
name Object uniq ID
reason If status not ok, we'll get some reasons about the status
  • Metric list
Metric Description
display_name Displayed name in UI
Type: string
Unit: N/A
uptime Current instance uptime
Type: int
Unit: time,s
version Current version of the instance
Type: string
Unit: N/A

Log Collection

Note

DataKit must be installed on the host where RabbitMQ is located to collect RabbitMQ logs.

To collect the RabbitMQ log, open files in RabbitMQ.conf and write to the absolute path of the RabbitMQ log file. For example:

    [[inputs.rabbitmq]]
      ...
      [inputs.rabbitmq.log]
        files = ["/var/log/rabbitmq/rabbit@your-hostname.log"]

When log collection is turned on, a log with a log source of rabbitmq is generated by default.

Log Pipeline Function Cut Field Description

  • RabbitMQ universal log cutting

Example of common log text:

2021-05-26 14:20:06.105 [warning] <0.12897.46> rabbitmqctl node_health_check and its HTTP API counterpart are DEPRECATED. See https://www.rabbitmq.com/monitoring.html#health-checks for replacement options.

The list of cut fields is as follows:

Field Name Field Value Description
status warning Log level
msg <0.12897.46>...replacement options Log level
time 1622010006000000000 Nanosecond timestamp (as row protocol time)

Feedback

Is this page helpful? ×