RocketMQ (Prometheus)¶

Apache RocketMQ is a distributed messaging and stream processing platform for decoupling services, smoothing traffic peaks, and event-driven applications.

See the official RocketMQ metrics documentation for supported versions. For environments using a standalone Exporter, see RocketMQ Integration.

Configuration¶

Prerequisites¶

• A running RocketMQ Broker, version 5.1.0 or later.
• Access to edit the configuration file actually loaded by the Broker and restart the service. In a cluster, enable the metrics endpoint on every Broker you plan to monitor.
• DataKit can access the metrics port of each Broker. This guide uses the default port 5557.
• A Topic, consumer group, and production/consumption traffic for verification. Without application traffic, some metrics may be zero or have no corresponding series.

Enable the built-in Prometheus metrics on the Broker¶

Edit the configuration file actually specified by the Broker startup argument -c, usually broker.conf, and add:

metricsExporterType=PROM
metricsPromExporterHost=0.0.0.0
metricsPromExporterPort=5557

After saving, restart the corresponding Broker using your existing deployment procedure; perform rolling restarts in production clusters according to your maintenance process. Then verify the metrics endpoint on the Broker host:

curl -fsS http://127.0.0.1:5557/metrics

The response should contain metrics prefixed with rocketmq_ and their type information. Also verify endpoint accessibility from the DataKit host using the actual Broker address.

metricsPromExporterHost=0.0.0.0 listens on all network interfaces. Restrict access with security groups or a firewall. For Docker deployments, map the container port 5557 to a host port accessible to DataKit. If DataKit also runs in a container, its 127.0.0.1 refers to the DataKit container itself.

Configure the Prom collector¶

Copy the sample Prom collector configuration on the DataKit host:

mkdir -p /usr/local/datakit/conf.d/prom
cp /usr/local/datakit/conf.d/samples/prom.conf.sample \
  /usr/local/datakit/conf.d/prom/rocketmq.conf

Edit rocketmq.conf and replace its contents with:

[[inputs.prom]]
  urls = ["http://127.0.0.1:5557/metrics"]
  ignore_req_err = false
  source = "rocketmq"
  # An empty list collects all metric types, including Counter, Gauge, and Histogram
  metric_types = []
  metric_name_filter = ["^rocketmq_"]
  interval = "10s"
  measurement_prefix = ""
  measurement_name = "rocketmq"
  # Preserve the full Prometheus metric field names
  keep_exist_metric_name = true

See the Prom collector documentation for more parameters.

For cluster deployments, replace urls in the configuration with the actual metrics addresses of the Brokers:

urls = [
  "http://<BROKER_1_IP>:5557/metrics",
  "http://<BROKER_2_IP>:5557/metrics"
]

Alternatively, deploy DataKit on each Broker host and collect the local endpoint. Choose one coverage method to avoid collecting the same endpoint twice.

After saving the configuration, restart DataKit to apply it:

datakit service restart

Verify collection results¶

Use the Prom configuration debug command to inspect the converted measurement, fields, and tags:

datakit debug --prom-conf /usr/local/datakit/conf.d/prom/rocketmq.conf

The debug output should include the rocketmq measurement and fields prefixed with rocketmq_. Use the following command to check collector status and request errors:

datakit monitor

Metrics¶

With this configuration, metrics are written to the rocketmq measurement and fields retain their full Prometheus names. The fields actually available depend on the Broker /metrics endpoint, DataKit debug output, and ingested data.

The official RocketMQ logical metrics rocketmq_consumer_lag_latency and rocketmq_rpc_latency have the unit suffix _milliseconds in this example. Histogram metrics expand into _bucket, _sum, and _count, such as rocketmq_rpc_latency_milliseconds_bucket. Check the actual fields and units before creating views and monitors.

Counters are cumulative values and cannot directly represent TPS or per-second throughput. After calculating the per-second rate, message counts use messages/s and byte counts use Byte/s. Gauges represent the current state; observe the values and trends of ready messages, inflight messages, and consumer lag directly. Consumer lag is the waiting time of the earliest unacknowledged message, rather than the average processing time of completed messages.

Common tags¶

Tags vary by metric; use the actual collected data as the reference. Broker storage metrics are usually viewed by cluster and node_id, while consumer group metrics can be viewed by cluster, topic, and consumer_group.