Hadoop Yarn ResourceManager¶

Collect Yarn ResourceManager metric information.

Installation and deployment¶

Since ResourceManager is developed in Java language, it can collect metric information using jmx-exporter.

1. ResourceManager Configuration¶

1.1 Download jmx-exporter¶

Download link：https://github.com/prometheus/jmx_exporter

1.2 Download jmx script¶

Download link：https://github.com/lrwh/jmx-exporter/blob/main/hadoop-yarn-resourcemanager.yml

1.3 ResourceManager startup parameter adjustment¶

Add startup parameters to resourcemanager

{{JAVA_GC_ARGS}} -javaagent:/opt/guance/jmx/jmx_exporter-1.0.1.jar=localhost:17109:/opt/guance/jmx/jmx_resource_manager.yml

1.4 Restart ResourceManager¶

2. DataKit Collector Configuration¶

2.1 Install DataKit¶

2.2 Configure collector¶

By using jmx exporter, the metrics URL can be directly exposed, so it can be collected directly through the prom collector.

Go to the installation directory of DataKit conf.d/prom and copy prom.conf.sample to resourcemanager.conf.

cp prom.conf.sample resourcemanager.conf

Adjust the content of resourcemanager.conf as follows:

  urls = ["http://localhost:17109/metrics"]
  source ="yarn-resourcemanager"
  [inputs.prom.tags]
    component = "yarn-resourcemanager" 
  interval = "10s"

Adjust other configurations as needed

，parameter adjustment instructions ：

• urls：jmx-exportermetric address, fill in the URL of the metric exposed by the corresponding component here
• source：Collector alias, it is recommended to make a distinction
• keep_exist_metric_name: Maintain metric name
• interval：Collection interval
• inputs.prom.tags: Add additional tags

3. Restart DataKit¶

Restart DataKit

Metric¶

Hadoop Metric Set¶

The ResourceManager metric is located under the Hadoop metric set, and here we mainly introduce the specifications of the ResourceManager related metrics.