Hadoop Yarn ResourceManager¶

Collect metrics information from Yarn ResourceManager.

Installation and Deployment¶

Since ResourceManager is developed in Java, you can use the jmx-exporter method to collect metrics information.

1. ResourceManager Configuration¶

1.1 Download jmx-exporter¶

Download address: https://github.com/prometheus/jmx_exporter

1.2 Download jmx Script¶

Download address: https://github.com/lrwh/jmx-exporter/blob/main/hadoop-yarn-resourcemanager.yml

1.3 Adjust ResourceManager Startup Parameters¶

Add the following parameters to the startup parameters of resourcemanager:

{JAVA_GC_ARGS} -javaagent:/opt/guance/jmx/jmx_exporter-1.0.1.jar=localhost:17109:/opt/guance/jmx/jmx_resource_manager.yml

1.4 Restart ResourceManager¶

2. DataKit Collector Configuration¶

2.1 Install DataKit¶

2.2 Configure Collector¶

The jmx-exporter can directly expose a metrics URL, so it can be collected directly using the prom collector.

Go to the DataKit installation directory under conf.d/prom, copy prom.conf.sample to resourcemanager.conf.

cp prom.conf.sample resourcemanager.conf

Adjust the content of resourcemanager.conf as follows:

  urls = ["http://localhost:17109/metrics"]
  source ="yarn-resourcemanager"
  [inputs.prom.tags]
    component = "yarn-resourcemanager" 
  interval = "10s"

Other configurations should be adjusted as needed

, parameter adjustment explanation:

• urls: The address of the jmx-exporter metrics, fill in the exposed metrics URL of the corresponding component.
• source: Alias for the collector, recommended to differentiate.
• keep_exist_metric_name: Keep metric names unchanged.
• interval: Collection interval.
• inputs.prom.tags: Add additional tags.

3. Restart DataKit¶

Restart DataKit

Metrics¶

Hadoop Measurement Set¶

ResourceManager metrics are located under the Hadoop Measurement set. Here we mainly introduce the relevant metrics of ResourceManager.

英语

Please continue translating.