Hadoop Yarn NodeManager¶

Collect Yarn NodeManager metric information.

Installation and deployment¶

Since NodeManager is developed in Java language, it is possible to collect metric information using jmx-exporter.

1. NodeManager Configuration¶

1.1 Download jmx-exporter¶

Download link: https://github.com/prometheus/jmx_exporter

1.2 Download jmx script¶

Download link: https://github.com/lrwh/jmx-exporter/blob/main/hadoop-yarn-nodemanager.yml

1.3 NodeManager start parameter adjustment¶

Add startup parameters to nodemanager

{{JAVA_GC_ARGS}} -javaagent:/opt/guance/jmx/jmx_exporter-1.0.1.jar=localhost:17108:/opt/guance/jmx/jmx_node_manager.yml

1.4 Restart NodeManager¶

2. DataKit Collector Configuration¶

2.1 Install DataKit¶

2.2 Configure collector¶

By using jmx exporter, the metrics URL can be directly exposed, so it can be collected directly through the prom collector.

Go to the installation directory of DataKit and copy prom.conf.sample to nodemanager.conf.

cp prom.conf.sample nodemanager.conf

Adjust the content of nodemanager.conf as follows：

  urls = ["http://localhost:17108/metrics"]
  source ="yarn-nodemanager"
  [inputs.prom.tags]
    component = "yarn-nodemanager" 
  interval = "10s"

Adjust other configurations as needed

，parameter adjustment instructions ：

• Urls: jmx-exporter metric address, fill in the corresponding metric URL exposed by the component here
• Source: Collector alias, it is recommended to distinguish it
• Keep.exe ist_stricic_name: Keep indicator name
• Interval: collection interval
• Inputsprom.tags: Add additional tags

3. Restart DataKit¶

Restart DataKit

Metric¶

Hadoop Metric Set¶

The NodeManager metric is located under the Hadoop metric set, and here we mainly introduce the description of NodeManager related metrics.