AWS Glue

Collect AWS Glue job assets and Spark job metrics.

Configuration¶

Install Func¶

We recommend enabling Guance Integrations - Extensions - Hosted Func: all prerequisites are installed automatically. Continue with script installation.

To deploy Func yourself, see Self-hosted Func.

Enable the Script¶

Tip: Prepare suitable AWS access keys in advance (for simplicity, you can grant the global read-only permission ReadOnlyAccess).

Activate Script in DataFlux Func (Automata)¶

1. Log in to the Guance console
2. Click the 【Integration】 menu and select 【Cloud Account Management】
3. Click 【Add Cloud Account】, select 【AWS】, and fill in the required information on the interface. If you have already configured the cloud account information before, ignore this step
4. Click 【Test】, and if the test is successful, click 【Save】. If the test fails, please check if the relevant configuration information is correct and test again
5. Click 【Cloud Account Management】, and you can see the added cloud account in the list. Click the corresponding cloud account to enter the details page
6. Click the 【Integration】 button on the cloud account details page, find AWS Glue under the Not Installed list, click the 【Install】 button, and the installation interface will pop up for installation.

Activate Script Manually¶

1. Log in to the Func console, click [Script Market] to open the Guance script market, and search for integration_aws_glue.
2. Click [Install], then enter the required parameters: AWS AK ID, AK Secret, and account name.
3. Click [Deploy Startup Script]. The system automatically creates the Startup script set and configures the corresponding startup script.
4. Once enabled, the corresponding automatic trigger configuration appears under [Management / Automatic Trigger Configuration]. Click [Execute] to run it immediately without waiting for the scheduled time. After a short wait, check the task execution records and logs.

Verification¶

1. Under [Management / Automatic Trigger Configuration], confirm that an automatic trigger configuration exists for the task, and check the task records and logs for errors.
2. In Guance, check [Infrastructure / Custom] for asset information.
3. In Guance, check [Metrics] for the corresponding monitoring data.

Metrics¶

The measurement is aws_glue.

AWS Glue Metrics Reference