APM Metrics Anomaly Detection¶

Data Scope: Traces (T), used to monitor key metric data of APM within the workspace. The system counts the number of traces meeting the conditions within a specified time period and triggers an anomaly event when it exceeds a custom threshold.

Detection Configuration¶

Detection Frequency¶

Set the time cycle for executing detection.

• Preset Options: 1 minute, 5 minutes, 10 minutes, 15 minutes, 30 minutes, 1 hour

• Crontab Mode: Click "Switch to Crontab Mode" to configure a custom cycle, supporting scheduled task execution based on seconds, minutes, hours, days, months, weeks, etc.

Detection Interval¶

Set the data time range queried for each detection (❗️The detection interval should be greater than or equal to the detection frequency and should match the actual data reporting cycle to avoid missed detection or false alarms).

• Custom Format: Custom input for detection interval, e.g., 20m (last 20 minutes), 2h (last 2 hours), 1d (last 1 day).

Detection Metrics¶

Set the metrics for detection data, supporting two detection modes:

• Service Metrics

• Trace Statistics

Service Metrics¶

Monitor the APM services within the current workspace.

Trace Statistics¶

Count the number of traces (Spans) meeting the conditions within a specified time period, triggering an anomaly event when exceeding a custom threshold. Can be used for notifications of service trace anomalies and errors.

Trigger Conditions¶

Configure trigger conditions and severity levels. When the query result contains multiple values, an event is generated if any value meets the trigger condition.

Supports configuring four levels of thresholds: Fatal, Severe, Important, Warning, and a Normal recovery condition.

For more details, refer to Event Level Description.

Advanced Options¶

Consecutive Trigger Judgment¶

When enabled, events are generated only when the trigger condition is continuously met, avoiding false alarms from transient fluctuations (❗️Maximum configuration limit is 10 times).

Bulk Alert Protection¶

Enabled by default.

When the number of alerts generated in a single detection exceeds a preset threshold, the system automatically switches to a status summary strategy: instead of processing each alert object individually, it generates a small number of summary alerts based on event status and pushes them.

This ensures notification timeliness while significantly reducing alert noise and avoiding timeout risks from processing too many alerts.

When this switch is enabled, subsequent event details generated by the monitor after detecting anomalies will not display historical records and related events.

Recovery Conditions¶

Configure recovery conditions and severity levels. When the query result contains multiple values, a recovery event is generated if any value meets the trigger condition.

Set independent recovery thresholds for different levels to achieve downgraded recovery. For example: a Severe alert recovers when the value drops below 70, while an Important alert recovers below 80.

Data Gap¶

Processing strategy when the detection metric query result is empty within the detection interval:

When trigger conditions, data gap, and information generation are configured simultaneously, the triggering priority is judged as follows: Data Gap > Trigger Conditions > Information Event Generation.

That is: first judge if there is a data gap, then judge if thresholds are triggered, and finally judge if information events should be generated.

Information Generation¶

When this option is enabled, the system writes all detection results that do not match the above trigger conditions as "Information" events.

Suitable for scenarios requiring recording normal status changes or low-priority information.

Subsequent Configuration¶

After completing the above detection configuration, please continue to configure:

1. Event Notification: Define event title, content, notification members, data gap handling, and associated incidents;
2. Alert Configuration: Select alert strategies, set notification targets, and mute periods;
3. Association: Associate dashboards for quick data viewing;
4. Permissions: Set operation permissions to control who can edit/delete this monitor.