Mutation Detection¶

By comparing the absolute change or relative percentage change of the same metric over two different time periods, it determines whether an anomaly has occurred. This method is commonly used to track peaks or fluctuations in metrics. When an anomaly is detected, it can generate event records more precisely for subsequent analysis and processing.

Use Cases¶

Mutation Detection is suitable for monitoring short-term relative changes or change rates compared to long-term data. For example, setting the MySQL connection count metric to a percentage difference greater than 500% between the last 15 minutes and the average of the past day means that if the average connection count in the last 15 minutes exceeds 5 times the average connection count of the past day, the system will trigger an alert.

It is recommended to use statistical functions such as average (AVG), maximum (MAX), minimum (MIN), etc., to calculate these metrics, rather than using the last value (LAST) function, to reduce the impact of abnormal data and improve monitoring accuracy.

Configuration File¶

Detection Metric¶

This refers to the metric data being monitored. The difference or percentage difference of this metric over two time periods can be compared.

The selectable detection intervals for time periods include last month, last week, yesterday, 1 hour ago, compared to the previous period, last 15 minutes, last 30 minutes, last 1 hour, last 4 hours, last 12 hours, and last 1 day.

Cross-Workspace Query Metric¶

After authorization, detection metrics from other workspaces under the current account can be selected. After the monitor rule is successfully created, cross-workspace alert configuration can be achieved.

Detection Frequency¶

The execution frequency of the detection rule automatically matches the larger time range between the two detection intervals selected by the user. Default is 5 minutes.

Trigger Conditions¶

Configure the trigger conditions for alert levels: You can configure any one of the following trigger conditions: Critical, Important, Warning, Data Gap, or Info:

1. Trigger Precondition Configuration: Enabled by default; when the detection value meets the threshold set in the trigger precondition (operators supported: >, >=, <, <=, default is >), the mutation detection rule judgment continues; if this configuration is turned off, only the mutation detection rule judgment is performed.

2. Mutation Rule Configuration: Three forms of data comparison for mutation detection rule judgment: Upward (data increase), Downward (data decrease), or Upward or Downward.

Configure trigger conditions and severity. When the query result contains multiple values, an event is generated if any value meets the trigger condition.

For more details, refer to Event Level Description.

Bulk Alert Protection¶

Enabled by default.

When the number of alerts generated in a single detection exceeds a preset threshold, the system automatically switches to a status-based aggregation strategy: instead of processing each alert object individually, it generates a small number of summary alerts based on event status and pushes them.

This ensures the timeliness of notifications while significantly reducing alert noise and avoiding timeout risks caused by processing too many alerts.

Alert Level¶

1. Alert Level Critical (red), Important (orange), Warning (yellow);

2. Alert Level Normal (green): Based on the configured number of detection times, explained as follows:

   • Each execution of a detection task counts as 1 detection. For example, if Detection Frequency = 5 minutes, then 1 detection = 5 minutes;

   • The number of detections can be customized. For example, if Detection Frequency = 5 minutes, then 3 detections = 15 minutes.

   Level

   Description

   Normal

   After the detection rule takes effect, if an abnormal event of Critical, Important, or Warning level occurs and the data detection result returns to normal within the configured number of custom detections, a recovery alert event is generated. ❗️ Recovery alert events are not subject to Alert Silence restrictions. If the number of recovery alert event detections is not set, the alert event will not recover and will remain in the Events > Unrecovered Events List.

Data Gap¶

Seven strategies can be configured for the data gap status.

1. Link to the detection interval time range, judge the query result of the detection metric for the recent number of minutes, do not trigger an event;

2. Link to the detection interval time range, judge the query result of the detection metric for the recent number of minutes, treat the query result as 0; at this time, the query result will be re-compared with the threshold configured in the Trigger Conditions above to determine whether to trigger an abnormal event.

3. Custom fill the detection interval value, trigger a data gap event, trigger a critical event, trigger an important event, trigger a warning event, and trigger a recovery event; if this type of configuration strategy is selected, the recommended custom data gap time configuration is >= detection interval time interval. If the configured time is <= the detection interval time interval, situations where both data gap and abnormal conditions are met may occur. In such cases, only the data gap processing result will be applied.

Info Generation¶

After enabling this option, detection results that do not match any of the above trigger conditions will generate "Info" events and be written.

Other Configuration¶

For more details, refer to Rule Configuration.