Alert Strategies¶

When monitors detect anomalies, they automatically create incident records. By associating monitors with alert strategies, you can ensure that relevant alert notifications are sent to the specified targets in a timely manner.

The configuration of alert strategies not only provides basic functions such as name, description, time zone, and operational permissions but also supports flexible definition of notification methods from two dimensions: alert level and notification targets. In addition, you can configure escalation notification functionality for alert strategies to deal with emergencies. At the same time, alert strategies allow you to customize the timing of notifications to meet the needs of different scenarios.

For continuously occurring anomaly events or specific alert conditions, you can set up repeat alert rules to flexibly control the frequency of notifications. When sending notifications, you can also choose whether to aggregate notification content to convey information to recipients more efficiently and concisely.

Start Creating¶

1. Define the name of the current alert strategy;
2. Enter a description for the strategy as needed;
3. Select the associated monitors;
4. Choose the notification time zone;
5. Select the alert strategy configured to trigger notifications based on level or based on members;
6. Choose the time range for repeat alerts (original alert silence);
7. Select the alert aggregation mode as needed to determine the final form of the alert notification;
8. Add operational permissions for the strategy rules as needed;
9. Save, and the creation will be successful.

Association¶

On the configuration page, you can click to select the monitoring rules associated with the current alert strategy, including:

• All
• Monitors
• Intelligent Monitoring
• SLO
• Security Monitoring

Here, you can quickly create new monitoring rules as needed.

Notification Rule Configuration¶

Currently supports two types of notification configurations: based on level and based on members.

In the former, after selecting the event level, notification targets are set for this type of anomaly event. If filter conditions are set, then the scope of event data under a certain level is further restricted, and finally notifications are sent to the targets.

In the latter, members or teams are selected first, defining the scope of events they need to pay attention to or take responsibility for within this data. Within this data, event levels and corresponding notification targets are further defined, achieving strong associations between events and targets.

Notification Configuration Based on Levels¶

Define the notification targets for sending alerts at each level.

1. Select event levels.

   • An event level can be chosen multiple times;
   • Based on the selected event level, you can link alert aggregation.

2. Select the notification targets for events at this level.

Notification Configuration Based on Members¶

Notification rules configured based on members can achieve precise point-to-point alert notifications. At the same time, within one alert rule, you can configure different notification scopes, levels, and methods for multiple groups of members, and customize notification time ranges for multiple groups of members.

1. Define the name of the notification rule;
2. Select the members and teams that need to be notified;
3. Add filter conditions to achieve tag matching;
4. For the filtered event data, you can set corresponding notification targets for different event levels;
5. Enable the configuration of custom notification time ranges as needed.

Add Filter Conditions¶

Whether configuring notifications based on levels or members, adding specific filter conditions can:

• For notifications based on levels, further refine the data scope of specific level events;
• For notifications based on members, limit members or teams to focus only on events that match specific tags.

After adding filters, only events that meet the level requirements and filter conditions will trigger notifications.

After clicking the filter button, the system will automatically retrieve the fields of the current workspace and set filter conditions in the form of key:value. You can choose the following matching methods: equals, not equals, wildcard, inverse wildcard, and regular expression matching. Multiple filter conditions for the same key field are in OR relationship, while filter conditions for different key fields are in AND relationship.

You can configure filter conditions in the following two ways:

• Directly select fields on the page and set conditions.
• Write regular expressions to implement more complex filtering logic to meet fine-tuning configuration needs.

Escalation Notifications¶

If a monitor frequently detects anomalies of the same level within a short period, it may indicate a persistent issue. At this point, other notification targets may be needed to resolve this type of problem, and you can adopt the method of adding escalation notification rules. Thus, when anomalies continue to occur, the system will automatically escalate them to critical notifications and send them to designated receivers to ensure that problems are promptly noticed and handled.

If a notification rule is configured with two escalation notifications:

• When alerts of the same level persist, the system checks the time intervals to determine whether to send the first escalation notification;
• After sending the first escalation notification, the system determines whether to send the second escalation notification based on the time interval configured for the second escalation notification.

Custom Notification Times¶

The scenarios discussed above mainly revolve around the immediacy of automatically triggering notifications upon detecting anomalies. However, in practice, you can also manually set the exact times for sending notifications based on your needs.

1. Modify the configuration name as needed;

2. Divide the event cycle based on four dimensions: day, week, month, and custom;

   • If you choose custom, upload a CSV file, and the system will automatically fill in the dates written in the file. The date format in the file must be year/month/day & YYYY/MM/DD; the number of dates in the file should not exceed 365.

3. Limit the time of event generation on the day based on the cycle and send notifications according to the selected time range. For example, if you select 09:00 - 10:00, any abnormal events generated within this hour when the policy is active will match this custom configuration;

4. After completing the cycle and time-related configurations, you can select the alert level and notification targets.

Repeat Alerts¶

After setting repeat alert notifications, within a certain time range, event data will continue to be generated, but alert notifications will not be sent again, and the generated data records will be stored in the event explorer.

Alert Aggregation¶

No Aggregation: Default configuration; under this mode, alert events are merged every 20 seconds and sent to the corresponding notification targets;

Rule-Based Aggregation: Under this mode, you can choose from the following four aggregation rules and send alert notifications based on the aggregation period:

Aggregation Rule	Description
All	Based on the level dimension configured in the alert strategy, generate corresponding alert notifications within the selected aggregation period.
Monitors/Intelligent Inspection/SLO	According to the detection rules of monitors, intelligent inspections, or the unique ID of SLOs, generate corresponding alert notifications linked to the aggregation period.
Detection Dimensions	Generate corresponding alert notifications based on detection dimensions linked to the aggregation period, such as host.
Tags	Multi-select; can be linked to global tags with monitors, generating corresponding alert notifications based on the aggregation period. If an event simultaneously contains multiple tag values, it will prioritize matching the corresponding alert notification according to the tag order configured on the page, and the relationship between multiple tag values is OR.

Intelligent Aggregation: Under this mode, events generated within the aggregation period will be grouped by clustering based on the selected title or content, with each group generating a single alert notification.

AI Aggregation: Use Guance's large model to merge monitoring alerts, reducing redundancy and avoiding a large number of repeated alerts in a short time.

Aggregation Period¶

Under the rule-based aggregation and intelligent aggregation modes, you can manually set a time range (1-30 minutes).

Within this time range, newly added events will be aggregated into a single alert notification. If this aggregation period expires, newly occurred events will be aggregated into a new alert notification.

Operational Permissions¶

After setting the operational permissions for alert strategies, roles, team members, and workspace users in your current workspace will perform corresponding operations on alert strategies based on assigned permissions. This ensures that different users carry out operations consistent with their roles and permission levels.

• Not enabling this configuration: Follow the default permissions for "alert strategy configuration management";
• Enabling this configuration and selecting custom permission objects: Only the creator and the objects granted permissions can enable/disable, edit, or delete the rules set for this alert strategy;
• Enabling this configuration but not selecting custom permission objects: Only the creator has the permissions to enable/disable, edit, or delete this alert strategy.

More Reading¶