Create Alert Strategy¶
Create¶
- Define the name of the current alert strategy;
- Input a description for the strategy as needed;
- Select the associated monitors;
- Choose the notification time zone;
- Select the alert strategy that triggers notifications based on level or member;
- Choose the time range for repeated alerts (original alert silence);
- Optionally select the alert aggregation mode to determine the final aggregation form of alert notifications;
- Optionally add operation permissions to the strategy rules;
- Save to create successfully.
Association¶
On the configuration page, you can click to select the monitoring rules associated with the current alert strategy, including:
- All
- Monitors
- Intelligent Monitoring
- SLO
- Security Monitoring
Here, you can quickly create new monitoring rules as needed.
Set Notification Rules¶
Configuration Notes
- Recovery Notification: When a previously sent abnormal alert event recovers, the system will send a recovery notification to the corresponding notification target. For example: if a
critical
notification for a related event was sent to a group, when this status starts to recover, a recovery notification will be sent to this group. - Notification Delay: Alert notifications are not sent immediately after they are generated; there may be a delay of up to 1 minute due to data storage issues.
Currently, two types of notification configurations are supported: based on level and based on member.
The former sets the notification target for such abnormal events after selecting the event level. If a filter condition is set, the data range of events under a certain level is further limited, and finally, notifications are sent to the target.
The latter first selects members or teams, defines the range of event data they need to pay attention to or be responsible for, and then within this data range, delineates the event level and the corresponding notification target, achieving a strong association between events and targets.
Level-based Notification Configuration¶
Define the notification targets for alerts at each level.
-
Select Event Level.
- One event level can be selected multiple times;
- Based on the selected event level, you can link alert aggregation.
-
Select the notification target for events at this level.
Type |
Description |
---|---|
Workspace Members | Email notification; can be viewed under Management > Member Management. |
Teams | Email notification; a team can add multiple workspace members, can be viewed under Management > Member Management > Team Management. |
Email Groups | Email notification; an email group can add multiple teams, can be viewed under Monitoring > Notification Targets. |
DingTalk/WeCom/Lark Bots | Group notification; can be viewed under Monitoring > Notification Targets. |
Webhook Custom | User-defined; can be viewed under Monitoring > Notification Targets. |
SMS | SMS notification; an SMS group can add multiple workspace members, can be viewed under Monitoring > Notification Targets. Free Plan does not support SMS notifications, other versions charge 0.1 yuan per SMS, billed daily, with no free quota. |
Custom External Email | Input email and press Enter; only available for Commercial Plan and Deployment Plan users. |
Member-based Notification Configuration¶
Configuring notification rules based on members enables precise point-to-point alert notifications. At the same time, in one alert rule, different notification scopes, levels, and methods can be configured for multiple groups of members, and custom notification time ranges can be set for multiple groups of members.
- Define the name of the notification rule;
- Select the members and teams to be notified;
- Add filter conditions to achieve tag matching;
- For the filtered event data, you can set the corresponding notification targets for different event levels;
- Optionally enable the custom notification time range configuration.
Configuration Notes
- Hover to quickly reuse existing member notification configurations;
- If you configure multiple custom notification time ranges, the system will match them in order from top to bottom and only use the first matching time range's notification rule to send alerts.
Add Filter Conditions¶
Whether configuring notifications based on level or member, adding specific filter conditions can:
- For level-based notifications, further refine the data range of specific level events;
- For member-based notifications, limit members or teams to only pay attention to events that match specific tags.
After adding filters, only events that meet the level requirements and filter conditions will trigger notifications.
After clicking the filter button, the system will automatically fetch the fields of the current workspace and set filter conditions in the form of key:value
. You can choose the following matching methods: equal, not equal, wildcard, wildcard negation, and regular expression matching. Multiple filter conditions for the same key
field are in OR relationship, and filter conditions for different key
fields are in AND relationship.
You can configure filter conditions in the following two ways:
- Directly select fields and set conditions on the page.
- Write regular expressions to achieve more complex filtering logic, meeting the needs of fine-grained configuration.
Configuration Notes
- Only one set of filter conditions can be added under each alert rule, and one set of conditions can contain one or more filter rules. The system will combine all rules for condition filtering;
- Filter conditions cannot be empty.
Alert Escalation Notification¶
If a monitor frequently detects abnormalities of the same level in a short period of time, it may indicate a persistent problem. At this time, other notification targets may be needed to solve such problems. You can adopt the method of adding escalation notification rules. In this way, when abnormalities persist, the system will automatically escalate them to urgent notifications and send them to designated recipients, ensuring that the problem can be noticed and handled in a timely manner.
If two escalation notifications are configured in one notification rule, then:
- When alerts of the same level continue to occur, the system will check the time interval to determine whether to send the first escalation notification;
- After sending the first escalation notification, the system will determine whether to send the second escalation notification based on the time interval configured in the second escalation notification.
Configuration Notes
-
Each notification rule supports up to two escalation notifications;
-
Each escalation notification is triggered only once, and there will be no repeated alerts.
Custom Notification Time¶
The above scenarios mainly focus on the immediacy of automatically triggering notifications when abnormalities are detected. In fact, you can also set specific times for sending notifications as needed.
- Modify the configuration name as needed;
-
Divide the event cycle based on day, week, month, and custom dimensions;
- If custom is selected, a CSV file needs to be uploaded, and the system will automatically fill in the dates according to the file. The date format in the file should be
year/month/day
&YYYY/MM/DD
; the number of dates in the file should not exceed 365.
- If custom is selected, a CSV file needs to be uploaded, and the system will automatically fill in the dates according to the file. The date format in the file should be
-
Limit the time when events occur on the day according to the cycle, and send notifications according to the selected time interval, such as selecting
09:00 - 10:00
. When the strategy takes effect, abnormal events generated within this hour will match and flow into this custom configuration; -
After completing the cycle and time-related configurations, you can select the alert level and notification target.
Configuration Notes
-
In a single custom notification configuration in the same alert strategy, if multiple rules are configured, the generated abnormal events will be matched in order from top to bottom, and alerts will be sent according to the first matching custom configuration. If no rule is matched, no notification will be sent;
-
When configuring monitors, if multiple alert strategies are selected, after the monitor is enabled, the generated abnormal events will match the selected alert strategies respectively.
Repeated Alerts¶
After setting repeated alert notifications, within a certain time range, event data will continue to be generated, but alert notifications will not be sent again, and the generated data records will be stored in the event viewer.
Configuration Notes
If you select the 【Permanent】 option for repeated alerts, the system will only send the first alert notification and will not send repeated notifications thereafter.
Set Notification Aggregation Rules¶
No Aggregation¶
Default configuration; in this mode, alert events will be merged into one notification every 20 seconds and sent to the corresponding notification target;
Rule Aggregation¶
In this mode, you can choose one of the following four aggregation rules and send alert notifications based on the aggregation cycle:
Aggregation Rule |
Description |
---|---|
All | Based on the level dimension configured in the alert strategy, generate corresponding alert notifications within the selected aggregation cycle. |
Monitors/Intelligent Inspection/SLO | According to the detection rules of monitors or intelligent inspection or the unique ID of SLO, link the aggregation cycle to generate corresponding alert notifications. |
Detection Dimension | According to the detection dimension, link the aggregation cycle to generate corresponding alert notifications, such as host . |
Tags | Multiple selections; can link global tags with monitors, and generate corresponding alert notifications according to the aggregation cycle. |
Intelligent Aggregation¶
In this mode, events generated within the aggregation cycle will be clustered into groups based on the selected title
or content
, and each group will generate one alert notification.
AI Aggregation¶
Using Guance's large model, new events can be aggregated into one alert within the set number of minutes, and a new alert will be automatically generated after the timeout, avoiding repeated disturbances.
Aggregation Cycle¶
In rule aggregation and intelligent aggregation modes, you can manually set a time range (1-30 minutes).
Within this time range, new events will be aggregated into one alert notification and sent. If it exceeds this aggregation cycle, new events will be aggregated into a new alert notification.
Set Operation Permissions¶
After setting the operation permissions of the alert strategy, the roles, team members, and workspace users of your current workspace will perform corresponding operations on the alert strategy according to the assigned permissions. This ensures that different users perform operations in accordance with their roles and permission levels.
- Do not enable this configuration: follow the default permissions of "Alert Strategy Configuration Management";
- Enable this configuration and select custom permission objects: only the creator and the objects granted permissions can enable/disable, edit, and delete the rules set in this alert strategy;
- Enable this configuration but do not select custom permission objects: only the creator has the permissions to enable/disable, edit, and delete this alert strategy.
Configuration Notes
The Owner role of the current workspace is not affected by the operation permission configuration here.