Detection Rules¶
The system has built-in rich detection rules that can accurately match the monitoring needs of various types of data, effectively avoiding false alarms and missed alarms.
Rule Types¶
Rule Name |
Data Scope |
Basic Description |
|---|---|---|
| Threshold Detection | All | Performs anomaly detection on metric data based on set thresholds |
| Mutation Detection | Metrics(M) | Performs anomaly detection on sudden abnormal behavior of metrics based on historical data, often suitable for business data and short time window scenarios |
| Interval Detection | Metrics(M) | Detects abnormal data points of metrics based on a dynamic threshold range, often suitable for stable trend time series |
| Interval Detection V2 | Metrics(M) Traces(T) RUM Data(R) |
Detects abnormal data points of metrics based on a dynamic threshold range, often suitable for stable trend time series |
| Outlier Detection | Metrics(M) | Detects whether metrics/statistical data of detected objects under specific groups have outlier deviations |
| Log Detection | Logs(L) | Performs anomaly detection on business applications based on log data |
| Process Anomaly Detection | Process Objects(O::host_processes) |
Periodically checks process data to understand process anomalies |
| Infrastructure Liveness Detection V2 | Objects(O) | Sets liveness conditions based on infrastructure object data to monitor infrastructure stability |
| Application Performance Metric Detection | Traces(T) | Sets threshold rules based on APM data to detect anomalies |
| RUM Metric Detection | RUM Data(R) | Sets threshold rules based on RUM data to detect anomalies |
| Composite Detection | All | Combines the results of multiple monitors into one monitor through expressions, and alerts based on the combined result |
| Synthetic Testing Anomaly Detection | Synthetic Data(D::Type) |
Sets threshold rules based on Synthetic Tests data to detect anomalies |
| Network Data Detection | Network(N) | Sets threshold rules based on network data to detect network performance stability |
| Third-Party Event Detection | Others | Generates event data by sending abnormal events or records produced by third-party systems to an HTTP server via a POST request to a specified URL address |
| Infrastructure Change Detection | Objects(O) | Monitors various change behaviors based on tracking the infrastructure lifecycle, accurately identifying abnormal conditions such as configuration drift and illegal operations |
| Programmable Detection | All | Writes detection rules via scripts, often suitable for monitoring scenarios with complex and frequently changing rules |
Rule Configuration¶
Detection Configuration¶
Set corresponding detection frequency, detection interval, detection metrics, and other information for different detection rules.
Event Notification¶
Event Title¶
Defines the event name for the alarm trigger condition. Predefined template variables can be used.
Note
In the latest version, the monitor name will be generated synchronously after the event title is entered. In older monitors, there might be inconsistencies between the monitor name and the event title. It is recommended to synchronize to the latest version.
Event Content¶
Write the event notification content. When the trigger condition is met, the system will send this content externally. It generally includes the following information:
- Body text in Markdown format.
- Ability to insert related links and template variables.
- Adding associated logs or error information based on Advanced Configuration.
- Target notification members to whom the event content is sent.
Note
The @ member configuration only takes effect and sends the event content here to the specified members when associating with an Incident is enabled.
Related Links¶
The monitor automatically generates jump links based on the detection metrics in the Detection Configuration. You can adjust the filter conditions and time range after inserting the link. It is generally a fixed link address prefix containing the current domain name and workspace ID. You can also choose to customize the jump link.
Among these, if you need to insert a link to jump to a dashboard, based on the above logic, you also need to supplement the dashboard ID and name, and adjust the view variables and time range as needed.
Custom Advanced Configuration¶
Through advanced configuration, you can add associated logs or error stacks to the event content to view the contextual data when the anomaly occurred.
- Add associated logs:
Query:
For example: Get one log message with index default:
Associated log:
- Add associated error stack
Query:
{% set dql_data = DQL("T::re(`.*`):(`error_message`,`error_stack`){ (`source` NOT IN ['service_map', 'tracing_stat', 'service_list_1m', 'service_list_1d', 'service_list_1h', 'profile']) AND (`error_stack` = exists()) } LIMIT 1") %}
Associated error stack:
Custom Notification Content¶
By default, the system uses the Event Content as the alarm notification content. If you need to customize the actual notification sent externally, you can choose to enable the switch here and fill in the notification information.
Note
Different alarm notification targets support different Markdown syntax. For example, WeCom does not support unordered lists.
Data Gap Events¶
This refers to customizing the notification content for data gaps. You can synchronously configure the title, content, and other information of such events that will ultimately be sent externally.
If not configured here, the official default notification template will be automatically used when finally sent externally.
Associate Incident¶
After enabling association, if an abnormal event is generated under this monitor, an Issue will be created synchronously. You can choose to create Issues synchronously for different event severities.
- Select event severity.
- Define the severity of the ultimately generated Issue.
- Select the responsible person for this type of Issue.
- Select the delivery channel.
- Optionally choose whether to close the Issue synchronously after the event recovers.
The Issues generated here can be viewed in Incident > the channel you selected.
Alert Configuration¶
Once the monitoring trigger conditions are met, immediately send alert messages to the specified notification targets. The alert strategy includes the event severity level to be notified, the notification targets, and the alert silence period.
Alert strategies support single or multiple selection. Click the strategy name to expand the details page. Click Edit Alert Strategy to modify the strategy.
Association¶
Supports associating the monitor with dashboards for quick jumping and visual viewing of monitoring data.
Permissions¶
Sets the operational permissions for the monitor to ensure different users perform compliant configuration operations according to their roles and permission levels.
- Do not enable this configuration: Follow the default permissions of "Monitor Configuration Management".
- Enable this configuration and select custom permission objects: Only the creator and the assigned permission objects can perform enable/disable, edit, and delete operations on the rules set by this monitor.
- Enable this configuration, but do not select custom permission objects: Then only the creator has the enable/disable, edit, and delete permissions for this monitor.
Note
The Owner role of the current workspace is not affected by the operational permission configuration here.
Trigger Detection Now¶
After the rule configuration is completed, you can choose to trigger detection immediately to test the overall effect of the current rule configuration.