Detection Rules¶
The system comes with a rich set of built-in detection rules, which can accurately match the monitoring needs of various data types, effectively avoiding false alarms and missed alarms.
Rule Types¶
Rule Name |
Data Scope |
Description |
|---|---|---|
| Threshold Detection | All | Performs anomaly detection on metric data based on set thresholds. |
| Mutation Detection | Metrics(M) | Detects anomalies based on sudden反常 behavior of metrics against historical data. Often used for business data and short time windows. |
| Interval Detection | Metrics(M) | Detects anomalous data points in metrics based on a dynamic threshold range. Best suited for stable trend time series. |
| Interval Detection V2 | Metrics(M) Traces(T) RUM Data(R) |
Detects anomalous data points in metrics based on a dynamic threshold range. Best suited for stable trend time series. |
| Outlier Detection | Metrics(M) | Detects whether metrics/statistical data of detected objects under specific groupings have outlier deviations. |
| Log Detection | Logs(L) | Performs anomaly detection for business applications based on log data. |
| Process Anomaly Detection | Process Objects(O::host_processes) |
Periodically checks process data to understand process anomalies. |
| Infrastructure Liveness Detection V2 | Objects(O) | Monitors infrastructure stability by setting liveness conditions based on infrastructure object data. |
| Application Performance Detection | Traces(T) | Sets threshold rules based on APM data to detect anomalies. |
| Real User Monitoring Detection | RUM Data(R) | Sets threshold rules based on RUM data to detect anomalies. |
| Composite Detection | All | Combines the results of multiple monitors into one monitor using expressions, and triggers alerts based on the combined result. |
| Synthetic Testing Anomaly Detection | Synthetic Data(D::Type) |
Sets threshold rules based on Synthetic Tests data to detect anomalies. |
| Network Data Detection | Network(N) | Sets threshold rules based on network data to detect network performance stability. |
| Third-Party Event Detection | Others | Generates event data by sending异常 events or records from third-party systems to an HTTP server via a POST request to a specified URL. |
| Infrastructure Change Detection | Objects(O) | Tracks the infrastructure lifecycle to monitor various change behaviors, accurately identifying anomalies like configuration drift and unauthorized operations. |
| Programmable Detection | All | Writes detection rules via scripts, suitable for monitoring scenarios with complex and frequently changing rules. |
Rule Configuration¶
Detection Configuration¶
Set corresponding detection frequency, detection interval, detection metrics, etc., for different detection rules.
Event Notification¶
Event Title¶
Defines the event name for the alert trigger condition; pre-defined template variables can be used.
Note
In the latest version, the monitor name will be generated automatically after entering the event title. In older monitors, the monitor name and event title might be inconsistent. It is recommended to sync to the latest version.
Event Content¶
Write the event notification content. When the trigger condition is met, the system will send this content externally. It generally includes the following information:
- Body text in Markdown format;
- Can insert related links and template variables;
- Add associated logs or error information based on Advanced Settings;
- Target notification members to whom the event content is sent.
Note
The @ member configuration only takes effect and sends the event content here to specified members when Associate Incident is enabled.
Related Links¶
The monitor automatically generates jump links based on the detection metrics in the Detection Configuration. You can adjust filter conditions and time range after inserting the link. It is generally a fixed link address prefix containing the current domain name and workspace ID; you can also choose to customize the jump link.
Among them, if you need to insert a link to a dashboard, based on the above logic, you also need to supplement the dashboard ID and name, and adjust view variables and time range as needed.
Custom Advanced Settings¶
Through advanced settings, you can add associated logs or error stacks to the event content to view the contextual data when the anomaly occurred.
- Add associated logs:
Query:
e.g., Get one log message with index default:
Associated log:
- Add associated error stack
Query:
{% set dql_data = DQL("T::re(`.*`):(`error_message`,`error_stack`){ (`source` NOT IN ['service_map', 'tracing_stat', 'service_list_1m', 'service_list_1d', 'service_list_1h', 'profile']) AND (`error_stack` = exists()) } LIMIT 1") %}
Associated error stack:
Custom Notification Content¶
By default, the system uses the Event Content as the alert notification content. If you need to customize the actual notification sent externally, you can enable the switch here and fill in the notification information.
Note
Different alert notification objects support different Markdown syntax. For example, WeCom does not support unordered lists.
Data Gap Events¶
i.e., customize the notification content for data gaps. You can synchronously configure the final title, content, etc., of such events when sent externally.
If not configured here, the official default notification template will be used automatically when finally sent externally.
Associate Incident¶
After enabling association, if an abnormal event is generated under this monitor, an Incident will be created同步ly. You can choose to create incidents同步ly for different event severities.
- Define tags for the incident;
- Select the event severity;
- Define the severity of the最终 resulting incident.
The incidents generated here can be viewed in the Incident Center (❗️Such incidents include tag filter conditions).
Alert Configuration¶
When the monitoring trigger condition is met, immediately send an Alert message to the specified notification targets. The alert strategy includes the event severity to be notified, the notification targets, and the alert silence period.
Alert strategies support single or multiple selection. Click the strategy name to expand the details page. Click Edit Alert Strategy to modify the strategy.
Associate¶
Supports associating the monitor with dashboards for quick jumping and visual viewing of monitoring data.
Permission¶
Sets the operational permissions for the monitor to ensure different users perform compliant configuration operations according to their roles and permission levels.
- Do not enable this configuration: Follow the default permissions of "Monitor Configuration Management";
- Enable this configuration and select custom permission objects: Only the creator and the assigned permission objects can enable/disable, edit, and delete the rules set by this monitor;
- Enable this configuration, but no custom permission objects are selected: Then only the creator has the permissions to enable/disable, edit, and delete this monitor.
Note
The Owner role of the current workspace is not affected by the operational permission configuration here.
Trigger Detection Now¶
After the rule configuration is completed, you can choose to trigger detection immediately to test the overall effect of the current rule configuration.