Log Detection¶
Document Scope
This document is the second step in the detection rule configuration workflow. After configuration is complete, return to the main document to continue to Step 3: Event Notification.
The Log Detection feature monitors all log data generated by log collectors in the workspace. It supports setting alert rules based on log keywords to quickly identify anomalous patterns that deviate from expected behavior (such as abnormal tags appearing in log text or excessively high error rates), helping you promptly detect and respond to potential security threats or system issues.
It is suitable for code exceptions or task scheduling detection in IT monitoring scenarios, for example, monitoring excessively high log error rates.
Detection Configuration¶
Detection Frequency¶
Set the time period for executing detection.
-
Preset options: 1 minute, 5 minutes (default), 15 minutes, 30 minutes, 1 hour, 6 hours, 12 hours, 24 hours;
-
Crontab mode: Click Switch to Crontab Mode to configure a custom schedule. Schedules based on seconds, minutes, hours, days, months, weeks, and other periods are supported for scheduled task execution.
Detection Interval¶
Set the data time range queried for each detection (❗️The detection interval should be greater than or equal to the detection frequency and must match the actual data reporting cycle to avoid missed or false alerts).
- Preset options:
| Detection Frequency | Detection Interval (drop-down options) |
|---|---|
| 1m | 1m/5m/15m/30m/1h/3h |
| 5m | 5m/15m/30m/1h/3h |
| 15m | 15m/30m/1h/3h/6h |
| 30m | 30m/1h/3h/6h |
| 1h | 1h/3h/6h/12h/24h |
| 6h | 6h/12h/24h |
| 12h | 12h/24h |
| 24h | 24h |
- Custom format: Enter a custom detection interval, for example, 20m (last 20 minutes), 2h (last 2 hours), 1d (last 1 day).
Detection Metric¶
Define the detection data source and aggregation method based on DQL to monitor the number of logs containing the configured keywords in the log list of a specified detection object within a certain time range (❗️Avoid selecting high-cardinality fields as detection dimensions. If configured improperly and trigger conditions are too loose, frequent alerts may be triggered. The current maximum number of returned records is 100,000).
Configuration Elements¶
| Configuration Item | Description |
|---|---|
| Index | The index to which the current detection metric belongs; multiple selections are supported. ❗️ After indexes are set in Logs > Indexes, when "Logs" is selected as the data source for chart queries, you can select the log content corresponding to different indexes. The default is index default |
| Source | The data source of the current detection metric. You can select all (*) or a single specified data source |
| Keyword Search | Supports keyword search to match specific content in log text |
| Filter Conditions | Filter the detection metric data based on metric tags to limit the data range; supports adding one or more tag filters; supports fuzzy match and fuzzy non-match filter conditions |
| Aggregation Algorithm | "*" is selected by default, and the corresponding function is count (counts the number of logs). If another field is selected, the function automatically changes to count distinct (counts distinct data points) |
| Detection Dimensions | All string-type (keyword) fields in the data can be selected as detection dimensions. Currently, up to three fields can be selected as detection dimensions. By combining multiple detection dimension fields, a specific detection object can be determined. The system determines whether the statistical metric corresponding to a detection object meets the trigger condition threshold and generates an event if it does.(For example, if host and host_ip are selected as detection dimensions, the detection object can be {host: host1, host_ip: 127.0.0.1}.)❗️When the detection object is "Logs", the default detection dimensions are status, host, service, source, and filename |
| Query Method | Supports simple queries and expression queries. ❗️If the query method is expression query and contains multiple queries, the log detection object is the same. For example, if the detection object of expression query A is "Logs", the detection object of expression query B is also "Logs" |
Click to view Query Method Details.
Trigger Condition¶
Configure trigger conditions and severity levels. When the query result has multiple values, an event is generated if any value meets the trigger condition.
You can configure Fatal, Critical, Major, Warning thresholds and a Normal recovery condition.
| Level | Configuration | Description |
|---|---|---|
| Fatal | When Result >= [value] |
Highest-level alert, requiring immediate action |
| Critical | When Result >= [value] |
High-severity alert, requiring priority handling |
| Major | When Result >= [value] |
Medium-severity alert, requiring attention |
| Warning | When Result >= [value] |
Low-severity alert, requiring monitoring |
| Normal | No event generated after [N] detections |
After the detection rule takes effect, if the data detection result recovers from an abnormal state (Fatal, Critical, Major, Warning) to normal within the configured number of custom detections, a recovery alert event is triggered. ❗️ Recovery alert events are not subject to alert muting. If the number of detections for recovery alert events is not set, the alert event will not recover and will remain in Events > Unrecovered Events List |
For more information, see Event Severity Levels.
Consecutive Trigger Evaluation¶
When enabled, an event is generated only when the trigger condition is continuously met, preventing false alerts from transient fluctuations (❗️the maximum configuration is 10 times).
High-Volume Alert Protection¶
Enabled by default.
When the number of alerts generated by a single detection exceeds the preset threshold, the system automatically switches to a status-based aggregation strategy: alerts are no longer processed individually; instead, a small number of summary alerts are generated and pushed based on event status.
This ensures timely notifications while significantly reducing alert noise and avoiding timeout risks caused by processing too many alerts.
When this switch is enabled, such event details generated after the monitor subsequently detects an anomaly will not display history or related events.
No Data¶
Processing strategy when the detection metric query result is empty within the detection interval:
| Option | Description |
|---|---|
| Do not trigger events (default) | No event is triggered when there is no data, suitable for scenarios where data absence is allowed. |
| Treat query result as 0 | Treat empty data as 0 and continue to determine whether to generate an abnormal event according to the configured trigger conditions. |
| Trigger No Data event | Generate one No Data event per round of data absence for the same object. If data is absent again after recovery, a new event can be generated. |
| Continuously trigger No Data events | When the same object continuously meets the no-data condition, No Data events are generated at the detection frequency until data recovers. For configuration, see No Data. |
| Trigger Fatal event | Trigger a Fatal-level event when there is no data. |
| Trigger Critical event | Trigger a Critical-level event when there is no data. |
| Trigger Major event | Trigger a Major-level event when there is no data. |
| Trigger Warning event | Trigger a Warning-level event when there is no data. |
| Trigger Recovery event | Trigger a Recovery event when there is no data. |
After selecting an event trigger action, enter the no-data duration. It is recommended to set it to ≥ the detection interval duration. When both the no-data condition and an abnormal condition are met, the system prioritizes the no-data handling result.
When trigger conditions, No Data, and info event generation are configured at the same time, triggering is evaluated in the following priority order: No Data > Trigger Condition > Info Event Generation.
That is, it first determines whether data is absent, then whether a threshold is triggered, and finally whether to generate an info event.
Info Event Generation¶
After this option is enabled, configure the Info Event Generation Conditions. The system writes an "Info" event only when the detection result does not trigger any of the Fatal, Critical, Major, or Warning thresholds and meets the info event generation conditions.
This is suitable for scenarios where normal state changes or low-priority information need to be recorded.
Next Steps¶
After completing the above detection configuration, continue with the following:
-
Event Notification: Define the event title, content, notification members, No Data handling, and related incidents;
-
Alert Configuration: Select an alert policy and set notification targets and mute periods;
-
Associate: Associate dashboards for quick navigation to view data;
-
Permissions: Set operation permissions to control who can edit/delete this monitor.