Log Detection¶

The log detection feature is used to monitor all log data generated by the log collector within the workspace. It supports setting up alert rules based on log keywords to quickly identify abnormal patterns that do not conform to expected behavior, such as abnormal tags appearing in the log text. This allows for timely detection and response to potential security threats or system issues.

Use Cases¶

Mostly applicable to IT monitoring scenarios such as code exceptions or task scheduling detection. For example, monitoring when the log error rate is too high.

Detection Configuration¶

Detection Frequency¶

This refers to the execution frequency of the detection rule; the default selection is 5 minutes.

Detection Interval¶

This refers to the time range for querying metrics each time a task is executed. Affected by the detection frequency, the selectable detection intervals will vary.

Detection Metrics¶

Monitor the number of logs containing the set keyword that appear in the specified log list over a certain period of time.

Field	Description
Indices	The index to which the current detection metric belongs. After setting the index under Logs > Indices, when selecting "Logs" as the data source for chart queries, you can choose the log content corresponding to different indices. By default, it is the default index.
Source	The data source for the current detection metric, supporting selection of all (*) or specifying a single data source.
Keyword Search	Supports keyword search
Filter Conditions	Filters the data of the detection metric based on the labels of the metric, limiting the scope of detected data; supports adding one or more label filters; supports fuzzy matching and fuzzy non-matching filter conditions.
Aggregation Algorithm	Default selected is “*”, corresponding function is count. If another field is selected, the function automatically changes to count distinct (counts the number of data points where the keyword appears).
Detection Dimensions	Any string type (keyword) fields in the configured data can be chosen as detection dimensions, with currently up to three fields supported for selection. By combining multiple detection dimension fields, a specific detection object can be determined, and the system will determine whether the statistical metric for a given detection object meets the threshold for triggering an event. If it does meet the condition, an event is generated. * (For example, if the detection dimensions host and host_ip are selected, then the detection object could be {host: host1, host_ip: 127.0.0.1}. When the detection object is "log", the default detection dimensions are status, host, service, source, filename.)
Query Method	Supports simple query and expression query. If the query method is an expression query and involves multiple queries, the log detection object remains the same. For instance, if the detection object of expression query A is "log", then the detection object of expression query B is also "log".

Trigger Conditions¶

Set trigger conditions for alert levels: You can configure any one of the following trigger conditions: urgent, critical, warning, normal.

Configure the trigger conditions and severity level. When the query result contains multiple values, any value that meets the trigger condition will generate an event.

For more details, refer to Event Level Description.

If continuous trigger judgment is enabled, you can configure the conditions to take effect after multiple consecutive judgments, triggering the generation of events again. The maximum limit is 10 times.

Data Gap¶

You can configure seven strategies for data gap states.

1. Linking with the detection interval time range, judge the query results of the most recent minutes for the detection metric, do not trigger events;

2. Linking with the detection interval time range, judge the query results of the most recent minutes for the detection metric, treat the query result as 0; at this point, the query result will be compared again with the thresholds configured in the trigger conditions above to determine whether to trigger an anomaly event.

3. Customize the detection interval value, trigger data gap events, trigger urgent events, trigger critical events, trigger warning events, and trigger recovery events; when choosing this type of configuration strategy, it is recommended that the custom data gap time be >= detection interval time. If the configured time <= detection interval time, there may be simultaneous satisfaction of data gaps and anomalies, in which case only the data gap processing results will apply.

Information Generation¶

Enabling this option will generate "information" events for detection results that do not match the above trigger conditions and write them into the system.

Other Configurations¶

For more details, refer to Rule Configuration.