Detection Rules¶

Guance provides a rich set of built-in detection rules that precisely match the monitoring needs of various data types, effectively reducing false alarms and missed alerts.

Configuration Steps¶

To create a monitor, complete the configuration in the following order:

1. Select Rule Type: Determines the data scope and algorithm logic of the detection configuration;
2. Detection Configuration: Different types correspond to different configuration items;
3. Event Notification: Defines event title, content, associated information, and data gap handling;
4. Alert Configuration: Sets notification policy and silence period;
5. Association: Configure dashboard association;
6. Permissions: Set operational permissions.

Except for "Detection Configuration", which varies by rule type, the logic for Event Notification, Alert Configuration, Association, and Permissions is consistent across all rule types.

Rule Type¶

You can select the appropriate detection logic based on your monitoring objectives:

After selecting a rule type, the configurable parameters in the Detection Configuration section will change accordingly. Event Notification and subsequent configurations are consistent across all types.

Detection Configuration¶

You can configure the detection frequency, detection interval, and detection metrics according to the selected rule type.

Different rule types have significantly different detection configurations. Please refer to the detailed configuration documentation for the corresponding type.

Data Waiting Window¶

The data waiting window applies to all monitor types. Upon reaching the scheduled execution time, the monitor waits for the specified duration before performing the monitoring check, reducing the likelihood of false alarms caused by data latency.

• Default value: No Wait;
• Optional wait durations: 1 minute, 2 minutes, 3 minutes, 5 minutes, 10 minutes, 15 minutes, 30 minutes;
• The wait duration must be shorter than the interval between two consecutive scheduled execution times; otherwise, the monitor cannot be saved, and the message "Data waiting window must be less than the monitoring execution interval" will be displayed. This also applies to Crontab mode, validated against the interval between consecutive scheduled execution times.

For example, if a monitor is scheduled to run at 10:00 and the data waiting window is set to 5 minutes, the task will start approximately at 10:05. Waiting only delays the execution time and does not change the query time range; if the detection interval is the last 1 hour, the data queried will still be from 09:00 ~ 10:00.

For detailed execution logic, refer to Monitor Internals.

Event Notification¶

Defines the event title, content, notified members, and associated handling when the monitor is triggered.

Event Title¶

Defines the event name when the alert condition is met; preset template variables can be used.

Event Content¶

Write the event notification content, which will be sent externally when the trigger condition is met. Typically includes the following information:

• Markdown body text;
• Can insert associated links and template variables;
• Add associated logs or error information based on Advanced Settings;
• Target notified members for the event content.

Associated Links¶

Click + Link, and the system automatically generates jump links based on the current detection metrics. The link address includes the current domain, workspace ID, detection time range ({{df_check_range_start}} ~ {{df_check_range_end}}), and dynamic filter conditions.

Example link format:

Log Explorer: [View Related Logs](<{{STUDIO_CONSOLE_BASE_URL}}/logIndi/log/all?time={{df_check_range_start}},{{df_check_range_end}}&w={{df_workspace_uuid}}>)

Trace Explorer: [View Related Traces](<{{STUDIO_CONSOLE_BASE_URL}}/tracing/link/all?time={{df_check_range_start}},{{df_check_range_end}}&w={{df_workspace_uuid}}>)

Template Variables¶

Click + Variable to insert preset template variables. Variables are dynamically replaced with actual values when the event is triggered:

Click to view the complete list of template variables.

Advanced Settings¶

Through "Advanced Settings", you can use DQL to embed related data context within the event.

1. Add Related Logs

Clicking automatically generates the template:

{% set dql_data = DQL("L::RE(`.*`):(`message`) { `index` = 'default' } LIMIT 1") %}
{{ dql_data.message | limit_lines(10) }}

Configuration points:

• Replace {index= 'default' } with the actual index name

• RE(``.*``) supports regex matching, e.g., RE(``error\|exception``)

• limit_lines(10) limits the output lines to avoid excessively long notifications

2. Add Related Error Stack Trace

Clicking automatically generates the template:

{% set dql_data = DQL("T::re(`.*`):(`error_message`,`error_stack`){ (`source` NOT IN ['service_map', 'tracing_stat', 'service_list_1m', 'service_list_1d', 'service_list_1h', 'profile']) AND (`error_stack` = exists()) } LIMIT 1") %}
{{ dql_data.error_message | limit_lines(10) }}
{{ dql_data.error_stack | limit_lines(10) }}

Configuration points:

• source NOT IN [...] excludes statistical aggregated data, preserving only raw traces

• (error_stack= exists()) ensures only errors with stack trace information are returned

Notify Members (@)¶

Click to select workspace members.

Logic:

1. The @ Members configuration takes effect only when Associate Incidents is enabled, and the event content is sent to the specified members;
2. This configuration is independent of the notification targets in Alert Configuration and does not affect the alert notification scope.

Custom Notification Content¶

By default, the system uses the Event Content as the alert notification content. If you need to customize the actual notification sent externally, you can enable the toggle here and enter the notification information.

• Expands an independent editor where you can define the external notification content separately;
• The original event content remains in the platform for event details display;
• The independent editor also supports Markdown, template variables, associated links, and advanced settings.

Data Gap Events¶

Customizes the notification content when data is missing (no data reported). You can configure the title, content, and other information for the final sent notification of such events.

If not customized, the system uses the official default template to send data gap alerts.

Associate Incidents¶

When enabled, if an anomaly event occurs under this monitor, an Incident will be created simultaneously.

Configuration Items¶

1. Add tags to automatically created incidents for easy filtering in the Incident Center;

2. Configure the mapping between event severity and incident severity, supporting multiple rules.

   • When events of severity Critical/Error/Warning/Info/Data Gap occur, a new incident of severity P0/P1/P2/P3/... is created simultaneously.

Incidents created here can be viewed in the Incident Center (❗️incidents of this type include tag filter criteria).

Linkage Mechanism¶

1. When an event is triggered, an incident record is automatically created in the Incident Center, and the incident description synchronizes the event content;
2. A new incident notification is sent according to the @ Members list in the event content;
3. You can view incident details in the Incident Center, and the system automatically associates and displays all related data (performance metrics, error logs, call traces, infrastructure topology, etc.).

Alert Configuration¶

When the monitor condition is met, an alert message is immediately sent to the specified notification targets.

Alert Policy¶

Select an existing alert policy, multiple selections are supported. Click the Policy Name to expand and view details. Click Edit Alert Policy to modify the configuration:

Association¶

Select an existing dashboard to establish an association between the monitor and the dashboard, enabling quick navigation and visual monitoring of the data.

Permissions¶

Set operational permissions for the monitor to ensure different users can perform appropriate operations based on their roles and permission levels. The Owner of the current workspace is not affected by this permission configuration.

• Disabled: Follows the default permissions of "Monitor Configuration Management";
• Enabled and custom permission objects are selected: Only the creator and the granted objects can enable/disable, edit, and delete the rules set by this monitor;
• Enabled but no custom permission objects are selected: Only the creator has the permission to enable/disable, edit, and delete this monitor.

Additionally, the workspace Owner can use the Unified Permission Configuration to grant monitor operation permissions to administrators, roles, or members. The authorized objects can operate all monitors with custom operation permissions enabled, without needing to be added to each monitor's custom permission objects individually.

Trigger Detection Now¶

After the rule configuration is complete, click Trigger Detection Now to manually perform a test and verify the overall effect of the current rule configuration. The test execution does not generate actual alert notifications.

Alert Cache Protection Mechanism¶

After configuration is complete, the monitor executes the following protection strategies during operation. The system uses these mechanisms to prevent system pressure caused by an excessive number of detection objects from high-cardinality aggregation:

The alert cache limit is 100,000, with a warning threshold of 80%.

Suspension Recovery Mechanism¶

When a monitor is paused by the system due to exceeding the alert cache limit, the monitor will automatically recover after performing any of the following operations:

• Modify the query conditions and re-save the monitor

• Directly re-save the monitor

The system automatically clears the alert cache marker and restores the monitor to normal operation, without requiring additional actions.

Further Reading¶

After successfully creating a monitor rule, you may need to: