Synthetic Testing Anomaly Detection¶
Document Positioning
This document is the second step in the detection rule configuration process. After completing the configuration, return to the main document to continue with Step 3: Event Notifications.
Used to monitor Synthetic Monitoring (dial test) data within the workspace. You can set a threshold range for a specified volume of data generated by Synthetic Tests within a time period. Once the data volume reaches these thresholds, the system triggers an alert. Supports monitoring based on performance metrics and count statistics for protocol types such as HTTP, TCP, ICMP, WEBSOCKET, and Multistep Tests.
Applicable to scenarios such as monitoring URL availability, service port connectivity, and network latency in production environments. For example, monitoring whether the average response time or availability rate of critical business APIs meets the expected target.
Detection Configuration¶
Detection Frequency¶
Set the time period for executing detection.
-
Preset options: 1 minute, 5 minutes (default), 10 minutes, 15 minutes, 30 minutes, 1 hour, 6 hours, 12 hours, 24 hours
-
Crontab mode: Click "Switch to Crontab Mode" to configure a custom period. Supports scheduled task execution based on periods such as seconds, minutes, hours, days, months, and weeks.
Detection Range¶
Set the data time range queried for each detection. (❗️The detection range must be greater than or equal to the detection frequency, and must match the actual data reporting period to avoid missed detections or false alarms.)
| Detection Frequency | Detection Range (Dropdown Options) |
|---|---|
| 30s | 1m/5m/15m/30m/1h/3h |
| 1m | 1m/5m/15m/30m/1h/3h |
| 5m | 5m/15m/30m/1h/3h |
| 15m | 15m/30m/1h/3h/6h |
| 30m | 30m/1h/3h/6h |
| 1h | 1h/3h/6h/12h/24h |
| 6h | 6h/12h/24h |
| 12h | 12h/24h |
| 24h | 24h |
- Custom format: Enter a custom detection range, e.g., 20m (last 20 minutes), 2h (last 2 hours), 1d (last 1 day).
Detection Metrics¶
Set the metrics for detection data. You can set the specified data generated by all or individual Synthetic Tests in the current workspace as detection metrics. (❗️Avoid selecting high-cardinality fields as detection dimensions. Improper configuration with overly loose trigger conditions may cause frequent alerts. The maximum number of records returned by a query is 100,000.)
Two query modes are supported:
-
Dial test metrics (based on specific performance metrics)
-
Count statistics (based on data source queries)
Dial Test Metrics¶
Performs detection based on specific performance metrics of Synthetic Monitoring.
| Configuration Item | Description |
|---|---|
| Dial Test Type | Includes protocol types such as HTTP Dial Test, TCP Dial Test, ICMP Dial Test, WEBSOCKET Dial Test, and Multistep Tests |
| Dial Test Address | Supports monitoring all or individual Synthetic Tests in the current workspace via Synthetic Monitoring; retrieves the corresponding Synthetic Test list based on the selected dial test type |
| Metrics | Supports detection based on metric dimensions, including: average response time, P50 response time, P75 response time, P90 response time, P99 response time, availability rate, number of error requests, number of requests, etc. |
| Detection Dimension | Any string-type (keyword) field in the configured data can be selected as a detection dimension. Currently, a maximum of three fields can be selected as detection dimensions. By combining multiple detection dimension fields, you can identify a specific detection object. The system evaluates whether the statistical metrics corresponding to a detection object meet the trigger condition thresholds, and generates an event if the conditions are met.(For example, if detection dimensions host and host_ip are selected, the detection object can be {host: host1, host_ip: 127.0.0.1}.) |
| Filter Conditions | Filters the data of detection metrics based on metric tags to limit the detection data range; supports adding one or more tag filters; supports fuzzy match and fuzzy non-match filter conditions |
Count Statistics¶
Performs statistical detection based on dial test data sources.
You can query and count Synthetic Tests based on different dial test types by selecting the corresponding data sources (http_dial_testing, tcp_dial_testing, icmp_dial_testing, websocket_dial_testing, multi_dial_testing, etc.).
Supports limiting the detection scope through keyword search or tag filtering.
In addition to simple queries, expression-based queries are also supported.
Additional Information¶
Additional fields are used only for supplementary queries and are not used for trigger condition evaluation. You can configure them into event notifications. If multiple matching values are detected, one record is returned at random.
Trigger Conditions¶
Configure trigger conditions and severity levels. When the query returns multiple values, an event is generated if any value satisfies the trigger condition.
Supports configuring four threshold levels — Critical, Error, Warning, Info — and the OK recovery condition.
| Severity | Configuration | Description |
|---|---|---|
| Critical | When Result >= [value] |
Highest severity alert; requires immediate handling |
| Error | When Result >= [value] |
High severity alert; requires priority handling |
| Warning | When Result >= [value] |
Medium severity alert; requires attention |
| Info | When Result >= [value] |
Low severity alert; requires awareness |
| OK | No events generated for [N] detections |
After the detection rule takes effect, if the data detection result returns to normal from abnormal (Critical, Error, Warning, Info) within the configured custom detection count, a recovery alert event is triggered. ❗️ Recovery alert events are not restricted by Alert Muting. If no recovery alert event detection count is configured, the alert event will not recover and will continue to appear in the Events > Unrecovered Events list |
For more details, refer to Event Level Descriptions.
Advanced Options¶
Consecutive Trigger Evaluation¶
When enabled, an event is generated only when the trigger condition is continuously satisfied, preventing false alarms caused by transient fluctuations. (❗️The maximum configurable limit is 10 times.)
Bulk Alert Protection¶
Enabled by default.
When the number of alerts generated by a single detection exceeds the preset threshold, the system automatically switches to a status-based aggregation strategy: instead of processing alert objects one by one, it generates and pushes a small number of summary alerts based on event status.
This ensures timely notification delivery while significantly reducing alert noise and avoiding the risk of timeouts caused by processing too many alerts.
When this switch is enabled, the event details generated after the monitor detects anomalies will not display historical records or associated events.
Data Gap¶
Handling strategy when the detection metric returns no query results within the detection range:
| Option | Description |
|---|---|
| Do not trigger events (default) | Using the detection range time window, determines whether to generate events based on the query results of the detection metric over the recent minutes. Suitable for scenarios where missing data is acceptable |
| Treat query results as 0 | Using the detection range time window, treats the query results of the detection metric over the recent minutes as 0, and compares them again against the thresholds configured in Trigger Conditions above to determine whether to trigger abnormal events |
| Custom fill and trigger events | Supports custom filling of detection range values and triggers the following event types respectively: Data gap events, Critical events, Error events, Warning events, Info events, and Recovery events. ❗️When selecting this strategy, it is recommended to configure the custom data gap duration ≥ the detection range interval. If the configured duration ≤ the detection range interval, data gap and anomaly conditions may both be satisfied, in which case the data gap handling result takes precedence |
When trigger conditions, data gap, and info generation are configured together, the trigger evaluation priority is as follows: Data gap > Trigger conditions > Info event generation.
That is: first determine whether there is a data gap, then determine whether thresholds are triggered, and finally determine whether an info event is generated.
Info Generation¶
When this option is enabled, you must configure the info generation condition. The system writes an "Info" event only when the detection result does not trigger any of the "Critical", "Error", "Warning", or "Info" thresholds and the info generation condition is satisfied.
Suitable for scenarios where normal status changes or low-priority information needs to be recorded.
Next Steps¶
After completing the above detection configuration, continue with:
-
Event Notification: Define the event title, content, notified members, data gap handling, and associated incidents;
-
Alert Configuration: Select an alert policy, set notification targets and the mute period;
-
Associate: Associate dashboards for quick navigation to view data;
-
Permissions: Set operation permissions to control who can edit or delete this monitor.