Generate Metrics¶

Generate new metric data from existing data within the current workspace, allowing you to design and implement new technical metrics based on actual business needs.

Important Notes¶

1. Only roles with the "Generate Metrics Configuration Management" permission can create and edit metrics.

2. After metrics are generated, they will be stored according to the current default data storage policy, and charges will be incurred based on the number of generated time series.

3. If no data is reported after generating metrics, they cannot be queried or analyzed within the workspace.

What Problems Can It Solve?¶

Scenario 1: Extract Business Metrics from Logs

Business logs contain key business data (e.g., order amounts, error codes) that need to be transformed into monitorable metrics. By generating metrics, you can select a log data source, configure query conditions to filter target logs, extract fields to generate metrics (e.g., error log count, average response time), and group them by business dimensions (e.g., service, interface, status code).

Scenario 2: APM Data Aggregation

Generate service SLA metrics based on APM trace data. Select the APM data source, group statistics by service and interface, calculate metrics such as P99 latency, error rate, and throughput, generate them periodically, and monitor continuously.

Scenario 3: Resource Usage Statistics

Statistics Kubernetes cluster resource usage. Select the Infrastructure Objects or Resource Catalog data source, group by cluster, namespace, node, count Pod numbers, resource requests/limits, etc., and generate resource utilization metrics.

Scenario 4: Multi-source Data Fusion

Combine multiple data sources to generate composite metrics. Create generation rules separately, and generate new metrics based on the generated metrics to build a complex business monitoring system.

Applicable Scope¶

Logs, APM, RUM, Metrics, Synthetic Tests, Infrastructure Objects, Resource Catalog.

Create¶

1. Select the data source.
2. Configure the data query conditions.
3. Define the generated metric content, setting the method and results for generating metrics, including the generation frequency, the tag names, and the measurement name for the newly generated metrics.

Data Query¶

Metric data additionally supports PromQL queries. Apart from that, other data types support simple queries and DQL queries.

For more details, refer to Chart Query.

Aggregation Functions¶

Dimensions¶

Aggregate data according to the selected objects, meaning a statistical value is generated for each selected object in the data request.

Generated Metric Content¶

1. Frequency: The execution cycle for generating metrics. The selected time for frequency also serves as the aggregation time. Choosing a frequency of 1 minute means aggregating and generating metrics every 1 minute, with each aggregation covering a time range of 1 minute.

   • 1 minute (default, meaning new metric data is generated every 1 minute)
   • 5 minutes (default selected when the data type is "Infrastructure Objects" or "Resource Catalog")

2. Select the data delay window, specifying the waiting duration for the calculation task after the task window ends.

3. Measurement: Set the name of the measurement where the metrics will be stored.

4. Metrics: Set the names of the metrics. Metric names can be duplicated, and multiple metrics can be added.

5. Tags: Automatically generated based on the dimensions selected in the query.

6. Unit: Optional. Set the unit for the metric. Once set for a generated metric, it can be applied in chart queries.

7. Description: Optional. Set a description for the metric. Once set for a generated metric, it can be applied in chart queries.

After completing the form, click Confirm to finish creating the metric generation rule and start data collection.

Data Delay Window¶

The duration the system continues to wait for data arrival after the task window ends. Enabling this can accommodate data reporting delays and improve metric accuracy.

When generating metrics, the system executes queries periodically according to the frequency you set (e.g., 1 minute). Each query retrieves data from a fixed time range (the "task window"). However, due to delays between data generation and reporting to the database, some data may arrive after the task window ends, causing these "late" data points to be missed in the statistics.

Enabling the data delay window allows the system to wait for an additional period after the task window ends, giving delayed data a chance to be included in the statistics, thereby improving metric accuracy.

1. Scenarios where enabling the data delay window is recommended:

2. Calculation logic example (frequency 1 minute, data delay window 1 minute):

Task execution time: 13:01:00
Task window (actual query time range): 13:00:00 ~ 13:01:00
Data delay window: 1 minute
Actual query execution time: 13:02:00 (after waiting 1 minute)

That is: The system executes the query at 13:02:00, retrieving data with time between 13:00:00 and 13:01:00. At this point, even if some data is stored after 13:01:00, as long as its time belongs to 13:00:00 ~ 13:01:00, it can be counted.

Manage List¶

• Edit: View all created metric generation rules and edit them.

• Enable/Disable: Modify the rule's status. When a metric generation rule is disabled, the corresponding data will not be written to the measurement. Writing resumes after enabling.

• Delete: Delete unnecessary rules. After a rule is deleted, the measurement is not deleted, but data writing stops.

• Batch operations: Perform batch operations on specific rules, including enable, disable, delete, and export rules.

• Import: Import metric generation data.

• View Metrics

  • View in Metric Analysis: Jump to the Metric Analysis page for query and analysis.

  • View in Metric Management: Jump to the Metric Management page to view metrics and tags. You can edit metric units and descriptions.

Use Cases¶

Chart Query¶

Query and analyze metric data in Visual Charts.

Query Tool¶

Query and analyze metric data in Shortcut > Query Tool > DQL Query.