Log Indexes¶

The system automatically archives log data into corresponding indexes based on configured filter conditions. By creating multiple log indexes, you can:

• Isolate log data by business line, environment, or project
• Configure differentiated storage strategies for different indexes
• Optimize query performance and reduce scanning of irrelevant data

Create¶

1. Go to Create > Log Indexes page.
2. Customize the index name.
3. Add a description as needed.
4. Add filter conditions.

5. Configure data storage strategies: select the log retention period, after which data is automatically cleaned up.

   • For Deployment Plan users, you can customize the storage duration, ranging from 1d to 1,800d.

6. Configure advanced options as needed.

7. Confirm.

Filter Conditions¶

Filter conditions define the rules for logs to enter the current index. Two configuration modes are currently supported: Normal Matching and Advanced Matching.

Normal Matching¶

Normal Matching follows the original configuration method and is suitable for scenarios with simpler filter logic.

1. Select the condition relationship:

   • Match all conditions: logs must satisfy all filter conditions simultaneously to enter the current index.
   • Match any condition: logs that satisfy any one of the filter conditions can enter the current index.

2. Add filter conditions:

   • Field name: select a log field, such as source, service, host, etc.
   • Operator: select the matching method.
   • Match value: enter the field value, with multiple values separated by commas.

Normal Matching is suitable for quickly configuring index splitting based on fixed fields.

Advanced Matching¶

Advanced Matching is suitable for scenarios requiring more flexible combinations of matching rules. When Advanced Matching is enabled, log data will enter the Pipeline for matching evaluation, and corresponding charges will be incurred.

In Advanced Matching, you can configure multiple filter conditions, each containing:

• Field name: enter the log field to be evaluated.
• Matching method: supports =, !=, in, not in, match, not match, wildcard, not wildcard.
• Match value: enter the corresponding match value for the field.

Click Add Filter to add more conditions; click Clear All Conditions to remove all currently configured conditions.

An Expression Preview is displayed at the bottom of the page, showing the final matching expression generated by the current filter conditions. If the conditions are incomplete or the expression is invalid, the page will prompt "Please complete or correct the filter conditions". In this case, you need to fill in the field name, matching method, or match value before saving.

Advanced Options (Optional)¶

Full-text Index Fields¶

1. Only message index field (default): full-text search matches only within the original message field content. In this mode, the system retains the original log content.

2. Full row indexing: full-text search matches across all business fields, excluding system fields. When this mode is selected, the system no longer retains the original message field. Extracted business fields will be written to variant for retrieval (❗️When switching to "Full row indexing", ensure that the field parsing rules are complete. Missing parsing rules may cause the log content to be irretrievable).

The original "Match Service" capability has been migrated to Log Index Mapping.

Index Limits¶

Further Reading¶