Log Indexes¶

The system automatically archives log data into corresponding indexes based on set filter conditions. By creating multiple log indexes, you can:

• Isolate log data by business line, environment, or project
• Configure differentiated storage policies for different indexes
• Optimize query performance and reduce scanning of irrelevant data

Create¶

1. Navigate to the Create > Log Index page.
2. Customize the index name.
3. Add a description as needed.
4. Add filter conditions.

5. Configure the data storage policy: Select the log retention period, after which data is automatically cleaned up.

   • Deployment Plan users can customize the storage duration, range: 1d ~ 1,800d.

6. Configure advanced options as needed.

7. Confirm.

Filter Conditions¶

Filter conditions define the rules for logs to enter the current index. Two configuration modes are currently supported: Basic Match and Advanced Match.

Basic Match¶

Basic Match follows the original configuration method and is suitable for scenarios with simpler filtering logic.

1. Select the condition relationship:

   • Meet all conditions: Logs must satisfy all filter conditions simultaneously to enter the current index.
   • Meet any condition: Logs can enter the current index if they satisfy any one of the filter conditions.

2. Add filter conditions:

   • Field Name: Select a log field, such as source, service, host, etc.
   • Operator: Select the matching method.
   • Match Value: Enter the field value. Separate multiple values with English commas.

Basic Match is suitable for quickly configuring index routing based on fixed fields.

Advanced Match¶

Advanced Match is suitable for scenarios requiring more flexible combination of matching rules. After enabling Advanced Match, log data will enter a Pipeline for matching judgment, and corresponding charges will apply.

In Advanced Match, you can configure multiple filter conditions. Each condition includes:

• Field Name: Enter the log field to be evaluated.
• Matching Method: Supports =, !=, in, not in, match, not match, wildcard, not wildcard.
• Match Value: Enter the corresponding match value for the field.

Click Add Filter to continue adding conditions. Click Clear All Conditions to remove all currently configured conditions.

An Expression Preview is displayed at the bottom of the page to confirm the final matching expression generated by the current filter conditions. If the conditions are incomplete or the expression is invalid, the page will prompt "Please complete or correct the filter conditions". In this case, you need to supplement the field name, matching method, or match value before saving.

Advanced Options (Optional)¶

Full-text Index Fields¶

1. Only message index field (default): Full-text search only matches within the original message field content. In this mode, the system retains the original log content.

2. Full-line index: Full-text search matches across all business fields, excluding system fields. After selecting this mode, the system no longer saves the original message field. Extracted business fields will be written to variant for retrieval (❗️ When switching to "Full-line index", please ensure the field parsing rules are complete. Missing parsing rules may prevent log content from being retrieved normally).

Matching Services¶

Used to associate APM data with application logs.

1. Input rules: Supports wildcard fuzzy matching (e.g., service*). Separate multiple values with English commas.
2. Scope: When viewing related logs on the APM > Details page, the system prioritizes matching logs from the index corresponding to the service.

Configuration example:

• payment*: Matches all services starting with payment
• order-service,user-service: Exactly matches the two specified services

Index Limits¶

Further Reading¶