Release Notes (2021)¶

This document records the update notes for each release of Guance, including DataKit, best practices for Guance, integration documentation for Guance, and Guance.

December 30, 2021¶

Optimize Binding Built-in Views¶

"Guance" supports associating built-in views with different trace services, applications, log sources, projects, etc., through binding fields. Based on the bound fields, the corresponding built-in view can be viewed on the detail page of the respective explorer (system views, user views). Supported explorers include scenario-based custom explorers, infrastructure, logs, traces, user access, security inspection, etc.

Optimize Log Explorer¶

• In the log explorer, the left source list is expanded by default upon first login, but can be manually collapsed. The explorer remembers the last state by default.
• In the log explorer, click the settings button on the left to manually add filter fields for quick filtering.
• In the log data list, you can "hover your mouse" over the log content to expand and view the full log content. Clicking the "copy" button copies the entire log content to the clipboard. When expanding, if possible, the system formats the log as JSON; otherwise, it displays normally.

Optimize Correlation Display on Explorer Detail Pages¶

Optimize correlation display on detail pages of logs, infrastructure, traces, and security inspection explorers, including host, metrics, traces, containers, logs, Pod correlations, supporting two types: fixed display and non-fixed display (display based on whether correlation fields exist).

Taking the example of correlating hosts on the log explorer detail page, the correlation fields are respectively "host". To view related hosts in the log details, the field "host" must match; otherwise, related host pages will not be visible in the log details. Once the fields match, under the correlated hosts, you can view basic host information and performance status metrics.

• Attribute View: Includes basic host information and integration runtime status. If cloud host collection is enabled, cloud provider information can also be viewed.

• Metrics View: Displays performance metrics such as CPU and memory for the relevant host within the default 15-minute window. Clicking "Open this view" in the built-in view allows customization by cloning the host view and saving it as a user view. User views can be bound for viewing on the log details page.

Optimize Snapshot Sharing¶

Snapshot sharing supports hiding the top bar of the shared page. In the snapshot list, click the share button to perform advanced settings "Hide Top Bar" in the pop-up dialog.

Optimize Chart Query Results to Show Thousands Separator¶

The Guance dashboard chart query results support automatic formatting with thousand separators. If units are set, the data format follows the specified unit, and the preview reflects the setting.

Other Optimizations¶

• Optimize host network topology query performance, adding target domain name display in network topology nodes. If the target host is not in the current workspace but the target domain exists and the port number of the target domain is less than 10000, the target domain name will be displayed in the topology diagram.
• Optimize sorting logic for favorited scenario dashboards and notebook explorers:
• Unfavorited dashboards are sorted in descending order by creation time
• Favorited dashboards are sorted in descending order by favorite time
• Favorited dashboards appear above unfavorited ones
• Optimize cascade query logic for view variables: when two variables are linked for querying, selecting * in the first one does not apply filters to the linked variable, displaying all value results.
• Enhance the left-collapsed navigation bar by adding links to help documentation.

December 16, 2021¶

New Time Series Similarity Metrics Query¶

In analysis mode of the time series chart, select "Chart Query" as metric queries' trend lines/bars to "view similar trend metrics." By selecting an absolute time range via box selection, you can query for similar metric trends within the space.

New Correlation Data Statistics Display on Explorer Detail Page¶

Add statistics display for related logs, traces, containers, etc., on the explorer detail page. This includes infrastructure, logs, application performance, user access, and security inspection explorers, showing statistical quantities directly in tab titles on the explorer detail page.

New Export Function for Explorer Charts¶

Guance supports exporting any chart from the "Guance" space explorer to dashboards, notebooks, or the clipboard.

New Wildcard Search in Explorer¶

Guance supports wildcard searches using field filters like "host:cc*" in the explorer search bar, useful for filtering log data from hosts with consistent naming prefixes. Additionally, a "delete icon" is added to the right side of the explorer search bar to support one-click deletion of text input.

Optimize Log Explorer and Log-related Data Queries¶

• "Guance" allows viewing all container (Container) data related to the host of that log within the last ten minutes via the "Container" section below the log details page.
• Log detail page-related trace and container data support filtering by "trace_id" or "host."
• Log detail page content display supports both JSON and text viewing modes, automatically displaying formats based on message type.
• Through the approximate text explorer, you can view detailed logs associated with Pattern clustering.
• Remove color from mouse hover tooltips in the log explorer.

Optimize Silence Policy, Support Setting Silence for Monitor Rules¶

Guance supports users managing all current space silence rules through "Silence Management" in the "Monitoring" function. "Guance" provides three types of silence: host silence, monitor silence, and group silence, allowing management of silences for different hosts, monitors, and groups so that silenced objects do not send alerts to any alert notification targets during the silence period.

Optimize Monitor Management¶

Duplicate monitors in the monitor list affect users' ability to quickly locate monitoring objects. "Guance" supports choosing whether to create duplicate monitors from templates. Additionally, users can use the newly added "Batch Management" tool to customize export/delete monitors.

Optimize View Variable Query Display¶

• Add default values to dashboard view variables, supporting previewing all current view variables and selecting the initial variable to view on the dashboard.
• Support dragging view variables with mouse hover to adjust their order.
• Remove the "Settings" button for view variables, allowing direct use of "Sort," "Hide," and "Delete" functions in the view variable list.

Optimize Lock Time Settings for Dashboards and Notes Charts¶

Guance supports increasing "More" options in lock time settings for dashboards and notes charts to customize lock time ranges. After this optimization, the lock time component and chart component functions remain consistent.

Optimize Help Center Page Entry¶

In the Guance help center, you can quickly view product documentation, product updates, blogs, live reviews, and more information. You can join our official service group for Guance by scanning a QR code.

December 2, 2021¶

New Owner Member Role¶

Guance adds the current workspace owner role, currently supporting four defined workspace member permissions: "Owner," "Administrator," "Standard Member," and "Read-only Member," each constraining management permissions, configuration permissions, operation permissions, and browsing permissions for different member categories. The "Owner" role has the highest operational permissions, allowing designation of current space "Administrators" and performing any management, configuration, operations, and data browsing. Edit and update member permissions in "Manage" - "Member Management" - "Modify" within the Guance workspace.

Scenario Adds Custom Explorer¶

"Guance" provides a quickly buildable viewer based on log data scope in scenarios, supporting space members to jointly build viewers based on custom scenarios for customized viewing needs. Completed "Explorers" can be exported and shared with others to share explorer templates.

New Export User View to Dashboard¶

"Guance" supports exporting created user views from the "Built-in View" as a json file, which can be used for importing into scenarios or built-in views in different workspaces.

Optimize Host Network, Add Network Flow Data Viewing List¶

"Guance" supports viewing network flow data lists in the "Network" section of the host details page under Infrastructure, displaying different visual charts based on selected protocols and customizing network flow data list display fields.

By clicking "View Network Flow Data," you can view network flow data from the past two days by default, including time, source IP/port, destination IP/port, source host, transmission direction, protocol, sent bytes, etc. Supports customizing display fields or adding filter conditions to filter all string-type keyword fields.

New Source Map Functionality¶

When deploying applications in production environments, files are generally transformed and compressed during the packaging process to prevent code leakage and other security issues. While these measures ensure code security, they also result in obfuscated error stack information, making it difficult to locate problems, thus complicating subsequent bug troubleshooting.

To address the above issue, "Guance" provides sourcemap functionality for Web applications, supporting deobfuscation of obfuscated code to facilitate error troubleshooting and help users resolve issues more quickly.

New Host Correlation Security Inspection Data Analysis¶

"Guance" supports viewing security inspection data related to the host within the past day and performing keyword searches, multi-tag filtering, and data sorting on this security inspection data.

New DataKit Collector Kubernetes Installation Guide¶

"Guance" supports adding installation instructions for the DataKit collector in Kubernetes clusters in the "Integration" - "DataKit" section of the workspace.

Optimize User Access Monitoring Correlation Trace Query¶

"Guance" supports adding attributes to the current filter or copying them (copying the tag to the local clipboard) on the trace details page and user access page performance details page (user access monitoring view explorer).

Optimize Registration and Upgrade Process¶

Guance adds the option to directly choose to activate the guance.com billing platform when creating an account and optimizes the process of upgrading to the Agile Plan after logging into the workspace.

Billing Price Adjustment¶

Guance adopts the latest technology to optimize data storage costs, hence adjusting the billing model downward and introducing a tiered billing model based on data storage strategies, including three billing items: log data, application performance Trace, and user access PV.

Additionally, to help enterprises at different development stages comprehensively cover IT infrastructure, application systems, and other enterprise assets at more favorable prices, Guance introduces three package plans: Startup Acceleration Pack, Entrepreneurship Development Pack, and Enterprise Standard Pack, along with optional traffic packs for叠加 usage according to enterprise needs and packages.

Other Optimization Features¶

• Added LCP, FID, CLS, FCP related detection indicators for user access metric monitoring in the Guance workspace. Go to "Monitoring" - Monitors, click "+New Monitor", select "User Access Metric Detection" to enter the detection rule configuration page.
• Changed grouping to detection dimensions in simple queries of monitors. Detection dimensions determine which dimension triggers the detection rules, i.e., triggering objects. "Guance" supports adding multiple detection dimensions, and any detection dimension meeting the alert condition will trigger an alert.
• Optimized navigation menu display. When the Guance workstation's navigation bar is collapsed, secondary menus display the feature's navigation name flush-left.

November 18, 2021¶

New SSO Login¶

Guance supports users creating identity providers based on the workspace. Users can obtain the corresponding SSO login link by entering their company email during login and click the SSO login link to complete the verification login. In the Guance workspace, go to "Management" - "SSO Management" - "New Identity Provider" to set up SSO single sign-on for employees.

New SLO Chart for Dashboards and Notes¶

Add SLO charts to scenario dashboards and notes. SLO charts can directly select configured monitoring SLOs to display SLO data. By selecting different SLO names, synchronize the display of SLO data results.

New Infrastructure Grouping Custom Add Grouping Fields¶

In the Guance Infrastructure explorer, click the settings button to the right of the grouping to customize adding grouping tags. After adding, you can choose to group and aggregate display based on the added grouping tags. Note: Custom grouping tags can only be added by administrators.

New SLO Notification Mute Configuration¶

If the same event is not very urgent but has high alert notification frequency, you can reduce the notification frequency by setting up notification mute. After setting up notification mute, events will continue to occur, but notifications will no longer be sent, and generated events will be stored in event management.

Optimize Chart Links¶

Guance supports adding built-in links and custom links to charts. Through these links, you can jump from the current chart to the target page and modify the corresponding variable values in the link using template variables to transmit data information, achieving data linkage. Chart links support opening from a new page, current page, or side-sliding detail page to achieve linkage analysis.

Billing Price Adjustment¶

• Daily DataKit price reduced to RMB 3 per device
• Adjusted availability API testing to RMB 1 per 10,000 tests

Other Optimization Features¶

• Launch free new version of China Zone 1 (Alibaba Cloud): Free Plan is divided into Alibaba Cloud login and AWS login. For Alibaba Cloud login, the timeline data retention policy is 30 days, while other data retention policies are 7 days. For AWS login, all data retention policies are 1 day;
• Optimize SLO deduction logic: Monitors in disabled state will not be counted in deductions;
• Add dashboard tag function: When creating/modifying dashboard views, you can add custom tags to help quickly filter;
• Optimize log highlight display: When searching logs in the search bar, the returned list retains only the highlighted keywords;
• Optimize explorer search: Add help document links in the explorer search dropdown;
• Optimize text chart Markdown format: Add support for unordered and ordered lists, optimize display format.

November 4, 2021¶

New Scenario Notes¶

Under Scenarios, you can create multiple notes for summary reports, supporting insertion of real-time visual charts for data analysis, inserting text documents for explanations, combining charts and documents for data analysis and summary reports; supporting exporting data from infrastructure, logs, application performance, user access, etc., to notes for data analysis; supporting sharing notes with all workspace members to retain abnormal data analysis, helping to trace back, locate, and solve problems.

Scenario Views Upgraded to Dashboards¶

The scenario views of Guance have been upgraded to dashboards, with adjustments made to the functions and layout of dashboards. Under scenarios, you can create multiple dashboards to build data insight scenarios, support searching dashboards by keyword, support adding visual charts for data analysis to dashboards, and support quickly filtering and finding corresponding dashboards from "My Favorites," "Import Projects," "My Creations," and "Frequently Browsed."

Explorer Adds Export to Dashboard and Notebooks¶

Infrastructure, log monitoring, application performance monitoring, availability monitoring, security inspection, and other explorers now support exporting to dashboards and notebooks.

Explorer Enhancements Include Export to Dashboard and Notebooks¶

Infrastructure, log monitoring, application performance monitoring, availability monitoring, security inspection, and other explorers now support exporting to dashboards and notebooks.

Upgrade of Anomaly Detection Library to Monitoring¶

The original "Anomaly Detection Library" has been upgraded to "Monitoring," with changes to the names of original terms such as "Anomaly Detection Library" and "Detection Rules." Specific changes are as follows:

• Monitors (originally referring to "Anomaly Detection Rules") provide multiple monitoring methods such as "Threshold Detection" and "Log Detection," allowing users to customize detection rules and trigger conditions and receive alert notifications immediately upon detecting anomalies.
• Templates (originally referring to "Built-in Detection Library") offer various ready-to-use availability monitoring templates developed by Guance, supporting one-click creation of Docker, Elasticsearch, Host, Redis monitoring. Successfully creating a template automatically adds the corresponding official monitor to the current workspace.
• Groups (originally referring to "Custom Monitoring Library") support custom creation of meaningful monitor combinations to facilitate grouped management of various monitors.

New Monitoring SLO Function, Support Export to Dashboard¶

The "Monitoring" section adds SLO monitoring, supporting quality score assessments of service providers (Providers) in the cloud era, comparing detected Service Level Indicators (SLI) to see if they meet the required objectives. At the same time, Guance supports exporting SLOs as views to dashboards for synchronized SLO monitoring on the dashboard.

Other Optimizations¶

• Chart optimizations:
  • Removed grouping selection from overview charts;
  • Adjusted style selection for time series charts, pie charts, and bar charts;
  • Added prompt information for time slicing;
  • Supported editing of chart JSON and linkage with queries/settings, supported validation of entered JSON, and displayed error prompts if errors were found;
  • Style optimizations for analysis mode in time series charts;
• Added prompts in the payment plan and billing section, allowing selection of settlement methods as aliyun or aws, with billing lists directly displaying links to view billing information on the corresponding cloud service console.

October 21, 2021¶

New Host Network Distribution Map¶

Added host network distribution map to provide infrastructure network monitoring. In "Infrastructure" - "Host," click the small icon in the upper left corner to switch to view host network distribution. In the "Network Distribution Map," you can visually query the network traffic between hosts in the current workspace, quickly analyzing TCP latency, fluctuations, retransmission counts, connection counts, and closure counts between different hosts.

New User Access Monitoring Tracking Feature¶

Added "Tracking" functionality to user access monitoring. Supports users creating tracking tasks through "User Access Monitoring" to monitor custom trace trajectories in real time. By predefining trace trajectories, you can centrally filter trace data and precisely query user access experiences, promptly discovering vulnerabilities, anomalies, and risks.

New Scenario Chart Json Format Query¶

In the "Scenario" edit chart, each correct chart query corresponds to a json text, supporting mutual parsing between json text and chart queries in the workspace, helping you understand chart drawing details.

Optimize DCA Desktop Client Application¶

Optimize DCA desktop client UI display, add host status.

• Online: Indicates normal data reporting, allowing viewing of DataKit's runtime status and configuring collectors through DCA;
• Unknown: Indicates remote management configuration is not enabled or not on the same LAN;
• Offline: Indicates the host has not reported data for more than 10 minutes, or the hostname was modified, resulting in the original hostname being displayed as offline. Hosts that have not reported data normally for more than 24 hours will be removed from the list.

Optimize Event Host Linked Query and Detection Rule Query¶

Optimize the event module to support联动查询 of associated hosts in events, enabling real-time monitoring of logs, containers, processes, traces, and inspection events related to the event within the selected time component range. Supports one-click viewing of associated detection rules for events and querying other triggered events under the corresponding detection rule.

Adjust Billing Method¶

• Choose billing settlement method by node login method, including CloudCare account, Aliyun, AWS. Among them, CloudCare account settlement is the general method, Aliyun login node corresponds to the choice of Aliyun account settlement method (launching), AWS login node corresponds to the choice of AWS account settlement method (already launched).
• Remove Session quantity and Browser test count billing dimensions, remove Session quantity and Browser test count statistics and views in the "Payment Plan & Billing" section, add security inspection count statistics chart.

Other Optimizations¶

• Optimize scenario add chart functionality, remove the "Edit" mode "Add Chart" button, all available chart types are selectable at the top of the scenario;
• Optimize scenario chart grouping functionality, support deleting groups only, retaining original charts in the default group;
• Add "Regional Ranking" switch for China/World maps, default closed;
• Optimize explorer query methods, merge search and filter bars, support keyword search based on tags, fields, text, tag field filtering, and associated search.

September 28, 2021¶

New Host Network Performance Monitoring¶

Host network performance monitoring supports viewing network traffic between hosts, containers, and any tagged infrastructure. It supports viewing network traffic and data connection status between source hosts and destinations based on IP/port in a visual manner in real-time, helping enterprises understand the network operation status of business systems in real-time, quickly analyze, track, and locate issues, preventing or avoiding business problems caused by network performance degradation or interruption.

After successful host network data collection is uploaded to Guance console, in the "Infrastructure" - "Host" details page under "Network", you can view all network performance monitoring data information within the workspace.

New Chart Linked Query¶

In scenario views, support viewing the DQL query statement of the chart, associated links, etc., by clicking the mouse, or associated querying related logs, containers, processes, traces, etc., under different group query tags. When the chart query contains host (host) information, support viewing the associated host monitoring view. For example, clicking any chart in the figure allows viewing the DQL query statement of the chart, the set links, and associated queries for related logs, containers, processes, traces, host monitoring views, etc.

New Chart Treemap¶

Treemaps are used to visualize the proportion distribution of metric data under different groups. The color in treemaps represents the tag (Tag) of the group, and the area represents the metric data. When multiple metric queries are added, you can choose the main metric to display, with the area size based on the metric data results of the selected metric, Top/Bottom, and quantity.

New Application Performance and User Access Monitoring Associated Query¶

Within the application performance monitoring trace details, support viewing related (same trace_id) real user access experience data through the "Related View" on the top of the page. This feature not only helps you view active application performance data but also allows you to quickly gain insights into real user access situations. For more application performance monitoring, refer to the documentation Trace Analysis.

New Absolute Time Range for Infrastructure Query¶

Add absolute time range query for infrastructure objects such as "Host," "Container," and "Process." By default, it shows host data for the last 24 hours, container data for the last 10 minutes, and process data for the last 10 minutes. Manual refresh of the query range is possible. Hovering the mouse over an offline host displays a prompt for handling offline hosts.

Optimize Chart Year-over-Year Comparison Function¶

Optimize the chart year-over-year comparison function to support comparison of同期 data for time series charts, overview charts, etc. You can choose to compare同期 data (comparison of a specific time point in adjacent time periods) or sequential comparison (comparison of adjacent time periods). Based on the locked chart time, you can choose sequential comparison, daily sequential comparison, weekly sequential comparison, monthly sequential comparison, weekly同期 comparison, monthly同期 comparison, etc.

Optimize Chart Query SLIMIT¶

Default add SLIMIT to group queries for charts, used to limit the number of points or series returned by time series chart queries. One chart limits returning 10,000 points, and one query returns a maximum of 10 time lines.

Rename Cloud Testing to Availability Monitoring¶

Cloud testing is divided into API testing and Browser testing. After renaming to availability testing, retain API testing. Other functions such as overview analysis, explorer analysis, and self-built node functions remain unchanged. For more details, refer to the document Availability Monitoring.

Billing Price Adjustment¶

• Log price adjusted from RMB 1 per million entries to RMB 0.5 per million entries
• User access monitoring price removed Session billing dimension, adjusted to RMB 0.3 per thousand PVs

September 9, 2021¶

DataFlux Renamed to “Guance”¶

As an observability platform for the cloud era launched by Zhuyun Technology, DataFlux has been officially renamed “Guance” to make it easier for enterprises and users to understand. The website URL is “guance.com”.

At the same time, we designed a new logo for “Guance,” composed of the letters C and G forming a radar wave-like graphic.

New Log Approximate Text Viewer¶

“Guance” supports similarity calculation analysis of the message field of raw log data. According to the fixed time range selected in the upper right corner, it obtains 10,000 log entries within that time frame for approximate analysis, aggregates logs with high similarity, extracts and counts common Pattern clusters, helping to quickly identify abnormal logs and locate issues.

Pattern Cluster Details

Click on the Pattern cluster list to slide out the current list of all Pattern clusters. The returned data list is arranged in reverse chronological order and supports displaying the latest 50 log entries. You can choose to display in 1 line, 3 lines, or 10 lines.

New Container Jobs and Cron Jobs Viewer¶

“Guance” adds container «Jobs», «Cron Jobs» viewers. In «Infrastructure» - «Containers», select the corresponding viewer from the object list on the left to view the running status and service capability metrics of Job and Cron Job in Kubernetes, thereby achieving real-time monitoring of Kubernetes clusters and various resources deployed within.

For details, please refer to the document Containers.

New DCA Desktop Client Application¶

DCA, DataKit Control APP, is a desktop client application aimed at facilitating the management of installed and configured collectors. It supports viewing integrated lists, managing configuration files, Pipeline management, and online viewing of integration configuration documents. You can remotely connect to DataKit via DCA, make changes to the collector online, and save and update them to take effect.

In the “Guance” workspace, click «Integrations» - «DCA» in sequence to download the installation package. After downloading, install it on your computer to remotely manage DataKit.

New Mobile APP¶

DataFlux Mobile helps you view log data and scenario views from Guance on mobile devices anytime, anywhere, and receive all alert events notified to you through email, DingTalk bot, WeCom bot, Webhook, etc.

In the Guance workspace, you can download the DataFlux Mobile APP via «Integrations» - «Mobile».

New Tag Filtering Function for Metrics, Optimize Three Viewing Modes for Metrics¶

DataKit collector defaults to appending the tag host=<hostname where DataKit is located> to all collected data.

Tags are collections of properties identifying the attributes of a data point collection object. Tags are divided into tag names and tag values. On the «Metrics» page, you can select the desired tag values to view metric views. As shown in the figure below: filter hosts in the tag bar and view metric views such as usage_system, usage_total, usage_user, usage_iowait.

On the «Metrics» page, you can switch metric viewing modes by clicking the settings button on the left. Three viewing modes are supported: flat mode, hybrid mode, and list mode, with flat mode selected by default.

Optimize Anomaly Detection¶

Add window function usage in anomaly detection rule queries and chart queries, i.e., support re-executing statistical calculations for each record based on a selected time interval window (record collection) and offset by detection frequency.

• In chart queries, support adding window functions, i.e., use a selected time interval window (supports selecting 1 minute, 5 minutes, 15 minutes, 30 minutes, 1 hour, 3 hours) combined with aggregation functions to execute statistical calculations for each record.
• In configuring anomaly detection rules, add window function as the drawing area for query results, returning real-time anomaly detection metric data that triggers alerts.

Add percent() function usage in anomaly rule queries and chart queries aggregation functions. Support adding percent() function in aggregation functions, including p50 (take median value), p75 (take value at 75% position), p90 (take value at 90% position), p99 (take value at 99% position).

Add label filtering usage in anomaly rule queries and chart queries, supporting selection of host label attributes for filtering display. Before selecting label in queries, you need to set label attributes for the host in «Infrastructure» - «Host».

Optimize Display Style for Unresolved Events¶

In the unresolved events list, you can preview the window function of the event for the last 6 hours:
• Dashed border display indicates the time period affected by the abnormal event
• When the detection library rule type is threshold, log, application performance metric, user access metric detection, security inspection, abnormal process, or cloud testing detection, color blocks corresponding to different alert levels can be viewed to check related abnormal metric data, including emergency, error, warning.
• When the detection library rule type is mutation, interval, water level, vertical lines in the chart can quickly identify the time point when the current event was triggered.

In the event details page, support viewing the status distribution of abnormal events, DQL query statements, and window function line charts.
• Status distribution: displays the event status (emergency, important, warning, data gap) within the selected time range (default display is the last 6 hours)
• DQL query statement: returns real-time metric data based on custom query statements from anomaly detection rules, default display is real-time metric data for the last 6 hours
• window function: based on anomaly detection rules, re-executes statistical calculations for each record based on the selected time range window (record collection) and offset by detection frequency, returning real-time anomaly detection metric data that triggers alerts. Default display is real-time metric data for the last 6 hours

Optimize Scenario View Analysis¶

In preview mode, double-click the chart to zoom in and enter analysis mode to view. In edit mode, double-click the chart to enter edit mode and perform add, delete, modify, and query operations on the chart.

Optimize Infrastructure Binding Built-in View¶

When selecting a built-in view, duplicate named views are filtered out. When system views and user views have the same name, system views are prioritized for display. Only user views can be selected. If you need to select a system view, you must rename or delete the duplicate user view.

Optimize Billing Method, Add Cloud Testing Billing, Adjust Data Expiration Strategy¶

• Add PV count statistics: session count and PV count are billed based on the lower actual cost
• Add cloud testing task call count billing statistics: open up self-built nodes management for cloud testing, support creating new testing nodes globally; cloud testing billing is based on the current workspace cloud testing task call count. Free Plan workspaces can create up to 5 testing tasks and only support using "China region" testing nodes; Agile Plan and higher versions support creating more testing tasks and using more overseas testing nodes.

August 26, 2021¶

DataFlux Studio¶

New Label Filtering in Chart Queries¶

DataFlux supports selecting host label attributes for display filtering. Before selecting labels in chart queries, you need to set label attributes for hosts in «Infrastructure» - «Host».

As shown in the figure below, there are two hosts with the "production" label attribute, so data for two hosts is displayed.

New Infrastructure Container Cluster, Replicate Set, Node Viewers¶

DataFlux adds container Cluster, Replicate Set, Node viewers, supporting viewing the running status and service capability metrics of Cluster, Replicate Set, Node in Kubernetes, thereby achieving real-time monitoring of Kubernetes clusters and various resources deployed within.

New User Access Monitoring View Viewer Quick Filter¶

DataFlux supports clicking the settings button on the left of the View viewer in user access monitoring to customize adding filter fields. After selecting the required fields, click "Done" to complete.

Note: If the added filter field represents a time field, support filtering by selecting a time range to view the corresponding page content. For example, loading_time (page load time).

New Help Entry in Navigation Menu, Multiple Viewer Data Collection Guidance Entries¶

DataFlux adds a help documentation entry in the navigation menu. Additionally, when viewers have not uploaded data, DataFlux provides a quick access entrance to "How to Start Data Collection" to help you start real-time data monitoring in different viewers. This includes how to enable log monitoring, how to conduct cloud testing, and how to collect host objects.

Optimize Log Viewer¶

DataFlux optimizes the log viewer by highlighting different parts of the log content in different colors, helping you quickly get key log data information. It supports both light and dark theme colors.

Optimize Security Inspection Overview Statistics¶

DataFlux supports customizing the view of the security inspection overview page. By clicking the "Jump" button, you can navigate to the corresponding built-in view in the overview page for viewing, editing, copying, and exporting.

Optimize Workspace Switching¶

DataFlux supports switching to another workspace or creating a new workspace by clicking the current workspace name.

Optimize System Theme Color Switching¶

DataFlux adds an "Auto" option for system theme colors, which automatically switches the theme color based on the computer's appearance settings.

Other Optimization Features¶

• Optimize application performance monitoring "Service" details. In the service details page, clicking the resource name of the service directly jumps to "Trace" to view the trace situation of the corresponding resource;
• Optimize user access monitoring View viewer "Performance" details. Clicking error data in the View list (data marked with "Error" in the View list) can view the details of the corresponding error;
• Optimize metric viewer label name and label value display;
• Optimize time series chart X-axis time display. The default time range is the last 15 minutes, with the X-axis segmented by minutes; for the last 1 day, the X-axis is segmented by hours; for the last 7 days, the X-axis is segmented by days;
• Optimize expression queries. When querying charts, if the expression query contains multiple query statements, the group labels need to remain consistent;
• Optimize host integration runtime status. In "Infrastructure" - "Host" - "Integration Runtime Status," add DataKit version information installed on the host;
• Optimize snapshot sharing. Support sharing snapshots with anyone who has the sharing link, or encrypted sharing with people who have the link and key.

August 12, 2021¶

DataFlux Studio¶

New 3 Permission Roles, Optimize Scenario View Permissions¶

DataFlux currently supports defining three workspace member permissions, including "Administrator," "Standard Member," and "Read-only Member," each constraining management permissions, operation permissions, and browsing permissions for different member categories. "Administrators" can customize the permission scope for members, such as limiting read-only members to access specific scenario views or standard members to edit specific scenario views.

In the DataFlux workspace, go to «Management» - «Member Management» - «Modify» to edit and update member permissions.

New Infrastructure Host Details View Connection Tracking and File and Layout Optimization¶

DataFlux supports viewing host system information on the infrastructure host details page, including host name, operating system, processor, memory, network, hard disk, as well as connection tracking count, file handle count, etc.

New Scenario View Snapshot Save Function¶

In the DataFlux workspace, enter «Scenario» - «Node», click the "Settings" button on the top navigation bar, select «Save Snapshot» to save the data content displayed in the current view as a snapshot.

In addition, DataFlux supports quickly saving snapshots through page function buttons and shortcut keys (Ctrl+K). In scenario views and all viewers, using the shortcut key Ctrl+K quickly pops up the [Save Snapshot] dialog box, enter a name, click [OK] to add the current page as a new snapshot.

Note: When there are other shortcut keys conflicting with the shortcut key Ctrl+K in the background program, you will not be able to use the "Save Snapshot" shortcut key function.

New Chart and Snapshot Share Management¶

DataFlux supports unified management of shared charts and snapshots within the current space. After completing chart sharing in the scenario view, you can view the current space's chart sharing list through «Management» - «Share Management» - «Chart Share», and perform chart viewing, view embedded code, and cancel sharing.

New User Email Change, Phone Change Unbinding Function¶

In the DataFlux workspace, click the bottom left «Account» - «Settings» - «Basic Settings» to change or unbind your phone and email.

New Customization of User Access Monitoring SDK Data Collection Content¶

DataFlux supports customizing user access monitoring SDK data collection content. By customizing user identifiers, customizing session settings, and adding additional data TAGs, users can set different types of identifiers in specific scenarios to locate and analyze data.

New Free Quotas for Paid Edition Sessions, Tasks, and Traces¶

The paid version of DataFlux adds daily赠送 of 1 million log data entries, 100 session counts, 10,000 task scheduling times, and 1 million trace counts per DataKit.

Optimize Scenario Chart Analysis Mode¶

DataFlux supports previewing the interactive changes of object data within four times the query cycle in the analysis mode of time series charts, and dragging the timeline to select the time range displayed in the time series chart. For example: the current time is 14:00, and the time range is selected as [10:00-11:00], then the timeline range is [08:00-12:00].

Note: A time series chart query returns a maximum of 10 time lines by default, i.e., according to the grouping (group by) condition results, data exceeding 10 time lines will only display the first 10 time lines in order.

Optimize Infrastructure Host and Container Honeycomb Chart¶

Hover the mouse over host and container objects to view host and container names, CPU utilization, and MEM utilization.

Optimize Log Source List Based on Viewer Time Range Display¶

DataFlux supports filtering log source lists based on the selected viewer time range. For example, in the DataFlux workspace «Log Viewer», select a 15-minute time range in the upper right corner, and the left log list will only display log sources with data in the last 15 minutes.

Optimize System Theme Color Switching¶

DataFlux supports switching system theme colors, divided into light and dark themes. Click the account in the lower left corner and click the desired theme to switch directly.

Other Optimization Features¶

• Webhook custom notification content type changed from text format to json format.
• Added 15-minute silence option for alert settings in the anomaly detection library, i.e., within 15 minutes, the same alert will not send notifications.
• Adjusted display style of the unresolved event list page, changing log alert event statistics display from table chart to time series chart.

August 4, 2021¶

DataFlux Studio¶

m

New Container Services and Deployments Viewers¶

DataFlux supports comprehensive display of container status from different perspectives through viewers such as «Container», «Pod», «Services», and «Deployments».

In the workspace «Infrastructure» - «Container», you can switch to «Services» to view detailed information about all retained Services in the space, including Service name, Service type, Cluster IP, External IP, runtime duration, etc., by using the data type filter bar in the upper left corner.

Switching to «Deployments» allows viewing detailed information about all retained Deployments in the space, including Deployment name, available replicas, upgraded replicas, readiness status, runtime duration, etc.

July 29, 2021¶

DataFlux Studio¶

New Global Label Function for Infrastructure¶

DataFlux adds global label functionality for infrastructure, allowing users to add custom labels to infrastructure objects such as hosts, containers, processes, etc., and filter and display infrastructure objects with the same labels based on the added labels.

In the infrastructure object list, click the object name to slide out the details page and add labels «Label Attribute» to the object.

After adding custom labels, you can filter and display object lists with the same labels in the object list.

New SMS Notification Object; New Paid Plan and Bill SMS Cost Statistics¶

DataFlux supports user management of sending alert notification objects, currently supporting adding configurations such as DingTalk robot, WeCom robot, Feishu robot, Webhook custom, email group, and SMS group.

Note: - Members of SMS notifications need to be invited into the workspace via «Management» - «Member Management» before they can be selected. - DataFlux Free Plan has no SMS notifications, other versions charge 10 SMS/10 yuan, billed daily.

New Scenario View Slideshow Mode¶

DataFlux adds scenario view slideshow, supporting browsing all node views under a specified scenario in slideshow mode and customizing the slideshow interval.

New Workspace Token Replacement Function¶

DataFlux supports current space administrators changing the token within the space and customizing the expiration time of the current token. Enter «Management» - «Basic Settings» - «Replace Token», select the expiration time and confirm «Replace». DataFlux will automatically generate a new token, and the old token will expire within the specified time.

Note: - After replacing the token, the old token will expire within the specified time. If any proxy uses this token, data reporting will stop immediately, so please check the integration configuration in time; - Replacing the token will trigger «Operation Events» and «Notifications».

New Metric DQL Query Mode¶

DataFlux supports switching «Search» to «DQL Query Mode» in the workspace «Metrics», directly inputting DQL statements for querying, and viewing the corresponding labels and metrics on the left, and viewing the visualization chart of the query results on the right.

Optimize Multi-line Display for Log Viewer¶

DataFlux supports selecting log display as "1 line", "3 lines", or "10 lines" in the "Settings" of the log viewer to view complete log content.

Optimize Application Performance Monitoring Service and Trace Switching View, Optimize Response Time Line Chart¶

DataFlux supports switching between «Service» and «Trace» viewers by default while retaining the current filter conditions (including: environment, version) and time range. For example: in application performance monitoring «Service», select the environment as production_a, time range as last 30 minutes.

Switching to application performance monitoring «Trace», the filter conditions remain consistent with «Service».

Optimize DataKit Collector Installation Guide¶

DataFlux supports customizing installation options when installing DataKit, such as cloud provider, namespace, global tag, etc.; DataKit offline installation supports direct download of offline installation packages for different operating systems.

Optimize Flame Graph Synchronous and Asynchronous Call Display¶

DataFlux application performance monitoring flame graph supports viewing service synchronous and asynchronous call chain performance data details. For example, the flame graph clearly shows which requests are asynchronous, when they started, when they ended, and how long they took.

Optimize Viewer Display Column Custom Field Alias¶

DataFlux supports custom aliases for viewer display fields. For example, in the infrastructure host object, users can modify field aliases in "Add Display Column", and after modification, can view them in the viewer and details.

Optimize Cloud Testing Self-built Nodes¶

DataFlux cloud testing self-built node geographical location is split into two filter boxes: country and province (or city). If the country filter box selects China, the second filter box will filter out the list of provinces; if the country is not China, the second filter box filters and displays the list of cities. The filter box defaults to displaying 20 items and supports fuzzy search.

Optimize DQL Query Functions¶

DataFlux DQL adds new query functions such as exists, wildcard, with_labels.

July 15, 2021¶

DataFlux Studio¶

New Third-party Developer Tool Integration for User Access¶

DataFlux adds support for mini-program integration based on the uniapp development framework. By introducing the SDK file, it monitors performance metrics, error logs, and resource request data of the mini-program and reports it to the DataFlux platform.

New Data Gap Anomaly Detection¶

DataFlux adds data gap anomaly detection, which generates events based on custom detection cycles. Generated data gap events are saved in events.

Once the data gap detection rule is activated, no data gap alert event is generated during the first detection of data gap and continuous data gap. However, if data is detected and a data gap occurs within the custom detection cycle, a data gap alert event is generated.

New Elasticsearch, Redis, and Docker Built-in Detection Libraries¶

DataFlux currently supports four built-in detection libraries: Elasticsearch, Redis, Host, and Docker monitoring. You can use them out-of-the-box without manual configuration. If you need more details, you can directly click on the corresponding built-in detection library.

• Redis Detection Library: Includes Redis client connections waiting for blocking commands
• Host Monitoring: Includes multiple data metrics such as host CPU, memory, and disk
• Elasticsearch Detection Library: Includes ES JVM heap memory, thread pool, search query load anomalies, etc.
• Docker Monitoring: Includes multiple data metrics such as container CPU and memory

New Application Performance "All Trace Viewer"¶

DataFlux provides three types of trace filtering and viewing lists: "All Spans", "Top-level Span", and "Traceid". A span represents a logical work unit in a distributed system over a given timeframe, and multiple spans form a trace trajectory (trace).

• All Spans: Displays all collected span data within the currently selected time range
• Service Top-level Span: Filters and displays the first incoming spans data within the currently selected time range
• All Traces: Filters and displays all trace data containing initial top-level spans within the currently selected time range

New Trace Details View Topology Call Relationship Chart

DataFlux adds a corresponding service topology call relationship chart in the application performance monitoring details page to view the call relationships between various services.

New User Access Error Details Page¶

DataFlux supports viewing user access error details on the user access details page, where you can view error data information, error types, and error occurrence times during the user access.

Clicking the error message allows jumping to the corresponding error details page.

New Security Inspection Overview Statistics¶

In «Security Inspection» - «Overview», you can view the overview of security inspection events across different hosts by filtering hosts, security inspection levels, and security inspection categories. This includes the number of security inspection events occurring at different levels and visual chart analysis, as well as rankings of security inspection events by category and rule.

New Security Inspection Event Handling Recommendations¶

In the security inspection viewer, click the inspection event you want to view. In the details page that slides out, you can view handling recommendations for the current security inspection event, including theoretical basis for the security inspection event, risk items, audit methods, remedial measures, etc.

New Secondary Navigation Bar¶

DataFlux adds support for displaying the secondary navigation by "mouse hover," thereby shortening the time users spend searching for target data.

New Member Management Search Function¶

DataFlux supports quick matching of current workspace members based on email or name through the search function in «Member Management».

New Built-in View Binding to Infrastructure Objects¶

DataFlux supports binding built-in views (system views, user views) to infrastructure object detail pages for current space administrators. As shown in the figure, the built-in system view "Disk Monitoring View" is bound.

Optimize Event List Display by Detection Library Type¶

DataFlux supports displaying corresponding event information based on different detection library types in the event list.

• When the detection library rule type is threshold, mutation, interval, water level, application performance metric, or user access metric detection, display the corresponding baseline and information according to the configured trigger conditions, including critical, error, and warning.
• When the detection library rule type is log, security inspection, abnormal process, or cloud testing detection, query all log data for abnormal events statistics.

As shown in the figure below, there are three different detection library types, including threshold detection, security inspection detection, and process data detection.

Optimize Metric Set Deletion¶

DataFlux supports administrators deleting metric sets within the space. Enter «Management» - «Basic Settings», click «Delete Metric Set», then enter keywords to query and select the metric set name, click «OK» to enter the deletion queue for deletion. Deleting a metric set generates a system notification event, such as users creating a metric set deletion task, metric set deletion task execution success, metric set deletion task execution failure, etc.

Optimize Viewer Built-in View Query Time Range¶

Change the query time range of built-in views, including infrastructure objects, trace, log detail pages' "Host Monitoring" view, and trace, log detail pages' "Process" view.

• When viewing the details page of traces and logs, the "Host Monitoring" view displays performance data of the associated host within the selected time component range by default.
• When viewing the details page of infrastructure objects (hosts, containers), the "Host Monitoring" view displays performance data of the host within the last 24 hours by default.
• When viewing the details page of traces, the "Process" view displays real-time process data within ±10 minutes of the trace occurrence time.
• When viewing the details page of logs, the "Process" view displays real-time process data within ±10 minutes of the log generation time.

Optimize Viewer Quick Filter Interaction¶

If the current field value is too long in various viewers' quick filters, the overflow part is displayed as ···. When the mouse moves over the current field value area, it expands to the right to display the full field value.

July 1, 2021¶

DataFlux Studio¶

New Unresolved Event Statistics Viewer¶

DataFlux adds an unresolved event statistics viewer. In the DataFlux workspace's Event, by switching the viewer in the upper left corner, you can view all continuously triggered unresolved events in the space and the statistics of unresolved events under different alert levels, alarm information details, etc.

New Chart Link Function¶

DataFlux adds chart link functions for time series charts, bar charts, overview charts, dashboards, pie charts, top lists, scatter plots, bubble charts, maps, honeycomb charts. Chart links help you jump from the current chart to the target page, supporting modifying the corresponding variable values in the link through template variables to transmit data information, completing data linkage.

Chart links support retrieving template variables through queries, support adding internal and external links to the platform, support opening in three ways: new page, current page, and slide-out detail page, and support adding alias display for links.

After adding the link, click the chart to view the clickable "custom link".

New Application Performance Monitoring Binding Links and Built-in Views¶

DataFlux supports adding built-in views to individual service details, including built-in views and web addresses that can be linked. In the service list, click «Settings» to add one or more jump link addresses or built-in views and customize the name.

After adding the built-in view, you can view the corresponding built-in page by checking the service details.

New User Access Monitoring Scenario PV/UV Overview Statistics¶

DataFlux adds PV/UV overview statistics for Web, iOS, Android, and mini-program scenarios in user access monitoring. Quickly compare PV/UV differences to understand the actual number of accessing users.

New Notification Object: Feishu Bot¶

DataFlux supports user management of sending alert notification objects. Currently supports adding emails, DingTalk bots, WeCom bots, Feishu bots, and Webhook custom notifications.

Optimize Infrastructure Resource Catalog¶

Besides hosts, containers, and processes, other object data in infrastructure can be reported to «Infrastructure» - «Custom» through DataFlux Function for viewing and analysis. Supports binding built-in views to resource catalog data and manually adding tags for filtering.

Resource catalog data reporting requires installing and connecting DataKIt and Function first, then reporting data to DataKit through Function, and finally DataKit reports data to the DataFlux workspace.

Optimize Detection Rules in Anomaly Detection Library¶

DataFlux adds recovery alert configuration for anomaly events; adds filter condition support for regular expression pattern adjustment, optimizing ＝～ to match, and adding not match; adds trigger condition support for non-numeric data matching, adding match, not match regular expressions, and between.

Optimize Correlated Logs on Viewer Detail Pages¶

DataFlux supports default filtering and screening based on trace_id for correlated logs on the trace side detail page, viewing correlated log details under the same trace_id.

In the trace details, click on the «Logs» below the flame chart to view correlated logs based on the current trace's host and trace ID (trace_id field). At the same time, you can perform keyword searches and multi-tag filtering on the logs. If you need to view more detailed log content, you can click on the log content to jump to the log details page or click the jump button to open the log page.

A new log correlation page has been added to the viewer.

Optimize Viewer Search Style¶

Adjust the search styles for scenarios, metrics, application performance monitoring, user access monitoring, anomaly detection library, host silence management, integration, management/built-in views, snapshots, etc.

Deployment Version Supports Modifying Data Retention Duration¶

Add change data storage strategy to the basic information of workspace management.

June 17, 2021¶

DataFlux Studio¶

New Log Analysis Mode¶

DataFlux supports grouping massive raw data based on 1-3 tags through the log grouping function to reflect the distribution characteristics and trends of grouped data at different times. DataFlux supports three grouped data browsing methods: line chart, area chart, bar chart.

New Log Filtering¶

Adds log filtering functionality (log blacklist), supporting setting filtering rules to filter out logs that meet the criteria. That is, after configuring log filtering, logs that meet the criteria will no longer be reported to the DataFlux workspace, helping users save on log data storage costs.

Go to «Log» - «Filter Rules» - «Create Rule», select "Log Source", add one or more data filtering conditions with the relationship and (AND) to filter the collected log data. Logs that meet the filtering conditions will not be reported to the workspace.

New Save Metric Browsing Snapshot, New Snapshot Link Copy Function¶

DataFlux supports creating accessible data copies for the metric viewer and storing them uniformly in the "Snapshot" directory. A new snapshot sharing link is added, supporting sharing snapshot links with other members in the space, quickly restoring instant copied data copies and recovering data to a specific time point and data display logic.

New Container Pod Viewer¶

DataFlux adds a Pod viewer in «Infrastructure» - «Container». Through the filter in the upper left corner, you can switch to «Pod» to view detailed information about all collected Pods, including Pod name, node Node it belongs to, one or more containers included, startup status, running status, and related metrics. You can view through the honeycomb chart.

Clicking the Pod in the honeycomb chart allows viewing detailed information about the Pod, including Pod name, namespace, visualization monitoring charts, etc.

New Free Quota Warning Notifications¶

Adds experience edition usage quota warning notifications and quota exhaustion notifications, supporting sending emails and generating system notifications. Learn more about message notifications. System message notifications can be viewed in the «Account» - «Notifications» section of the DataFlux workspace.

New Chart Sharing¶

DataFlux supports chart sharing, which can be used to insert charts into platforms outside of DataFlux for visual data display and analysis. Shared charts will change according to the charts in DataFlux. Shared charts in the scenario view of DataFlux are stored uniformly in «Integration» - «Embedded Sharing».

Optimize Chart Query and Chart Style and Settings¶

Adjust and optimize visualization chart queries, styles, and settings layouts in scenario views. New features include hiding chart titles, setting time slices, setting overview chart data precision, setting leaderboard baselines, and filling color schemes for maps/honeycomb charts.

Optimize Integration, Add DataKit and Function Installation Instructions, Add Embedded Sharing¶

DataFlux supports viewing all supported collectors, DataKit installation instructions, DataKit upgrade commands, Function installation instructions, and embedded shared chart code in the Integration section.

In the DataFlux workspace, click the left «Integration» menu to view all supported collectors. Clicking on a collector allows viewing the collector's configuration details.

In the DataFlux workspace, click «Integration» - «DataKit» to get DataKit installation instructions, including Linux, Windows, MacOS operating systems, and offline installation methods. If DataKit is already installed but not the latest version, you can get update scripts through this section's "DataKit Upgrade".

In the DataFlux workspace, click «Integration» - «Function» to get Function installation instructions.

Optimize Host Detail Page¶

In «Infrastructure» - «Host», click the host name in the list to slide out the detail page to view the host's basic information, processor, memory, network, hard drive, etc. Supports automatic conversion of data units and displays different host monitoring views based on different operating systems.

June 3, 2021¶

DataFlux Studio¶

New Browser Log Collection¶

DataFlux supports actively sending log data of different levels (corresponding source: browser_log metric type log data) from web browsers or JavaScript clients to DataFlux.

New User Access Monitoring Session Viewer¶

DataFlux user access monitoring viewer adds viewing of user access behavior monitoring data (session data) and session data details related to Web/miniprogram/Android/iOS applications. Open «User Access Monitoring» - «Select Any Application» - «Viewer» in the workspace to switch to the session viewer (Session). The session viewer counts the entire session duration, number of pages, number of operations, number of errors, etc. Clicking a single session allows viewing the detailed records of the entire session. By viewing and analyzing session data, you can understand real user access data from multiple dimensions and help improve application performance and user experience.

New Browser Testing, Cloud Testing Anomaly Detection Template and Optimize Self-hosted Nodes¶

1. Browser Testing

DataFlux adds browser testing. In DataFlux «Cloud Testing», click «New» - «Browser Testing» to create a new browser testing task, which allows real-time acquisition of user access experience data for Web pages, including page load time, resource load time, etc.

1. Anomaly Detection Template

«Cloud Testing Data Detection» is used to monitor cloud testing data within the workspace. By setting threshold (boundary) ranges for the specified amount of data produced by testing tasks within a certain time period, alerts can be triggered when the data reaches the threshold range. At the same time, you can customize the alert level, and when the specified data reaches different threshold ranges, alerts of different levels can be triggered.

1. Self-hosted Node

DataFlux cloud testing self-hosted node adds a "Node Code" for obtaining node information. Within the current space, node codes cannot be duplicated.

New Registration Welcome Page¶

After registering and logging in to DataFlux, a welcome page is displayed where you can watch a short video introduction of DataFlux, and scan the QR code to join the DataFlux WeChat and DingTalk service group.

New Free Plan Support for DataFlux Func, Adjusted Billing Items¶

All versions of DataFlux fully support the DataFlux Func platform for custom functions. DataFlux Func is a script development, management, and execution platform based on Python class ServerLess.

At the same time, to reduce the cost of using DataFlux for users, DataFlux has adjusted the billing logic for application performance monitoring and user access monitoring. After the adjustment, application performance monitoring counts trace numbers, and user access monitoring counts daily session numbers.

Optimize Metric Generation¶

Add user performance monitoring, user access monitoring, security inspection metric generation, and optimize log metric generation. Metric generation can be configured through three steps: data filtering, data query, and metric generation, including configuring dimensions, aggregation time, defining aggregation frequency, metric set, and metric name generation.

Example:

1. Application Performance Monitoring

1. User Access Monitoring

1. Security Inspection

Optimize Built-in Views, Add Clone Function¶

Built-in views are divided into "System Views" and "User Views," supporting cloning from "System Views" to "User Views," and allowing system views and user views to have the same name. Clicking on a system view takes you to the system view details.

Click "Clone," open the clone view pop-up, and click OK to create the corresponding user view based on the current view template.

After successful creation, it navigates to the user view page created above.

If the user view and system view have the same name, and the system view is applied in the explorer detail view, such as the host view under "Infrastructure - Host - Host Details," the user view will replace the system view in the host view display.

Optimize View Variables¶

DataFlux supports adding global variables to views. When you want to dynamically filter charts in a view, you can choose to use view variables. After configuring view variables, you can choose whether to enable "Match All" in the "Advanced Settings."

After enabling, you can view a comprehensive view of all hosts through 「*」 and manually select variables for filtering.

Optimize Explorers¶

• When a single column does not fully display information, hovering the mouse for 1s will show all the information in a tooltip;
• Matching content in explorer search will highlight in the list;
• Detail page tabs support copy function; host tags support 【View Related Logs】【View Related Containers】【View Related Processes】【View Related Traces】【View Related Inspections】;
• Statistics charts in logs, application performance monitoring, user access monitoring, cloud testing, security inspection explorers are changed to status statistics;
• Add help links for enabling user performance monitoring, user access monitoring, and security inspection.

May 20, 2021¶

DataFlux Studio¶

New Snapshot Saving¶

DataFlux snapshots support creating snapshots for infrastructure, logs, events, application performance monitoring, user access monitoring, cloud testing, and security inspection explorers. With snapshots, you can quickly view and analyze data and abnormalities in different time periods and under different tags.

New Infrastructure Host Integration Interaction¶

The "Integration Runtime" in the infrastructure host shows the runtime status of the host's related collectors. There are two states for the runtime status of integrations: integrations running normally and integrations with errors. Normally running integrations are displayed in 'light blue' by default, and integrations with errors are displayed in 'red' and support clicking to view error information.

At the same time, clicking on integrations with built-in view symbols (as shown in the figure) supports "View Monitoring View," clicking which allows viewing the related built-in view.

New Self-hosted Node, Optimize Cloud Testing Node Display¶

«Cloud Testing» supports customizing cloud testing data collection through the DataFlux Studio console. By creating probes, you can periodically monitor websites, domains, backend interfaces based on HTTP protocol, and analyze site quality anytime and anywhere. DataFlux covers 16 nodes globally, classified into domestic, overseas, and self-hosted nodes. At the same time, it supports self-hosted testing nodes to fully ensure service availability.

New Process, Application Performance Monitoring, and User Access Monitoring Check Rules¶

«Process Anomaly Detection» is used to monitor process data within the workspace, supporting setting alerts for one or more field types in process data. For example, you can set alerts based on the number of times the 'host' field is 'izaqbin' and the 'state' field is 'sleep'.

«User Access Metric Detection» is used to monitor metric data in the «User Access Monitoring» of the workspace. By setting threshold ranges, alerts are triggered when metrics reach thresholds. Single metric alerts and custom alert levels are supported.

«Application Performance Metric Detection» is used to monitor metric data in the «Application Performance Monitoring» of the workspace. Alerts are triggered when metrics reach thresholds. Single metric alerts and custom alert levels are supported.

Optimize Integration¶

Log in to the DataFlux workspace, enter the «Integration» page to view all supported collectors. You can also click the "Quick Get DataKit Installation Command" in the upper right corner to directly obtain the DataKit installation command for installation. After installation, you can enable more collectors to gather data metrics.

Optimize Log Metric Generation¶

Log metric generation adds the selection of log sources. You can generate metrics through selecting log sources, filtering logs, configuring dimensions, aggregation frequency, defining metric sets, metric names, and aggregation rules. Example:

Optimize Log Viewer¶

After log data is collected to DataFlux, you can view all log sources and content in the DataFlux Studio console «Log» section. Support filtering and selecting the log details to be viewed through searching log sources.

Optimize User Access Monitoring Viewer¶

In the user access monitoring details page «Performance», you can view front-end page performance, including page load time, content rendering time, interaction time, input delay, etc. Taking the figure below as an example, the LCP (Largest Contentful Paint) metric reaches 5.43 seconds, while the recommended time is within 2.5 seconds, indicating slow page loading requiring optimization. Support for filtering and searching helps users quickly locate resources and content.

Switch to «Trace» to view requests sent to the backend application, including the occurrence time, content, and duration of resource calls. For example, you can view every ajax request returned by the xhr object.

Switch to «Error» to quickly locate front-end errors through detailed error information, error type, and occurrence time.

Optimize Built-in Views¶

Built-in views are divided into system views and user views.

System views are system-provided integrated view templates, supporting modification, export, and restoration. Users can directly «Export» system views or «Modify» system views. By «Restoring» system views, modified system views can be converted back to original system views. Restoration cannot be undone.

User views are custom views saved by users as templates, supporting modification, export, and deletion. In «Built-in Views» - «User Views», clicking «New User View» allows creating custom view templates. For specific configurations, please refer to Scenarios and Views.

Optimize Chart Lock Time¶

Support locking the time range of charts in the dashboard view,不受全局时间组件限制。设置成功后的图表右上角会显示用户设定的时间，如「xx分钟」、「xx小时」、「xx天」。

优化嵌套函数¶

所有图表组件的嵌套函数优化为内置函数，不再调用DataFlux Func的函数，通过DQL实现，删除嵌套函数时间间隔选项。

2021 年 5 月 7 号¶

DataFlux Studio¶

新增日志备份¶

DataFlux基础日志最多存储60天，如果需要更长时间的存储和查看需要对基础日志进行备份。DataFlux支持备份日志最多存储长达720天。

进入 «日志» - «备份日志» 页面，默认数据为空，选择时间范围即可查看对应的备份日志，DataFlux 支持通过选择时间范围、搜索关键字，筛选等方式查询和分析日志。点击日志可查看该条日志详情包括日志产生的时间和内容等。

新增安全巡检¶

DataFlux 支持你通过 «安全巡检» 及时监控、查询和关联全部巡检事件。在及时发现漏洞，异常和风险的同时，帮助你提高巡检质量、问题分析和问题处理的能力。

在开始安全巡检之前需要先安装[DataKit]及[security-checker]，配置完成后即可进入 «安全巡检» 并通过选择时间范围、搜索关键字，筛选等方式查询和分析。

新增新手指导¶

第一次注册登录DataFlux时，可以通过新手引导的方式开启使用DataFlux，根据引导步骤，用户可以自行完成datakit的安装，nginx数据的采集，在日志可查看对应的数据。新手引导关闭后，可通过点击工作空间的用户名称 - «新手引导»重新打开。

新增柱状图分组堆叠显示¶

图表堆叠柱状图

柱状图支持【指标堆叠】和【分组堆叠】，默认关闭，开启后，默认选中【指标堆叠】。右侧出现“方式”选项，包含默认、百分比。

• 指标堆叠：以指标为维度堆叠对比查看分组的标签数据
• 分组堆叠：以分组的标签为维度堆叠对比查看指标数据

事件查看器堆叠柱状图

DataFlux 支持通过柱状图堆叠的方式查看不同时间点发生的、不同状态的事件统计。

新增腾讯云、AWS云主机详情¶

若主机是云主机且配置了「云关联」，点击列表中的主机名称即可划出查看云主机对象的「云厂商信息」，包括云平台、实例名、实例ID、实例规格、地域、可用区、创建时间、网络类型、付费类型、IP地址等。

新增主机运行情况集成¶

主机集成运行情况共有三种状态：正常运行状态的集成、正常运行且存在内置监控视图的集成以及发生错误的集成：

• 正常运行状态的集成点击无反应
• 内置监控视图的集成点击后下拉“查看监控视图”，点击即可查看对应的监控视图。

• 发生错误的集成点击后可以查看最后一条错误反馈信息。

新增关联性分析内置页面¶

• 基础设施主机新增进程和容器内置页面
• 日志新增进程和链路内置页面
• 链路新增主机监控视图和日志内置页面

优化应用性能监测拓扑图¶

在 «应用性能监测» 的 «服务» 列表中，可切换列表至拓扑图模式查看各个服务之间的调用关系。

• 新增按时间轴选取时间范围
• 新增拓扑图放大缩小，放大状态可在左下角查看小缩略图
• 新增三种节点大小尺寸
• 新增选择服务后对应服务高亮显示，其他服务及连接线变灰

优化操作事件和通知事件¶

• 操作事件移至 «管理» - «基本设置» - «安全审计»，点击“查看”即可看到操作安全事件。
• 通知事件移至左侧菜单 «通知»，点击“通知”即可查看所有的通知事件。

优化付费计划与账单使用统计视图¶

• 付费计划与账单使用统计视图新增日志备份统计

2021 年 4 月 15 号¶

DataFlux Studio¶

启用新的 DataFlux Logo¶

采用机器人大脑的方式，不仅具有强烈的科技感官，突出 DataFlux 海量数据集成、处理、分析的强大能力，像机器人一样能实时、快速、无限扩展，为数据分析、业务运营分析等提供易于使用、灵活多样的可视化图表，提升运营效率。

新增云拨测¶

DataFlux 为用户提供开箱即用的云测解决方案，支持 HTTP 协议拨测，利用覆盖全球的监控网络，端到端的跟踪网页和应用性能等，记录所有拨测的实时反馈信息，提供DNS、SSL、TTFB等性能测试结果，帮助你提前发现问题，定位问题，提升用户体验。

DataFlux 云拨测支持从地理和趋势两个维度分析当前拨测任务的响应时间和可用率情况。

新增容器对象分布图¶

[容器对象]新增蜂窝分布图。在容器列表页面，通过点击列表左侧的图标，即可切换到容器分布图显示。通过分布图，你能够可视化查询容器分布信息。分布图支持以下功能:

• 搜索：输入搜索关键词
• 筛选：筛选标签并选择过滤对象，可多选
• 分组：分组展示对象，可多选
• 填充：选择填充的指标，指标值的大小将决定填充的图例颜色。
• 填充规则：根据填充指标的结果填充图例颜色。将指标的最大和最小值五等分，得到5个区间，分别对应五个颜色。

新增阿里云云主机详情¶

若主机是云主机且配置了 「云关联」，点击列表中的主机名称即可划出查看云主机对象的「云厂商信息」，包括云平台、实例名、实例ID、实例规格、地域、可用区、创建时间、网络类型、付费类型、IP地址等。

新增表格图指标排序¶

表格图支持指标排序，默认按第一个查询的指标排序，点击表头切换升降序，对应查询中的 Top/Bottom 同步调整，点击其他查询的指标进行排序，对应查询中的 Top/Bottom 同步调整。

新增链路快速筛选项「持续时间」¶

• 默认进度条最小值、最大值为 [链路] 数据列表里最小和最大的持续时间
• 支持拖动进度条调整最大/最小值，输入框中的值同步变化
• 支持手动输入最大/最小值，"按回车键"或"点击输入框外"进行过滤搜索
• 输入不规范时输入框变红，不进行搜索，正确格式：纯“数字”或“数字+ns/μs/ms/s/min”
• 若没有输入单位进行搜索，默认直接在输入的数字后面填入"s"然后进行过滤搜索
• 若手动输入单位，则直接进行搜索

新增邮件组¶

在 [通知对象] 管理新增邮件组通知对象，邮件组可同时添加多个成员，添加完成后在异常检测库的“告警设置”选择该邮件组，告警邮件即可同时发送到添加的成员邮箱。

优化异常检测库¶

DataFlux 内置多种检测库，开箱即用。支持主机等多种内置检测库，开启后，即可接收到相关的异常事件告警。

在 «检测库» 中支持 «导入/导出检测库»，配置完所有的检测规则后，可以通过 json 文件的方式导出检测库，导出的检测库可以导入其他的检测库，导入检测库时会验证是否存在重名检测项，可进行 «替换» 或 «跳过»。

优化对象查看器和详情¶

• 主机对象新增操作系统小图标显示
• 容器对象新增容器类型小图标显示
• 进程侧滑详情页新增显示「启动时间」

优化数据存储策略、付费计划和账单¶

• 工作空间管理的基本信息增加变更数据存储策略
• 工作空间管理的使用统计优化为付费计划和账单

2021 年 4 月 7 号¶

DataFlux Studio¶

新增视图变量对象映射¶

使用 «对象映射» 时，必须先定义一个基于对象类字段的视图变量，在 «视图变量» 中配置完对象映射以后，需要在 «图表查询» 中以映射的标签字段作为分组，在 «图表设置» 中开启 «字段映射»，然后就可以在视图图表中查看设置的对象映射显示。

优化事件分组¶

DataFlux 支持 «异常事件» 按照 «检测项» 分组聚合相关事件，点击查看 «聚合事件»，可以快速查看关联事件列表、不同主机的异常事件状态分布和指标监控等。异常事件状态分布包括 critial 、 error 、 warning 、 ok 和 静默 ，对应五种状态颜色，底色为绿色，按照事件发生的时间点向后填充对应状态颜色，填充范围即检测频率，当分组中存在host，且设置了主机静默，静默时间范围显示为灰色。

优化检测项查询¶

«异常检测库» 中 «检测规则» 的 «检测指标» 只允许添加一条查询，默认添加简单查询，支持简单查询和 «表达式查询» 来回切换；统一将 «检测规则» 的 «触发条件» M1修改为Result；支持为查询添加AS别名。

DataKit（v1.1.4-rc2）¶

• 修复阿里云监控数据采集器（aliyuncms）频繁采集导致部分其它采集器卡死的问题。

2021 年 3 月 25 号¶

DataFlux Studio¶

新增云服务关联¶

DataFlux 支持通过 «云关联» 统一集中管理云服务账号的 AccessKey 信息，配置完成以后可在 [基础设施-主机] 定期同步所属云账号下的云主机信息，展示所有云主机信息，如CPU、内存、网络、磁盘等。

若不再需要关联，可删除已关联的阿里云 RAM 子账号 AK 信息。关联信息删除后，会触发关联失效通知事件，可在 事件-通知事件 中查看。更多云关联的说明可参考文档 云关联 。

新增操作事件审计¶

DataFlux 支持对工作空间的操作事件进行安全审计，操作事件统一存储在事件-«操作事件» 中。

新增主机静默¶

若需要对单个云主机进行如压测等操作，会触发频繁的告警，对于这种告警通知，可临时开启主机静默功能。在 «基础设施» - «主机»，点击云主机详情即可选择主机静默时间。

所有静默的主机列表可在 «异常检测库» - «主机静默管理» 中查看。删除静默的主机后，可以重新接收到对应主机的告警通知。

新增通知对象：企业微信机器人¶

DataFlux支持用户管理发送告警的通知对象，目前支持添加邮件、钉钉机器人、企业微信机器人和Webhook自定义。

优化事件查看器，新增事件分组聚合查看¶

在事件列表查看器，通过分组功能，DataFlux支持快速依据分组标签聚合和统计相关事件。例如，在 «异常事件» 中，基于"检测项“分组，你可以快速获取基于该检测项触发的全部事件。通过切换聚合事件列表，你可以查看这些事件的详情。

优化链路服务筛选及拓扑图¶

在 «应用性能监测» 的 «服务» 列表中，支持切换列表至拓扑图模式查看各个服务之间的调用关系。支持通过不同的性能指标进行筛选显示，并可自定义链路服务性能指标颜色区间。

其他优化功能¶

• 优化日志、事件、基础设施、应用性能监测、用户访问检测查看器搜索和导出，新增导出到视图。优化日志、事件、链路、基础设施-容器、基础设施-进程详情页标签筛选、关联查询分析。
• 优化时间组件，优化场景视图下单查询图表（概览图、仪表盘、排行榜）简单查询和表达式查询互相切换
• 优化主机 offline 显示：当主机 offline 状态时，cpu使用率、mem使用率、负载均不作显示，值填充为“ - ”
• 优化异常检测库检测规则，新增水位检测、区间检测和突变检测。

DataKit（v1.1.4-rc1）¶

• 进程采集器 message 字段增加更多信息，便于全文搜索
• 主机对象采集器支持自定义 tag，便于云属性同步

DataKit（v1.1.4-rc0）¶

• 增加文件采集器、拨测采集器以及HTTP报文采集器
• 内置支持 ActiveMQ/Kafka/RabbitMQ/gin（Gin HTTP访问日志）/Zap（第三方日志框架）日志切割
• 丰富 [http://localhost:9529/stats] 页面统计信息，增加诸如采集频率（n/min），每次采集的数据量大小等
• DataKit 本身增加一定的缓存空间（重启即失效），避免偶然的网络原因导致数据丢失
• 改进 Pipeline 日期转换函数，提升准确性。另外增加了更多 Pipeline 函数（parse_duration()/parse_date()）
• trace 数据增加更多业务字段（project/env/version/http_method/http_status_code）
• 其它采集器各种细节改进

2021 年 3 月 4 号¶

DataFlux Studio¶

1. 优化时序图、柱状图、排行榜、日志流图、告警统计图、对象列表图¶

• 时序图：新增 «显示密度» 包括 «较低：60个点», «低：180个点», «中：360个点», «高：720个点»。新增 «堆叠显示»，仅支持柱状图，默认关闭
• 排行榜：图表显示优化
• 柱状图：图表显示优化
• 日志流图：增加搜索框，支持对message内容进行搜索
• 告警统计图：查询显示优化
• 对象列表图：查询显示优化

2. 新增自动转换单位¶

• 默认查询结果值自动转换单位展示遵循 «科学计数 K、M、B»，保留两位小数点（1千 = 1K，100万 = 1M，10亿 = 1B）
• 人民币自动转换单位 «元、万、亿»，保留两位小数点
• 设置单位后，取消科学计数，按用户设置的单位显示

3. 新增对象管理¶

DataFlux支持用户管理发送告警的通知对象，目前支持添加钉钉机器人通知配置和Webhook自定义通知配置两种方式。

4. 调整事件、日志、链路追踪、应用监测、对象主机、对象容器、对象进程、对象其他查看器，支持导出CSV文档，最大支持导出最近5000条数据。¶

• 事件

DataFlux 支持采集基于 «异常检测库» 所产生的所有事件数据。用户可在事件页面查看和搜索工作空间内所有的事件数据。

• 对象主机

DataFlux 支持采集对象主机数据。在 «对象» 的主机列表，可以查看和搜索所有的主机对象数据信息。

点击列表中的主机名称即可划出详情页查看主机对象的详细信息，包括基本信息，CPU，内存，网络，硬盘等。

• 对象容器

DataFlux 支持采集对象容器数据。在 «对象» 的容器列表，可以查看和搜索所有的容器对象数据信息。点击列表中的容器名称即可划出详情页查看容器对象的详细信息。

• 对象进程

DataFlux 支持采集对象进程数据。在 «对象» 的进程列表，可以查看和搜索所有的进程对象数据信息。点击列表中的进程名称即可划出详情页查看进程对象的详细信息。

• 对象其他

DataFlux 支持采集对象其他数据，如阿里云ECS等。在 «对象» 的其他列表，可以查看和搜索所有的其他对象数据信息。点击列表中的其他对象名称即可划出详情页查看其他对象的详细信息。

• 日志

DataFlux 支持采集各种格式的日志数据，包括系统日志、Niginx 日志、Apache 日志在内的多种日志数据，可快速查看和分析当前工作空间所有日志数据。

• 链路追踪

DataFlux 支持在 «链路追踪» 查看链路服务和链路列表，在服务列表中，可以查看或搜索全部接受链路数据的服务。在链路列表中，可以查询链路数据，导出链路数据，查看链路详情并通过火焰图、span列表等对链路性能进行全量分析。

DataFlux 支持同时支持浅色和深色的主题显示，这里以深色为例。

服务列表

链路拓扑

链路列表

火焰图

• 应用监测

DataFlux 支持采集Web、Android、iOS和小程序应用数据，并提供了应用监控场景，可快速查看和分析应用状况。以Web应用为例：

Web应用查看器

Web应用详情

2021 年 2 月 2 号¶

DataFlux Studio¶

• 新增时序图样式面积图
• 新增对象主机、容器、进程，新增主机蜂窝图
• 优化异常检测，新增内置检测库
• 优化事件、对象、日志、链路、应用查看器及页面功能
• 优化日志流图图表查询
• 优化排行榜、仪表盘、概览图图表查询，新增表达式查询模式。
• 优化可视化图表查询，简单查询和DQL查询支持互相切换
• 优化分组逻辑，包括散点图、气泡图、地图、蜂窝图、表格图
• 优化筛选条件支持实时过滤

DataKit¶

• 新增主机对象数据采集
• 新增容器对象数据采集
• 新增进程对象数据采集

2021 年 1 月 14 号¶

DataFlux Studio¶

• 新增Explorer布局，优化数据结构，支持对象/链路/RUM/日志等模块查询/分析
• 新增小程序应用监测
• 新增时序图，图表类型可以选择折线图或柱状图
• 调整图表UI
• 调整日志流图图表查询和设置
• 优化柱状图、排行榜、表格图相关设置
• 阈值检测调整为DQL的UI查询模式