Skip to content

Changelog (2021)


This document records the update content descriptions for each release of Guance, including DataKit, Guance Best Practices, Guance Integration Documentation, and Guance.

December 30, 2021

Optimized Binding Built-in Views

"Guance" supports associating built-in views with different link services, applications, log sources, projects, and other fields. Based on the bound fields, bound built-in views (system views, user views) can be viewed on the corresponding Explorer detail pages. Supported Explorers include scenario custom Explorers, Infrastructure, Logs, Links, User Access, Security Check, Availability, and other Explorers.

Optimized Log Explorer

  • In the Log Explorer, the source list on the left is expanded by default upon first login and can be manually collapsed. The Explorer will remember the last state by default.
  • In the Log Explorer, click the settings button on the left to manually add filter fields for quick filtering.
  • In the log data list, you can use "mouse hover" over the log content to expand and view the full content of the log. Click the "Copy" button to copy the entire log content to the clipboard. When expanded, if possible, the system will format the log JSON; if not, it will display the log content normally.

Optimized Explorer Detail Page Association Display

Optimized association display on the detail pages of Logs, Infrastructure, Links, and Security Check Explorers, including associations for hosts, metrics, links, containers, logs, Pods, etc. Supports two types of association display: fixed display and non-fixed display (requires containing association fields to display the association query).

Taking the association of hosts in the Log Explorer detail page as an example, the association field is "host". To view related hosts in the log details, the "host" field must be matched; otherwise, the related hosts page cannot be viewed in the log details. After field matching, basic host information and metric performance status can be viewed under the associated hosts.

  • Attribute View: Includes basic host information and integration operation status. If cloud host collection is enabled, cloud vendor information can also be viewed.

  • Metric View: Default performance metric views (e.g., CPU, memory) for the related hosts within the last 15 minutes can be viewed. Click "Open this view" to go to the built-in view, where the host view can be custom modified by cloning and saved as a user view. User views can be viewed on the log detail page through binding.

Optimized Snapshot Sharing

Snapshot sharing supports hiding the top bar of the shared page. In the snapshot list, click the share button to access advanced settings in the pop-up dialog box and select "Hide top bar".

Optimized Chart Query Results Display with Thousand Separator

Guance dashboard chart query results support automatic formatting with thousand separators. If a unit is set, the data format is displayed according to the set unit. After configuration, the data format can be previewed.

Other Optimized Features

  • Optimized host network topology map query performance. Added target domain name display on network topology map nodes. If the target host is not in the current workspace but the target domain name exists and the target domain name's port is less than 10000, the target domain name will be displayed in the topology map.
  • Optimized scenario dashboard and note Explorer favorite pinning logic:
  • Unfavorited dashboards are sorted in reverse order of creation time.
  • Favorited dashboards are sorted in reverse order of favorite time.
  • Favorited dashboards are displayed above unfavorited dashboards.
  • Optimized view variable cascade query logic: when two variables are linked for query, if the previous one selects *, the linked variable is not filtered and shows all value values.
  • Optimized the collapsed left navigation bar by adding a help documentation link to the navigation bar.

December 16, 2021

Added Time Series Chart Similarity Metric Query

In the analysis mode of the time series chart, supporting selecting "Chart Query" as the trend line/bar of a metric query allows you to "View Similar Trend Metrics". By selecting an absolute time range, similar metric trends within the space can be queried.

Added Explorer Detail Page Association Data Statistics Display

Added statistics display for the quantity of related logs, links, containers, etc., on Explorer detail pages. Supported directly in the tab titles of detail pages for Infrastructure, Logs, APM, User Access, Security Check, and other Explorers.

Added Explorer Chart Export Function

Guance supports exporting any chart from the "Guance" workspace Explorer to a dashboard, note, or clipboard.

Added Explorer Search Match Matching

Guance supports wildcard search in the Explorer search bar through field filtering like "host:cc*", which can be used for filtering log data from hosts with consistent naming prefixes. Also added a "delete" icon on the right side of the Explorer search bar, supporting one-click deletion of entered text.

  • "Guance" allows viewing all container (Container) data related to the host associated with a log within the last ten minutes via the "Containers" section below the log details page.
  • Related link and container data on the log details page support filtering by "trace_id" or "host".
  • Content display on the log details page supports both Json and text viewing modes, automatically displayed based on the message type.
  • Approximate text Explorer allows viewing details of logs associated after Pattern clustering.
  • Removed color from mouse hover tooltips in the Log Explorer.

Optimized Mute Strategy, Supporting Mute Settings for Monitor Rules

Guance supports users managing all mute rules in the current space through "Mute Management" in the "Monitoring" function. "Guance" provides three mute types: Host Mute, Monitor Mute, and Group Mute, supporting mute management for different hosts, monitors, and groups, ensuring muted objects do not send alert notifications to any alert notification targets during the mute period.

Optimized Monitor Management

Duplicate monitors in the monitor list can affect users' ability to quickly locate monitoring objects. "Guance" supports choosing whether to create duplicate monitors from templates. Additionally, users can use the new "Batch Management" tool to custom export/delete monitors.

Optimized View Variable Query Display

  • Added default values for dashboard view variables, supporting previewing all variables of the current view variable and selecting the initial variable to view by default on the dashboard.
  • Supports dragging view variables by mouse hover (hover) to adjust their order.
  • Removed the "Settings" button for view variables; functions like "Sort", "Hide", and "Delete" can be used directly in the view variable list.

Optimized Dashboard and Note Chart Lock Time Settings

Guance supports adding a "More" option in the dashboard and note chart lock time settings to custom define the locked time range. After this optimization, the lock time component functionality is consistent with the chart component.

Optimized Help Center Page Entry

In the Guance Help Center, you can quickly view product documentation, product updates, blogs, live broadcast reviews, and other information. You can also join our Guance official service group by scanning a QR code.

December 2, 2021

Added Owner Member Role

Guance added the Owner role for the current workspace. Four types of workspace member permissions are currently supported: "Owner", "Administrator", "Standard Member", and "Read-only Member", each with constraints on management permissions, configuration permissions, operation permissions, and viewing permissions for different member categories. The "Owner" role has the highest operation permissions, can designate current space "Administrators", and perform any management, configuration, operations, and data viewing. In the Guance workspace, go to "Management" - "Member Management" - "Modify" to edit and update member permissions.

Added Custom Explorer in Scenarios

"Guance" provides a quickly built Explorer based on log data range in Scenarios, supporting space members jointly building Explorers based on custom scenarios for customized viewing needs. Completed "Explorers" can be exported and shared with others, sharing Explorer templates.

Added Export User Views to Dashboard

"Guance" supports exporting created user views as a json file in "Built-in Views". The json file can be used for importing into scenarios or built-in views in different workspaces.

Optimized Host Network, Added Network Flow Data Viewing List

"Guance" supports displaying different visual charts based on the selected protocol in the "Network" section of the Infrastructure "Hosts" detail page, and allows customizing the display fields of the network flow data list.

By clicking "View Network Flow Data", you can view network flow data from the last 2 days by default, including time, source IP/port, target IP/port, source host, transmission direction, protocol, sent bytes, etc. Supports customizing display fields or adding filter conditions to filter all string-type keyword fields.

Added Sourcemap Function

Applications are often transformed and compressed during the packaging process in production environments to prevent code leakage and other security issues. While these measures protect code security, they also result in collected error stack information being obfuscated, making it difficult to locate problems directly and hindering subsequent bug troubleshooting.

To solve the above problems, "Guance" provides a sourcemap function for web applications, supporting the restoration of obfuscated code to facilitate source code location during error troubleshooting, helping users solve problems faster.

Added Host-associated Security Check Data Analysis

"Guance" supports viewing security check data related to the host within the last day, and performing keyword search, multi-tag filtering, and data sorting on this security check data.

Added DataKit Collector Kubernetes Installation Guide

"Guance" supports adding a guide for installing the DataKit collector in a Kubernetes cluster in the workspace "Integration" - "DataKit".

"Guance" supports adding attributes to the current filter or copying (copying the tag to the local clipboard) on the link detail page and the user access page performance detail page (User Access Monitoring view Explorer).

Optimized Registration and Upgrade Process

Guance added the option to directly select and activate the Guance billing platform during account registration, and optimized the process for upgrading to the Agile Edition after logging into the workspace.

Billing Price Adjustment

Guance uses the latest technology to optimize data storage costs, leading to a price reduction adjustment for billing. A new gradient billing model based on data storage strategies has been added, including three billing items: log data, APM Trace, and User Access PV.

Furthermore, to assist enterprises in obtaining comprehensive IT infrastructure, application systems, and other corporate assets at more favorable prices, Guance offers three package plans based on different stages of enterprise development: Startup Acceleration Pack, Growth Pack, and Enterprise Standard Pack, along with add-on data packs for enterprises to use according to their needs and package叠加.

Other Optimized Features

  • Added LCP, FID, CLS, FCP related detection indicators for monitor user access indicator detection. In the Guance workspace "Monitoring" - Monitors, click "+ New Monitor", select "User Access Indicator Detection" to enter the configuration page for detection rules.
  • The grouping in the monitor simple query has been changed to detection dimension. The detection dimension determines which dimension the detection rule is based on for triggering, i.e., the trigger object. "Guance" supports adding multiple detection dimensions; an alert is triggered if the indicators of any detection dimension meet the alert condition.
  • Optimized navigation menu display. When the navigation bar of the "Guance" workspace is collapsed, the secondary menu displays the navigation name of the function flush left.

November 18, 2021

Added SSO Login

Guance supports users creating an identity provider based on the workspace. Users can enter their company email during login to obtain the corresponding SSO login link. Clicking the SSO login link enables corresponding verification login. In the Guance workspace, go to "Management" - "SSO Management" - "New Identity Provider" to set up SSO single sign-on for employees.

Added SLO Charts to Dashboards and Notes

Added SLO charts to scenario dashboards and notes. SLO charts can directly select set monitoring SLOs for SLO data display. By selecting different SLO names, SLO data results are displayed synchronously.

Added Custom Grouping Field Addition for Infrastructure Grouping

In the Guance Infrastructure Explorer, click the settings button to the right of the grouping to custom add grouping tags. After adding, you can choose to group and aggregate displays according to the added grouping tags. Note: Only administrators can add custom grouping tags.

Added SLO Notification Mute Configuration

If the same event is not very urgent but has a high notification frequency, the notification frequency can be reduced by setting notification mute. After notification mute is set, events will continue to be generated, but notifications will not be sent. The generated events will be stored in event management.

Guance supports adding built-in links and custom links to charts. Through links, you can jump from the current chart to the target page, and use template variables to modify the corresponding variable values in the link to transmit data information, completing data linkage. Chart links support opening in new pages, current pages, or side-slide detail pages to achieve linkage analysis.

Billing Price Adjustment

  • Daily DataKit unit price reduced to 3 yuan per unit.
  • Availability API test adjusted to 1 yuan per 10,000 times.

Other Optimized Features

  • Launched China Region 1 (Alibaba Cloud) free new version: The Free Plan is divided into Alibaba Cloud login and AWS login. Alibaba Cloud login timeline data retention policy is 30 days, other data retention policy is 7 days. AWS login all data retention policy is 1 day.
  • Optimized SLO deduction logic: Disabled monitors are not included in the deduction scope.
  • Added dashboard tag functionality: When creating/modifying dashboard views, custom tags can be added to dashboards to help with quick filtering.
  • Optimized log highlighting: When searching logs in the search bar, the returned list only retains highlight display for matched keywords.
  • Optimized Explorer search: Added help documentation links to the Explorer search dropdown.
  • Optimized text chart Markdown format: Added support for unordered lists and ordered lists, optimized display format.

November 4, 2021

Added Scenario Notes

Under Scenarios, multiple notes can be created for summary reports. Supports inserting real-time visual charts for data analysis and inserting text documents for explanation, combining charts and documents for data analysis and summary reports. Supports exporting data from Infrastructure, Logs, APM, User Access, and other Explorers to notes for data analysis. Supports sharing notes with all members of the workspace, retaining abnormal data analysis, helping with backtracking, locating, and solving problems.

Scenario Views Upgraded to Dashboards

Guance's scenario views have been upgraded to dashboards, and the functionality and layout of the dashboards have been adjusted. Under Scenarios, multiple dashboards can be created to build data insight scenarios. Supports keyword search for dashboards, supports adding visual charts to dashboards for data analysis, supports quickly filtering and finding corresponding dashboards from "My Favorites", "Import Project", "My Creations", and "Frequently Browsed".

Added Export to Dashboard and Note from Explorers

Infrastructure, Log Monitoring, APM, Availability Monitoring, Security Check, and other Explorers added export to dashboard and note functionality.

Anomaly Detection Library Upgraded to Monitoring

The original "Anomaly Detection Library" has been upgraded to "Monitoring". The original names "Anomaly Detection Library", "Detection Rules", etc., have been changed. Specific changes are as follows:

  • Monitors (originally "Anomaly Detection Rules") provide various monitoring methods such as "Threshold Detection" and "Log Detection", allowing users to custom configure detection rules and trigger conditions, and receive alert notifications第一时间 through alerts.
  • Templates (originally "Built-in Detection Library"): "Guance" has built-in多种开箱即用的 availability monitoring templates, supporting one-click creation of Docker, Elasticsearch, Host, Redis monitoring. After successfully creating a template, the corresponding official monitor is automatically added to the current workspace.
  • Groups (originally "Custom Monitoring Library"): The grouping function supports you in custom creating meaningful monitor combinations for convenient group management of various monitors.

Added Monitoring SLO Function, Supporting Export to Dashboard

"Monitoring" added SLO monitoring, supporting service quality scoring for system service providers (Provider) in the cloud era, comparing and detecting whether the corresponding SLI (Service Level Indicator) meets target needs. Meanwhile, "Guance" also supports exporting SLO as a view to the dashboard for convenient SLO monitoring同步 on the dashboard.

Other Optimized Features

  • Chart optimization:
    • Removed grouping selection from overview charts.
    • Adjusted chart style selection for time series charts, pie charts, and bar charts.
    • Added prompt information for time slicing.
    • Chart json supports editing, and query/setting linkage. Supports validation of input Json; error messages are displayed if there are errors.
    • Optimized styles in the time series chart analysis mode.
  • Added prompts for Paid Plans and Bills. If the settlement method is aliyun or aws, the bill list directly displays跳转到对应云服务控制台 for viewing bill information.

October 21, 2021

Added Host Network Distribution Map

Added a host network distribution map, providing infrastructure network monitoring. In "Infrastructure" - "Hosts", click the small icon in the upper left corner to switch to viewing the host network distribution. In the "Network Distribution Map", you can visually query the network traffic between hosts in the current workspace, quickly analyzing TCP latency, TCP fluctuation, TCP retransmission count, TCP connection count, and TCP close count between different hosts.

Added User Access Monitoring Trace Function

Added a User Access Monitoring "Trace" function. Supports users creating trace tasks through "User Access Monitoring" to monitor custom link trace trajectories in real-time. By pre-setting link trace trajectories, link data can be集中筛选, user access experience can be精准查询, and vulnerabilities, anomalies, and risks can be及时发现.

Added Scenario Chart Json Format Query

When editing charts in "Scenarios", each correct chart query corresponds to a json text. Supports mutual parsing between json text and chart queries within the workbench, helping you gain insight into chart drawing details.

Optimized DCA Desktop Client Application

Optimized DCA desktop client UI display, added host status.

  • online: Indicates normal data reporting. DataKit operation status and configured collectors can be viewed through DCA.
  • unknown: Indicates remote management configuration is not enabled, or not on the same LAN.
  • offline: Indicates the host has not reported data for over 10 minutes, or the host name was modified. The original host name will显示成 offline status. Hosts not reporting data normally will be removed from the list if they exceed 24 hours without data reporting.

Optimized Event Host Linkage Query and Detection Rule Query

Optimized the event module, supporting linkage query for hosts associated with events. Allows real-time monitoring of logs, containers, processes, links, and inspection events related to the host associated with the event within the selected time component range. Supports one-click viewing of detection rules associated with the event and querying other triggered events under the corresponding detection rule.

Adjusted Billing Method

  • Choose the billing settlement method based on the node login method, including CloudCare account, Aliyun, AWS. Among them, CloudCare account settlement is the general method. Aliyun login nodes can correspondingly choose Aliyun account settlement method (under development). AWS login nodes can correspondingly choose AWS account settlement method (already launched).
  • Removed Session count and Browser test count billing dimensions. Removed Session count and Browser test count statistics and views from "Paid Plans and Bills". Added security check count statistics chart.

Other Optimized Features

  • Optimized scenario add chart function, removed the "Add Chart" button in "Edit" mode. All addable chart types are optional at the top of the scenario.
  • Optimized scenario chart grouping function, supports deleting only the group while retaining the original charts under the default group.
  • Added "Region Ranking" switch for China map/world map, region ranking is off by default.
  • Optimized Explorer query method, merged search and filter bar, supports keyword search, tag field filtering, and association search based on tags, fields, and text.

September 28, 2021

Added Host Network Performance Monitoring

Host network performance monitoring supports viewing network traffic between host services, containers, and any tagged infrastructure. Supports viewing network traffic and data connections from source hosts to targets based on IP/port, displaying in real-time visually, helping enterprises understand the network operation status of business systems in real-time, quickly analyze, trace, and locate fault problems, preventing or avoiding business problems caused by decreased or interrupted network performance.

After host network data is successfully collected, it is reported to the Guance console. In "Infrastructure" - "Hosts" detail page - "Network", you can view all network performance monitoring data information within the workspace.

Added Chart Linkage Query

In scenario views, supports clicking the mouse to view the DQL query statement of the chart, associated links, etc., or linkage querying related logs, containers, processes, links, etc., under different grouping query tags. When the chart query contains host (host) information, supports viewing related host monitoring views. For example, clicking any chart in the view allows viewing the chart's DQL query statement, set links, and linkage querying related logs, containers, processes, links, host monitoring views, etc.

Added Treemap Chart

Treemap charts are used to visualize the proportional distribution of metric data under different groupings. Treemap color represents the grouping tag (Tag), and area represents the metric data. When multiple metric queries are added, the main displayed metric can be selected. The area size is displayed based on the selected metric data results, Top/Bottom, and quantity. For example, this chart visually displays the CPU usage ranking of hosts.

Added APM and User Access Monitoring Association Query

Within APM link details, supports viewing associated (same trace_id) real user access experience data via the "Related view" above the page. This function not only helps you view proactively monitored APM data but also helps you quickly透视 real user access situations. More APM can be referred to the document Link Analysis.

Added Absolute Time Range Query for Infrastructure

Infrastructure added "Hosts", "Containers", "Processes" query absolute time range. By default, displays host data from the last 24 hours, container data from the last ten minutes, and process data from the last ten minutes. The query range can be refreshed manually. Hovering the mouse over offline hosts displays host offline processing prompts.

Optimized Chart Period-over-Period Comparison Function

Optimized the chart period-over-period comparison function, supporting comparison of period data for time series charts, overview charts, etc. Options include year-over-year (comparison of the same time point in adjacent time periods) or month-over-month (comparison of adjacent time periods). Based on the chart lock time, options like month-over-month, day-over-day, week-over-week, month-over-month, week-over-year, month-over-year can be selected.

Optimized Chart Query SLIMIT

Added SLIMIT by default for grouped query charts to limit the number of points or series returned by the time series chart query. One chart limits returning 10000 points, one query returns up to 10 time series at most.

Cloud Testing Adjusted to Availability Monitoring

Cloud testing is divided into API tests and Browser tests. After adjusting to availability testing, API tests are retained. Other functions such as overview analysis, Explorer analysis, and self-built node functions remain unchanged. For more details, refer to the document Availability Monitoring.

Billing Price Adjustment

  • Log price adjusted from 1 yuan per 1 million lines to 0.5 yuan per 1 million lines.
  • User Access Monitoring price removed Session billing dimension, adjusted to 0.3 yuan per 1000 PVs.

September 9, 2021

DataFlux Renamed to "Guance"

DataFlux, as the observability platform launched by ClouGence for the cloud era, has been officially renamed to "Guance" with this release to make it easier for enterprises and users to understand. The website is "guance.com".

We have also designed a new Logo for "Guance", based on the letters C and G forming a图形 similar to radar waves.

Added Log Approximate Text Explorer

"Guance" supports similarity calculation and analysis on the message field of原始日志数据. Based on the time range selected in the upper right, the current time period is fixed, and 10000 log data entries within this time period are obtained for approximate analysis. Logs with high similarity are aggregated, common Patterns are extracted and statistically clustered, helping to quickly discover abnormal logs and locate problems.

Pattern Clustering Details

Click the Pattern clustering list to划出 the current all Pattern clustering list. The returned data list is arranged in reverse chronological order, supports displaying the last 50 log data entries, and can be displayed by choosing 1 line, 3 lines, or 10 lines.

Added Container Jobs and Cron Jobs Explorers

"Guance" added container "Jobs" and "Cron Jobs" Explorers. In "Infrastructure" - "Containers", select the corresponding Explorer from the left object list to view the running status and service capability related indicators of Jobs and Cron Jobs in Kubernetes, thereby achieving real-time monitoring of Kubernetes clusters and various resources deployed within them. For details, refer to the document Containers.

Added DCA Desktop Client Application

DCA, DataKit Control APP, is a desktop client application designed to facilitate the management of installed and configured collectors. It supports viewing the integration list, configuration file management, Pipeline management, online viewing of integration configuration documents, and other functions. You can use DCA to remotely connect to DataKit, change collectors online, and save updates to take effect after changes are completed.

In the "Guance" workspace, click "Integration" - "DCA" to download the installation package. After downloading, it can be installed on your computer to remotely manage DataKit.

Added Mobile APP

DataFlux Mobile可以帮助您在移动设备上随时随地查看来自Guance的日志数据、场景视图等,接收通过邮件、钉钉机器人、企业微信机器人、Webhook等通知到您的全部告警事件。

在Guance的工作空间内,您可以通过「集成」-「移动端」进行 DataFlux Mobile APP的下载。

Added Tag Filtering Function for Metrics, Optimized Three Viewing Modes for Metrics

The DataKit collector will automatically append the tag host=<hostname where DataKit is located> to all collected data by default.

Tags are sets of properties that identify the object from which a data point was collected. Tags are divided into tag names and tag values. In the "Metrics" page, you can select the tag value corresponding to the metric view you want to view. As shown in the figure below: Filter hosts in the tag bar and view their usage_system, usage_total, usage_user, usage_iowait, and other metric views.

On the "Metrics" page, you can switch the metric viewing mode on the left. Three viewing modes are supported: Tiled mode, Mixed mode, and List mode. Tiled mode is selected by default.

Optimized Anomaly Detection

Added window function usage in anomaly detection rule queries and chart queries. This means using the selected time interval as the window (record set) and the detection frequency as the offset to re-perform statistical calculations on each record.

• When performing chart queries, supports adding window functions, i.e., using the selected time interval as the window (supports choosing 1 minute, 5 minutes, 15 minutes, 30 minutes, 1 hour, 3 hours), combined with aggregate functions to perform statistical calculations on each record. • When configuring anomaly detection rules, added window function as the drawing display area for query results, returning real-time anomaly detection indicator data used to trigger alerts.

Added percent() function usage in aggregate functions for anomaly rule queries and chart queries. Supports adding the percent() function in aggregate functions, including p50 (取中位数值), p75 (取处于75%位置的值), p90 (取处于90%位置的值), p99 (取处于99%位置的值).

Added label filtering usage in anomaly rule queries and chart queries. Supports selecting host label attributes for filtering display. Before selecting labels in queries, you need to set label attributes for hosts in "Infrastructure" - "Hosts".

Optimized Unrecovered Event Display Style

In the unrecovered event list, you can preview the window function for the last 6 hours of the event: • The display effect of the dashed border is the time period affected by the abnormal event. • When the detection library rule type is threshold, log, APM indicator, user access indicator detection, security check, abnormal process, or cloud test detection, color blocks corresponding to different alert levels can be used to view related anomaly detection indicator data, including critical, error, warning. • When the detection library rule type is突变, interval, or level, the trigger time point of the current event can be quickly identified based on the "vertical line" in the chart.

The event details page supports viewing the status distribution of abnormal events, DQL functions, and window function line charts. • Status distribution: Displays the event status (critical, important, warning, data gap) within the selected time range (default last 6 hours). • DQL query statement: Real-time indicator data returned based on the custom query statement of the anomaly detection rule, default display of real-time indicator data for the last 6 hours. • Window function: Based on the anomaly detection rule, using the selected time range as the window (record set) and the detection frequency as the offset, re-performs statistical calculations on each record, returning real-time anomaly detection indicator data used to trigger alerts. Default display of real-time anomaly detection indicator data for the last 6 hours.

Optimized Scenario View Analysis

In preview mode, double-clicking the chart enlarges it into analysis mode for viewing. In edit mode, double-clicking the chart enters edit mode for making additions, deletions, modifications, and queries to the chart.

Optimized Infrastructure Binding Built-in Views

When selecting built-in views, duplicate named views are filtered. When system views and user views have the same name, user views are prioritized for display, meaning only user views can be selected. If you need to select a system view, you need to change or delete the duplicate named user view.

Optimized Billing Method, Added Cloud Test Billing, Adjusted Data Expiration Policy

  • Added PV count statistics: Session count and PV count are charged based on the lower of the two actual costs.
  • Added cloud test task count cost statistics: Open cloud test self-built node management, supports workspace administrators to self-build new test nodes globally. Cloud test billing dimension will be based on the number of cloud test task calls in the current workspace. Free Plan workspaces can create up to 5 test tasks and only support the use of "China Region" test nodes. Agile Edition and above workspaces support creating more test tasks and using more国外 test nodes.

August 26, 2021

DataFlux Studio

Added Chart Query Label Filtering

DataFlux supports selecting host label attributes for filtering display. Before selecting labels in chart queries, you need to set label attributes for hosts in "Infrastructure" - "Hosts". As shown in the figure below, there are two hosts containing the "production" label attribute, so data from these two hosts is displayed.

Added Infrastructure Container Cluster, ReplicaSet, Node Explorers

DataFlux added container Cluster, ReplicaSet, Node Explorers. Supports viewing the running status and service capability related indicators of Clusters, ReplicaSets, and Nodes in Kubernetes, thereby achieving real-time monitoring of Kubernetes clusters and various resources deployed within them.

Added User Access Monitoring View Explorer Quick Filtering

DataFlux supports, in the View Explorer, clicking the settings button for quick filtering on the left to custom add filter fields. After selecting the fields to add, click "Done".

Note: If the added filter field represents time, supports filtering and viewing corresponding page content by selecting a time interval. For example, loading_time (page loading time).

Added Navigation Menu Help Entry, Added Multiple Explorer Data Collection Guide Entries

DataFlux added a help documentation entry in the navigation menu. Meanwhile, when no data is uploaded in an Explorer, DataFlux provides quick access to "How to start data collection" entries to help you promptly start the real-time monitoring journey in different Explorers. This includes how to enable log monitoring, how to enable cloud testing, how to collect host objects, etc.

Optimized Log Explorer

DataFlux optimized the Log Explorer, using different colors to highlight different parts of the log, allowing you to quickly obtain key data information from the log. Supports both light and dark theme colors.

Log Content Light Theme Dark Theme
Date (time the log occurred) Yellow Light Yellow
Keywords (HTTP protocol related, e.g., GET) Green Light Green
Text (quoted strings) Blue Light Blue
Default (text without any indication) Black Gray
Numbers (log status codes, etc., e.g., 404) Purple Light Purple

Optimized Security Check Overview Statistics

DataFlux supports customizing the views on the security check overview page. By clicking the "Jump" button, you can jump to the corresponding built-in view on the overview page for viewing, and edit, copy, and export that view.

Optimized Workspace Switching

DataFlux supports switching to other workspaces or creating new workspaces by clicking the "Current Space" name.

Optimized System Theme Color Switching

DataFlux added an "Auto" option for system theme color, which can automatically switch the theme color based on the computer's appearance settings.

Other Optimized Features

  • Optimized APM "Service" details. On the service details page, clicking the service's resource name可以直接跳转到「链路」 to view the corresponding resource's link situation.
  • Optimized User Access Monitoring View Explorer "Performance" details. Clicking error data (data with an "error"标志 in the View list) in the performance details allows viewing the details corresponding to the error.
  • Optimized metric Explorer tag name and tag value display.
  • Optimized time series chart X-axis time display. Default time range is last 15 minutes, X-axis segmentation is by minute interval; last 1 day, X-axis segmentation is by hour interval; last 7 days, X-axis segmentation is by day interval.
  • Optimized expression queries. When performing chart queries, if the expression query contains multiple query statements, the grouping tags must be consistent.
  • Optimized host integration operation status. Added DataKit version information for the host installed in "Infrastructure" - "Hosts" - "Integration Operation Status".
  • Optimized snapshot sharing. Supports sharing snapshots with "anyone" who has the sharing link, or encrypted sharing with those who have both the link and the key.

August 12, 2021

DataFlux Studio

Added 3 Permission Roles, Optimized Scenario Viewing Permissions

DataFlux currently supports defining three types of workspace member permissions: "Administrator", "Standard Member", and "Read-only Member", each with constraints on management permissions, operation permissions, and viewing permissions for different member categories. "Administrators" can custom define the permission scope of members, such as limiting the scenario views accessible by read-only members, limiting the scenario views editable by standard members, etc.

In the DataFlux workspace, go to "Management" - "Member Management" - "Modify" to edit and update member permissions.

Added Infrastructure Host Detail View Connection Tracking and File, and Layout Optimization

DataFlux supports viewing host system information in the infrastructure host details page, including host name, operating system, processor, memory, network, hard disk, as well as connection tracking count, file handle count, and other data.

Added Scenario View Snapshot Save Function

In the DataFlux workspace, enter "Scenarios" - "Nodes", click the "Settings" button in the top navigation bar, and select "Save Snapshot" to save the data content currently displayed by the view as a snapshot.

Additionally, DataFlux supports quickly saving snapshots through page function buttons and shortcuts (Ctrl+K). In scenario views and all Explorers, use the shortcut Ctrl+K to quickly pop up the [Save Snapshot] window. Enter a name, click [OK] to add the current page as a new snapshot.

Note: If your background program has other shortcuts that conflict with the shortcut Ctrl+K, you will not be able to use the "Save Snapshot" shortcut function.

Added Chart and Snapshot Sharing Management

DataFlux supports unified management of shared charts and snapshots within the current space. For example, after completing chart sharing in "Scenario Views", you can view the chart sharing list within the current space through "Management" - "Sharing Management" - "Chart Sharing", and perform chart viewing, view embedded code, and cancel sharing.

Added User Email Change, Phone Change Unbinding Function

In the DataFlux workspace, click the lower left corner "Account" - "Settings" - "Basic Settings" to change or unbind your phone and email.

Added Custom User Access Monitoring SDK Collection Data Content

DataFlux supports customizing the data content collected by the User Access Monitoring SDK. By custom setting user identifiers, custom setting sessions, and custom adding additional data TAGs, it helps users locate and analyze data in specific scenarios by setting different types of identifiers.

Added Paid Edition Session, Task Scheduling, Trace Free Quota

DataFlux paid editions新增每台 DataKit 每天赠送 100 万条日志类数据、100 个 Session 数量、1 万次任务调度次数以及 100万个Trace数量。

Optimized Scenario Chart Analysis Mode

DataFlux supports, in the analysis mode of the time series chart, previewing the interactive changes of object data within four times the query cycle through the time axis below the chart, and selecting the time range displayed by the time series chart by dragging the time axis. For example: the current time point is 14:00, the time range selected is [10:00-11:00], then the time axis range is [08:00-12:00].

Note: A time series chart query statement returns up to 10 time series at most, i.e., based on the grouping (group by) condition results. Data exceeding 10 time series is displayed in order, showing only 10 time series.

Optimized Infrastructure Host, Container Honeycomb Chart

Hover the mouse over host or container objects to view the host/container name, CPU usage rate, and MEM usage rate.

Optimized Log Source List Display Based on Explorer Time Range

DataFlux supports filtering the log source list by selecting the Explorer time range. For example, in the DataFlux workspace "Log Explorer", select 15 minutes in the time range in the upper right corner. The log list on the left only displays log sources that have data within 15 minutes.

Optimized System Theme Color Switching

DataFlux supports switching system theme colors, divided into light and dark两种. Click the account in the lower left corner, and click the desired theme under "System Theme" to switch directly.

Other Optimized Features

  • Webhook custom notification sending content type changed from text format to json format.
  • Added a 15-minute alert mute option in the anomaly detection library alert settings, meaning within a 15-minute time range, the same alert will not send notifications.
  • Adjusted the display style of the unrecovered event list page. Log alert event statistics display changed from table chart to time series chart.

August 4, 2021

DataFlux Studio

m

Added Container Services and Deployments Explorers

DataFlux supports comprehensively displaying container status from different dimensions through "Container", "Pod", "Services", "Deployments", and other Explorers.

In the workspace "Infrastructure" - "Containers", switch to the "Services" Explorer via the data type filter bar in the upper left to view detailed information of all Services retained in the space, including Service name, service type, Cluster IP, External IP, running duration, etc.

Switch to "Deployments" to view detailed information of all Deployments retained in the space, including Deployment name, available replicas, upgraded replicas, ready status, running duration, etc. You can switch to the honeycomb chart for viewing.

Click the Pod in the honeycomb chart to view the details of that Pod, including Pod name, namespace, visual monitoring charts, etc.

July 29, 2021

DataFlux Studio

Added Infrastructure Global Label Function

DataFlux added an infrastructure global Label function, supporting users in adding custom tags for host, container, process, and other infrastructure objects, and filtering and displaying infrastructure objects with the same tags through the added tags.

In the infrastructure object list, click the object name to划出 the details page and add "Label attributes" tags for the object.

After custom tags are added, you can filter and display object lists with the same tags in the object list by adding the tags.

Added SMS Notification Objects; Added SMS Cost Statistics for Paid Plans and Bills

DataFlux supports users managing alert notification objects sent. Currently supports adding DingTalk robots, WeCom robots, Lark robots, Webhook custom, mail groups, and SMS groups.

Note:

  • Members for SMS notifications need to be invited to join the workspace in "Management" - "Member Management" first before they can be selected.
  • DataFlux Free Plan does not have SMS notifications. For other editions, SMS notifications cost 100 messages / 10 yuan, billed daily.

Added Scenario View Slideshow Mode

DataFlux added a scenario view slideshow, supporting using slideshow mode to browse all node views under a specified scenario, and customizing the time interval for the slideshow.

Added Workspace Token Change Function

DataFlux supports current space administrators changing the Token within the space and customizing the expiration time of the current Token. Go to "Management" - "Basic Settings" - "Change Token", select the expiration time and confirm "Change". DataFlux will automatically generate a new Token. The old Token will expire within the specified time.

Note:

  • After changing the Token, the original Token will expire within the specified time. If any agent uses this Token, data reporting will stop immediately. Please check the integration configuration promptly.
  • Changing the Token will trigger "Operation Events" and "Notifications".

Added Metric DQL Query Mode

DataFlux supports switching "Search" to "DQL Query Mode" in the workspace "Metrics", then directly entering DQL statements for querying. The query can view corresponding tags and metrics on the left, and view the visual charts returned by the query statement on the right.

Optimized Log Explorer Multi-line Display

DataFlux supports selecting log display as "1 line", "3 lines", or "10 lines" in the "Settings" of the Log Explorer to view complete log content.

DataFlux supports retaining the current filter conditions (including: environment, version) and time range when switching between "Service" and "Link" Explorers. For example: In APM "Service", select "Environment" as production_a, "Time Range" as Last 30 minutes.

Switch to APM "Link", the filter conditions remain consistent with "Service".

Optimized DataKit Collector Installation Guide

DataFlux supports custom installation options when installing DataKit, such as cloud vendor, namespace, global tag, etc. DataKit offline installation supports directly clicking to download offline installation packages for different operating systems.

Optimized Flame Chart Synchronous and Asynchronous Call Display

DataFlux APM flame charts support viewing service synchronous and asynchronous call link performance data details. For example, through the flame chart, you can clearly see which requests are processed asynchronously, when they start, when they end, and how much time they take in total.

Optimized Explorer Display Column Custom Field Aliases

DataFlux supports custom aliases for fields displayed in Explorers. For example, in infrastructure hosts, users can modify field aliases in "Add Display Column". After modification, they can be viewed in the Explorer and details.

Optimized Cloud Test Self-built Nodes

DataFlux cloud test self-built node geographic location is split into two filter boxes: country and province (or city). If the country filter box selects China, the latter filter box will filter out a list of provinces. If the country selected is not China, the latter filter box will filter and display a list of cities. The filter box displays 20 items by default and supports fuzzy search.

Optimized DQL Query Functions

DataFlux DQL added exists, wildcard, with_labels, and other query functions.

July 15, 2021

DataFlux Studio

Added User Access Third-party Development Tool Access

DataFlux added access based on the uniapp development framework for mini-programs. By introducing the SDK file, it monitors the performance indicators, error logs, and resource request situation data of the mini-program and reports it to the DataFlux platform.

Added Data Gap Anomaly Detection

DataFlux added data gap anomaly detection, which triggers events based on configuring a custom detection cycle. The generated data gap events are uniformly stored in events.

After the data gap detection rule is configured and生效, the first time data gap is detected and the data gap persists, no data gap alert event is generated. If data is detected and within the configured custom detection cycle, data reporting has a gap, then a data gap alert event is generated.

Added Elasticsearch, Redis, and Docker Built-in Detection Libraries

DataFlux currently supports 4 built-in detection libraries: Elasticsearch, Redis, Host, and Docker monitoring. You can use them out of the box without manual configuration. If you need to learn more details, you can directly click on the corresponding built-in detection library.

  • Redis detection library: Includes abnormal increase in the number of client connections waiting for blocking commands in Redis.
  • Host monitoring: Includes various data indicators such as host CPU, memory, disk, etc.
  • Elasticsearch detection library: Includes ES JVM heap memory, thread pool, search query load anomalies, and other indicators.
  • Docker monitoring: Includes various data indicators such as container CPU, memory, etc.

Added APM "All Trace Explorer"

DataFlux provides three link filtering view lists: "All Spans", "Top-level Spans", and "Traceid". A span represents a logical unit of work in a given time period within a distributed system. Multiple spans form a link trajectory (trace).

  • All Spans: Displays all Span data collected within the selected time range.
  • Service Top-level Spans: Filters and displays all Span data that首次进入 within the selected time range.
  • All Trace: Filters and displays all trace data that contains the initial top-level span within the selected time range.

Added Link Details View Topology Call Relationship Diagram

DataFlux added a corresponding service topology call relationship diagram in the APM details page to view the calling relationships between various services.

Added User Access Error Details Page

DataFlux supports viewing user access error details on the user access details page. You can view the error data information, error type, and error occurrence time that appeared during this user access.

Click the error information to jump to the details page of the corresponding error.

Added Security Check Overview Statistics

In "Security Check" - "Overview", you can view the overview of security check events occurring on different hosts by filtering hosts, security check level, and security check category. This includes the number of security check events of different levels and visual chart analysis, as well as rankings of security check events by different categories and rules.

Added Security Check Event Handling Suggestions

In the Security Check Explorer, click the security check event you want to view. In the划出 details page, you can view handling suggestions for this security check event, including the theoretical basis of the security check event occurrence, risk items, audit methods, remedial measures, etc.

Added Secondary Navigation Bar

DataFlux added support for displaying secondary navigation through "mouse hover", thereby shortening the time users spend finding target data.

Added Member Management Search Function

DataFlux supports quickly matching current workspace members based on email or name through the search function in "Member Management".

Added Binding Built-in Views to Infrastructure Objects

DataFlux supports current space administrators binding built-in views (system views, user views) to the details pages of infrastructure objects. As shown in the figure below, the built-in system view "Disk Monitoring View" is bound.

Optimized Event List Display by Detection Library Type

DataFlux supports displaying corresponding event information based on different detection library types in the event list.

  • When the detection library rule type is threshold, mutation, interval, level, APM indicator, or user access indicator detection, it displays the corresponding baseline and its information based on the configured trigger conditions, including critical, error, warning.
  • When the detection library rule type is log, security check, abnormal process, or cloud test detection, it queries all log data统计 by the abnormal event.

As shown in the figure below, there are three different detection library types, including threshold detection, security check detection, and process data detection.

Optimized Measurement Deletion

DataFlux supports administrators deleting measurements within the space. Go to "Management" - "Basic Settings", click "Delete Measurement", then enter a keyword query and select the measurement name. Click "OK" to enter the deletion queue and wait for deletion. When deleting a measurement, system notification events will be generated, such as the user created a delete measurement task, the delete measurement task executed successfully, the delete measurement task executed failed, etc.

Optimized Explorer Built-in View Query Time Range

Changed the built-in view query time range, including the "Host Monitoring" view on infrastructure object, link, and log details pages, and the "Process" view on link and log details pages.

  • When viewing the details page of links or logs, "Host Monitoring" by default displays the performance data of the associated host within the selected time component range.
  • When viewing the details page of infrastructure objects (hosts, containers), "Host Monitoring" by default displays the host's performance data within the last 24 hours.
  • When viewing the details page of links, "Process" by default displays the real-time process data within 10 minutes before and after the link occurrence time point.
  • When viewing the details page of logs, "Process" by default displays the real-time process data within 10 minutes before and after the log generation time point.

Optimized Explorer Quick Filter Interaction

If the value corresponding to the current field in the quick filter of each Explorer is too long, the excess part is displayed with ···. When the mouse moves to the current field value area, it expands to the right for display based on the field value length.

July 1, 2021

DataFlux Studio

Added Unrecovered Event Statistics Explorer

DataFlux added an unrecovered event statistics Explorer. In the Events of the DataFlux workspace, by switching the Explorer in the upper left corner, you can view all unrecovered events that are continuously triggered within the space, as well as the data volume statistics and alert information details of unrecovered events under different alert levels.

DataFlux added a chart link function for time series charts, bar charts, overview charts, dashboards, pie charts, top lists, scatter plots, bubble charts, maps, and honeycomb charts. Chart links can help you jump from the current chart to the target page. Supports modifying the corresponding variable values in the link through template variables to transmit data information, completing data linkage.

Chart links support obtaining template variables through queries. Supports adding internal platform links and external links. Supports three opening methods: new page, current page, and slide-out details page. Supports adding alias display for links.

After the link is added, click the chart to view the clickable "Custom Link".

DataFlux supports adding built-in views, including built-in views and linkable web addresses, in individual service details. In the service list, click "Settings". You can add one or more jump link addresses or built-in views and custom names.

After adding a built-in view, you can view the corresponding built-in page by viewing the service details.

Added User Access Monitoring Scenario PV/UV Overview Statistics

DataFlux added PV/UV overview statistics for User Access Monitoring Web, iOS, Android, and mini-program scenarios. It can quickly compare the difference between PV and UV to understand the actual number of visiting users.

Added Notification Object: Lark Robot

DataFlux supports users managing alert notification objects sent. Currently supports adding email, DingTalk robots, WeCom robots, Lark robots, and Webhook custom.

Optimized Infrastructure Resource Catalog

Object data other than hosts, containers, and processes in Infrastructure can be custom reported to "Infrastructure" - "Custom" for viewing and analysis through DataFlux Function. Supports binding built-in views to resource catalog data, manually adding tags for filtering.

Resource catalog data reporting requires first installing and connecting DataKit and Function, then reporting data to DataKit through Function, and finally DataKit reports data to the DataFlux workspace.

Optimized Anomaly Detection Library Detection Rules

DataFlux added recovered alert configuration for anomaly events. Added filter condition support for regular expression matching style adjustment, optimized =~ to match, added not match. Added trigger condition support for non-numeric data matching, added match, not match regular expression matching, added between.

Optimized Explorer Detail Page Associated Logs

DataFlux supports filtering associated logs based on trace_id by default on the associated logs page in the link slide-out details page, to view log details associated under the same trace_id condition.

Within the link details, click "Logs" below the flame chart. You can view associated logs based on the host associated with the current link and the link ID (trace_id field). Meanwhile, you can perform keyword search and multi-tag filtering on the logs. If you need to view more detailed log content, you can click the log content to jump to the log details page, or click the jump button to open it in the log page.

Added associated logs page for the Explorer.

Optimized Explorer Search Style

Adjusted the search styles for new scenarios, metrics, APM, User Access Monitoring, anomaly detection library, host mute management, integration, management/built-in views, snapshots, etc.

Deployment Edition Supports Modifying Data Retention Duration

Added the ability to change data storage policies in the basic information of workspace management.

June 17, 2021

DataFlux Studio

Added Log Analysis Mode

DataFlux supports grouping庞大的原始数据 based on 1-3 tags through the log grouping function to reflect the distribution characteristics and trends of grouped data over time. DataFlux supports three ways to view grouped data: line chart, area chart, and bar chart.

Added Log Filtering

Added log filtering function (log blacklist), supports setting filtering rules to filter out logs that meet the conditions. That is, after configuring log filtering, log data that meets the conditions will no longer be reported to the DataFlux workspace, helping users save log data storage costs.

In "Logs" - "Filter Rules" - "New Rule", select "Log Source", add one or more data filter conditions with the relationship and, to filter the collected log data. Log data that meets the filter conditions will not be reported to the workbench.

DataFlux supports creating quickly accessible data copies for the metric Explorer, uniformly saved in the "Snapshots" directory. Added generate snapshot sharing link, supports sharing the snapshot link with other members within the space, can quickly reproduce the instantly copied data copy information, restoring the data to a certain time point and a certain data display logic.

Added Container Pod Explorer

DataFlux added a Pod Explorer in "Infrastructure" - "Containers". Switch to the "Pod" Explorer via the filter in the upper left to view detailed information of all collected Pods, including Pod name, the Node it belongs to, one or more containers it contains, startup status, running status, and associated indicators. You can switch to the honeycomb chart for viewing.

Click the Pod in the honeycomb chart to view the details of that Pod, including Pod name, namespace, visual monitoring charts, etc.

Added Free Quota Early Warning Notification

Added Free Plan usage quota early warning notification and quota exhaustion notification function, supports sending emails and generating system notifications. Learn more about message notifications. System message notifications can be viewed in the DataFlux workspace "Account" - "Notifications".

Added Chart Sharing

DataFlux supports chart sharing, which can be used to insert charts into code on platforms other than DataFlux for visual data display and analysis. Shared charts will change according to the changes of the charts in DataFlux. Charts shared in DataFlux scenario views are uniformly stored in "Integration" - "Embedded Sharing".

Optimized Chart Query and Chart Style and Settings

Adjusted and optimized the visual chart query, style, and settings layout in scenario views. Added functions such as chart title hiding, time slice settings, overview chart data precision settings, top list baseline settings, map/honeycomb chart color scheme filling settings, etc.

Optimized Integration, Added DataKit and Function Installation Guidance, Added Embedded Sharing

DataFlux supports viewing all supported collectors, DataKit installation instructions, DataKit upgrade commands, Function installation instructions, and embedded sharing chart code in Integration.

In the DataFlux workspace, click the left "Integration" menu to view all currently supported collectors. Click a collector to view the configuration details of that collector.

In the DataFlux workspace, click "Integration" - "DataKit" to obtain DataKit installation instructions, including for Linux, Windows, MacOS三大操作系统 and offline installation methods. If DataKit is already installed but the version is not the latest, you can obtain the update script through "DataKit Upgrade" here.

In the DataFlux workspace, click "Integration" - "Function" to obtain Function installation instructions.

Optimized Host Details Page

In "Infrastructure" - "Hosts", click the host name in the list to划出 the details page and view the host's basic information, processor, memory, network, hard disk, etc. Supports automatic conversion of data units, supports displaying different host monitoring views based on different operating systems.

June 3, 2021

DataFlux Studio

Added Browser Log Collection

DataFlux supports actively sending log data of different levels (log data corresponding to the source:browser_log metric type) from web browsers or javascript clients to DataFlux.

Added User Access Monitoring Session Explorer

DataFlux User Access Monitoring Explorer added viewing user access behavior monitoring data (session data) and session data details related to Web/mini-program/Android/iOS applications. Open "User Access Monitoring" - "Select any application" - "Explorer" in the workspace, you can switch to the Session Explorer. The Session Explorer statistics the entire session's duration, number of pages, number of operations, number of errors, etc. Clicking a single session allows viewing the detailed records of the entire session. By viewing and analyzing session data, you can understand user real access data situations from multiple dimensions, helping to improve application performance experience.

Added Browser Test, Cloud Test Anomaly Detection Templates and Optimized Self-built Nodes

  1. Browser Test

DataFlux added browser testing. In DataFlux "Cloud Test", click "New" - "Browser Test" to create a new Browser test task, obtaining user access experience data of Web pages in real-time, including page loading time, resource loading time, etc.

  1. Anomaly Detection Template

"Cloud Test Data Detection" is used to monitor cloud test data within the workspace. By setting a threshold (boundary) range for the specified data volume generated by test tasks within a certain period, an alert is triggered when the data volume reaches the threshold range. Meanwhile, you can custom define alert levels. When the specified data volume reaches different threshold ranges, alerts of different levels are triggered.

  1. Self-built Nodes

DataFlux cloud test self-built nodes added "Node Code", used to obtain the Code code for node information. Node Code does not support duplication within the current space.

Added Registration Welcome Page

After registering and logging into a DataFlux account, a welcome page is prompted. You can watch a short introduction video about DataFlux and scan the QR code to join the DataFlux WeChat and DingTalk service groups.

Added Free Plan Support for DataFlux Func, Adjusted Billing Items

All editions of DataFlux fully support the DataFlux Func platform custom functions. DataFlux Func is a Python-based类ServerLess script development, management, and execution platform. Meanwhile, to reduce the cost of using DataFlux for users, DataFlux adjusted the billing logic for APM and User Access Monitoring. After adjustment, APM statistics trace count, and User Access Monitoring statistics daily session count.

Optimized Generate Metrics

Added User Performance Monitoring, User Access Monitoring, Security Check generate metrics. Optimized log generate metrics. Generate metrics can be configured through three steps: data filtering, data query, generate metrics. This includes configuring dimensions, aggregation time, defining aggregation frequency, measurement, and metric name to generate metrics. Examples are as follows:

  1. APM

  1. User Access Monitoring

  1. Security Check

Optimized Built-in Views, Added Clone Function

Built-in views are divided into "System Views" and "User Views". Supports cloning from "System Views" to "User Views", and allows system views and user views to have the same name. Click "View" on the system view to jump to the system view details.

Click "Clone", open the clone view pop-up window, click OK to create the corresponding user view based on the current view template.

After creation is successful, it jumps to the user view page created above.

If a user view and a system view have the same name, and the system view is applied in the Explorer detail view, such as the host view under "Infrastructure - Hosts - Host Details", the user view will replace the system view in the host view display.

Optimized View Variables

DataFlux supports adding global variables to views. When you want to dynamically complete chart filtering in a view, you can choose to use view variables to achieve this. After the view variable is configured, you can choose whether to enable "Match All by Default" through "Advanced Settings".

After enabling, you can view the comprehensive view of all hosts through *, and you can manually select variables to filter and view.

Optimized Explorer

  • When single-column information display is incomplete, mouse hover delays for 1s before displaying the full prompt information.
  • When searching for key content in the Explorer, the matched content in the list is highlighted.
  • Details page tags support copy function. Host tags support [View Related Logs], [View Related Containers], [View Related Processes], [View Related Links], [View Related Checks].
  • Statistical charts for Logs, APM, User Access Monitoring, Cloud Test, and Security Check Explorers changed to status statistics.
  • Added help links for enabling User Performance Monitoring, User Access Monitoring, and Security Check.

May 20, 2021

DataFlux Studio

Added Save Snapshot

DataFlux snapshots support creating snapshots for Infrastructure, Logs, Events, APM, User Access Monitoring, Cloud Test, and Security Check Explorers. Through snapshots, you can quickly view and analyze data and anomalies from different time periods and different tags.

Added Infrastructure Host Integration Interaction

The "Integration Operation Status" of infrastructure hosts shows the operation status of collectors related to the host. There are two statuses: integrations in normal operation and integrations with errors. Integrations in normal operation are displayed as 'light blue' by default. Integrations with errors are displayed as 'red' by default and support clicking to view error information.

Meanwhile, by clicking integrations with built-in view symbols (as shown below), it supports "View Monitoring View". Click to view the related built-in view.

Added Self-built Nodes, Optimized Cloud Test Node Display

"Cloud Test" supports custom cloud test data collection through the DataFlux Studio console. By creating new detection points, you can periodically monitor websites, domains, backend interfaces, etc., based on the HTTP protocol, analyzing site quality situations anytime, anywhere. DataFlux has covered 16 nodes globally, classified and selected according to domestic, overseas, and self-built nodes. Meanwhile, it supports self-building test nodes, fully ensuring service availability.

Added Process, APM, and User Access Monitoring Check Rules

"Process Anomaly Detection" is used to monitor process data within the workspace. Supports setting trigger alerts for one or more field types of process data. For example: You can set an alert based on the number of times the 'host' field in the process data is 'izaqbin' and the 'state' field is 'sleep'.

"User Access Indicator Detection" is used to monitor indicator data of "User Access Monitoring" within the workspace. By setting a threshold range, an alert is triggered when the indicator reaches the threshold. Supports setting alerts for a single indicator and custom alert levels.

"APM Indicator Detection" is used to monitor indicator data of "APM" within the workspace. By setting a threshold range, an alert is triggered when the indicator reaches the threshold. Supports setting alerts for a single indicator and custom alert levels.

Optimized Integration

Log in to the DataFlux workspace, enter the "Integration" page to view all supported collectors. Meanwhile, you can click "Quickly get DataKit installation command" in the upper right corner to directly obtain DataKit installation instructions for installation. After installation, you can enable more collectors to collect data indicators.

Optimized Log Generate Metrics

Log generate metrics added selection of log source. You can generate metrics by selecting log source, filtering logs, configuring dimensions, aggregation frequency, defining measurement, metric name, and aggregation rules. [Learn more about log generate metrics]

Optimized Log Explorer

After log data is collected by DataFlux, you can view all log sources and their content in the DataFlux Studio console "Logs". Supports selecting the log details you want to view by searching for the log source.

Optimized User Access Monitoring Explorer

On the user access monitoring details page "Performance", you can view front-end page performance, including page loading time, content drawing time, interaction time, input delay, etc. Taking the figure below as an example, the LCP (Largest Contentful Paint) indicator reached 5.43 seconds, while the recommended time is within 2.5 seconds, indicating that the page loading speed is slow and needs optimization. Supports filtering and searching, helping users quickly locate resources and content.

When switching to "Links", you can view requests sent to the backend application, including the occurrence time, content, and duration of the resource call. For example: You can view each ajax request returned through the xhr object.

When switching to "Errors", you can quickly locate front-end errors through the detailed information of the error occurrence, including error information, error type, and occurrence time.

Optimized Built-in Views

Built-in views are divided into system views and user views两种类型, which can be applied in scenarios.

System views are view templates provided by the system that have been integrated. They support modification, export, and restore. That is, users can directly "Export" system views or "Modify" system views. By "Restoring" a system view, the modified system view can be converted back to the original system view. It cannot be recovered after restoration.

User views are views saved as templates after user custom views. They support modification, export, and deletion. In "Built-in Views" - "User Views", you can create user custom view templates by clicking "New User View". For specific configuration, refer to Scenarios and Views.

Optimized Chart Lock Time Optimization

Dashboard views support locking the time range of charts,不受全局时间组件的限制. After successful setting, the user-set time will appear in the upper right corner of the chart, such as "xx minutes", "xx hours", "xx days".

Optimized Nested Functions

All chart component nested functions are optimized to built-in functions, no longer calling DataFlux Func functions, implemented through DQL. Removed the time interval option for nested functions.

May 7, 2021

DataFlux Studio

Added Log Backup

DataFlux basic logs are stored for up to 60 days at most. If longer storage and viewing are needed, basic logs need to be backed up. DataFlux supports storing backup logs for up to 720 days.

Enter the "Logs" - "Backup Logs" page. The data is empty by default. Select a time range to view the corresponding backup logs. DataFlux supports querying and analyzing logs by selecting time ranges, searching for keywords, filtering, etc. Click a log to view the details of that log, including the time the log was generated and its content.

Added Security Check

DataFlux supports you in timely monitoring, querying, and associating all inspection events through "Security Check". While及时发现漏洞, anomalies, and risks, it helps you improve inspection quality, problem analysis, and problem handling capabilities.

Before starting security checks, you need to install [DataKit] and [security-checker] first. After configuration, you can enter "Security Check" and query and analyze by selecting time ranges, searching for keywords, filtering, etc.

Added Beginner's Guide

When registering and logging into DataFlux for the first time, you can start using DataFlux through the beginner's guide. Following the guide steps, users can complete the installation of datakit, the collection of nginx data on their own, and view the corresponding data in the logs. After closing the beginner's guide, you can reopen it by clicking the user name in the workspace - "Beginner's Guide".

Added Bar Chart Grouping Stacked Display

Chart Stacked Bar Chart

Bar charts support [Metric Stack] and [Group Stack], which are off by default. After enabling, [Metric Stack] is selected by default. A "Method" option appears on the right, including default and percentage.

  • Metric Stack: Stacks and compares the tagged data of groupings维度 with metrics.
  • Group Stack: Stacks and compares the metric data of groupings维度 with grouping tags.

Event Explorer Stacked Bar Chart

DataFlux supports viewing the statistics of events of different statuses occurring at different time points through bar chart stacking.

![](img/4.5.7_event_filter22- Added Tencent Cloud, AWS Cloud Host Details If the host is a cloud host and "Cloud Association" is configured, clicking the host name in the list will slide out to view the "Cloud Vendor Information" of the cloud host object, including cloud platform, instance name, instance ID, instance specification, region, availability zone, creation time, network type, payment type, IP address, etc.

Added Host Operation Status Integration

Host integration operation status has three states: integrations in normal operation, integrations in normal operation with built-in monitoring views, and integrations with errors: - Integrations in normal operation: Clicking has no reaction. - Integrations with built-in monitoring views: Clicking drops down "View Monitoring View". Click to view the corresponding monitoring view. - Integrations with errors: Clicking allows viewing the last error feedback message.

Added Correlation Analysis Built-in Pages

  • Infrastructure hosts added process and container built-in pages.
  • Logs added process and link built-in pages.
  • Links added host monitoring views and log built-in pages.

Optimized APM Topology Map

In the "APM" "Service" list, you can switch the list to topology map mode to view the calling relationships between various services. - Added selecting time range via timeline. - Added zoom in/out for topology map. A small thumbnail can be viewed in the lower left corner when zoomed in. - Added three node size dimensions. - Added highlighting of the corresponding service when selected, other services and connection lines turn gray.

Optimized Operation Events and Notification Events

  • Operation events moved to "Management" - "Basic Settings" - "Security Audit". Click "View" to see operation security events.
  • Notification events moved to the left menu "Notifications". Click "Notifications" to view all notification events.

Optimized Paid Plans and Bills Usage Statistics Views

  • Paid Plans and Bills usage statistics views added log backup statistics.

April 15, 2021

DataFlux Studio

Adopted a robot brain approach, not only possessing a strong technological sense but also highlighting DataFlux's powerful capabilities in massive data integration, processing, and analysis. Like a robot, it can be real-time, fast, and infinitely scalable, providing easy-to-use, flexible, and diverse visual charts for data analysis and business operation analysis, improving operational efficiency.

Added Cloud Testing

DataFlux provides users with an out-of-the-box cloud testing solution, supporting HTTP protocol testing. Utilizing a global monitoring network, it tracks webpage and application performance end-to-end, records all testing real-time feedback information, and provides performance test results such as DNS, SSL, TTFB, helping you discover problems in advance, locate issues, and improve user experience. DataFlux cloud testing supports analyzing the response time and availability rate of current testing tasks from both geographical and trend dimensions.

Added Container Object Distribution Map

[Container Objects] added honeycomb distribution map. On the container list page, click the icon on the left side of the list to switch to the container distribution map display. Through the distribution map, you can visually query container distribution information. The distribution map supports the following functions: - Search: Enter search keywords. - Filter: Filter tags and select filtering objects, multiple selections supported. - Group: Display objects by group, multiple selections supported. - Fill: Select the metric to fill. The size of the metric value will determine the fill color of the legend. - Fill Rules: Fill the legend color according to the results of the fill metric. Divide the maximum and minimum values of the metric into five equal parts to get 5 intervals, corresponding to five colors respectively.

Added Alibaba Cloud Cloud Host Details

If the host is a cloud host and "Cloud Association" is configured, clicking the host name in the list will slide out to view the "Cloud Vendor Information" of the cloud host object, including cloud platform, instance name, instance ID, instance specification, region, availability zone, creation time, network type, payment type, IP address, etc.

Added Table Chart Metric Sorting

Table charts support metric sorting, sorted by the first query metric by default. Click the table header to switch ascending/descending order, and the Top/Bottom in the corresponding query is adjusted synchronously. Click other query metrics to sort, and the Top/Bottom in the corresponding query is adjusted synchronously.

  • The default progress bar minimum and maximum values are the minimum and maximum duration in the [Link] data list.
  • Supports dragging the progress bar to adjust the max/min values, and the values in the input box change synchronously.
  • Supports manually entering max/min values, press "Enter" or "click outside the input box" to perform filtering search.
  • The input box turns red when the input is irregular, and no search is performed. Correct format: pure "number" or "number + ns/μs/ms/s/min".
  • If no unit is entered for search, "s" is directly appended to the input number for filtering search.
  • If the unit is manually entered, the search is performed directly.

Added Mail Group

Added mail group notification objects in [Notification Object] management. A mail group can add multiple members at the same time. After adding, select this mail group in the "Alert Settings" of the anomaly detection library, and alert emails will be sent to the added members' mailboxes simultaneously.

Optimized Anomaly Detection Library

DataFlux has built-in多种 detection libraries, ready to use out of the box. Supports host and other built-in detection libraries. After enabling, you can receive related anomaly event alerts. "Import/Export Detection Library" is supported in the "Detection Library". After configuring all detection rules, the detection library can be exported as a json file. The exported detection library can be imported into other detection libraries. When importing a detection library, it will verify if there are duplicate named detection items, and you can choose "Replace" or "Skip".

Optimized Object Explorer and Details

  • Host objects added operating system small icon display.
  • Container objects added container type small icon display.
  • Process slide-out details page added "Start Time" display.

Optimized Data Storage Policy, Paid Plans and Bills

  • Workspace management basic information added change data storage policy.
  • Workspace management usage statistics optimized to Paid Plans and Bills.

April 7, 2021

DataFlux Studio

Added View Variable Object Mapping

When using "Object Mapping", you must first define a view variable based on an object class field. After configuring object mapping in "View Variables", you need to use the mapped tag field as the grouping in "Chart Query", and enable "Field Mapping" in "Chart Settings". Then you can view the set object mapping display in the view chart.

Optimized Event Grouping

DataFlux supports grouping and aggregating related events for "Anomaly Events" by "Detection Item". Click to view "Aggregated Events" to quickly view the associated event list, anomaly event status distribution on different hosts, and metric monitoring, etc. Anomaly event status distribution includes critical, error, warning, ok, and mute, corresponding to five status colors. The background color is green, filled with the corresponding status color backward from the event occurrence time point. The fill range is the detection frequency. When the group contains host and host mute is set, the mute time range is displayed in gray.

Optimized Detection Item Query

"Detection Metric" in "Anomaly Detection Library" "Detection Rules" only allows adding one query. Simple Query is added by default. Supports switching between Simple Query and "Expression Query". Unified the "Trigger Condition" M1 in "Detection Rules" to Result. Supports adding AS alias for queries.

DataKit (v1.1.4-rc2)

  • Fixed an issue where the Alibaba Cloud monitoring data collector (aliyuncms)频繁采集 caused some other collectors to freeze.

March 25, 2021

DataFlux Studio

Added Cloud Service Association

DataFlux supports unified and centralized management of cloud service account AccessKey information through "Cloud Association". After configuration, it can regularly synchronize cloud host information under the belonging cloud account in [Infrastructure - Hosts], displaying all cloud host information, such as CPU, memory, network, disk, etc. If association is no longer needed, the associated Alibaba Cloud RAM sub-account AK information can be deleted. After the association information is deleted, an association invalidation notification event will be triggered, which can be viewed in Events - Notification Events. More details about cloud association can be found in the document Cloud Association.

Added Operation Event Audit

DataFlux supports security audit of workspace operation events. Operation events are uniformly stored in Events - "Operation Events".

Added Host Mute

If you need to perform operations like stress testing on a single cloud host, it will trigger frequent alerts. For such alert notifications, you can temporarily enable the host mute function. In "Infrastructure" - "Hosts", click the cloud host details to select the host mute time. All muted host lists can be viewed in "Anomaly Detection Library" - "Host Mute Management". After deleting a muted host, you can receive alert notifications for the corresponding host again.

Added Notification Object: WeCom Robot

DataFlux supports users managing alert notification objects sent. Currently supports adding email, DingTalk robots, WeCom robots, and Webhook custom.

Optimized Event Explorer, Added Event Grouping Aggregation View

In the event list explorer, through the grouping function, DataFlux supports quickly aggregating and counting related events based on grouping tags. For example, in "Anomaly Events", based on the "Detection Item" grouping, you can quickly obtain all events triggered based on this detection item. By switching the aggregated event list, you can view the details of these events.

In the "APM" "Service" list, supports switching the list to topology map mode to view the calling relationships between various services. Supports filtering display through different performance metrics, and allows customizing the color range of link service performance metrics.

Other Optimized Features

  • Optimized Logs, Events, Infrastructure, APM, User Access Detection explorer search and export, added export to view. Optimized Logs, Events, Links, Infrastructure - Containers, Infrastructure - Processes details page tag filtering, associated query analysis.
  • Optimized time component, optimized single query charts (overview chart, dashboard, top list) simple query and expression query switching under scenario views.
  • Optimized host offline display: When the host is in offline status, CPU usage, memory usage, and load are not displayed, values are filled with "-".
  • Optimized anomaly detection library detection rules, added level detection, interval detection, and mutation detection.

DataKit (v1.1.4-rc1)

  • Process collector message field added more information for full-text search.
  • Host object collector supports custom tags for cloud attribute synchronization.

DataKit (v1.1.4-rc0)

  • Added file collector, dial test collector, and HTTP message collector.
  • Built-in support for ActiveMQ/Kafka/RabbitMQ/gin (Gin HTTP access log)/Zap (third-party log framework) log cutting.
  • Enriched [http://localhost:9529/stats] page statistics information, added metrics such as collection frequency (n/min), data volume per collection, etc.
  • DataKit itself added a certain cache space (invalid after restart) to avoid data loss due to occasional network reasons.
  • Improved Pipeline date conversion function, enhanced accuracy. Also added more Pipeline functions (parse_duration()/parse_date()).
  • Trace data added more business fields (project/env/version/http_method/http_status_code).
  • Various detail improvements for other collectors.

March 4, 2021

DataFlux Studio

1. Optimized Time Series Chart, Bar Chart, Top List, Log Flow Chart, Alert Statistics Chart, Object List Chart

  • Time Series Chart: Added "Display Density", including "Lower: 60 points", "Low: 180 points", "Medium: 360 points", "High: 720 points". Added "Stacked Display", only supported for bar charts, off by default.
  • Top List: Chart display optimized.
  • Bar Chart: Chart display optimized.
  • Log Flow Chart: Added search box, supports searching message content.
  • Alert Statistics Chart: Query display optimized.
  • Object List Chart: Query display optimized.

2. Added Automatic Unit Conversion

  • Default query result values are automatically converted and displayed following "Scientific notation K, M, B", retaining two decimal places (1 thousand = 1K, 1 million = 1M, 1 billion = 1B).
  • RMB is automatically converted to units "Yuan, Ten Thousand, Hundred Million", retaining two decimal places.
  • After setting the unit, scientific notation is canceled, and displayed according to the user-set unit.

3. Added Object Management

DataFlux supports users managing alert notification objects sent. Currently supports adding DingTalk robot notification configuration and Webhook custom notification configuration两种方式.

  • Events DataFlux supports collecting all event data generated based on the "Anomaly Detection Library". Users can view and search all event data within the workspace on the Events page.

  • Object Hosts DataFlux supports collecting object host data. In the "Objects" host list, you can view and search all host object data information. Click the host name in the list to slide out the details page and view the detailed information of the host object, including basic information, CPU, memory, network, hard disk, etc.

  • Object Containers DataFlux supports collecting object container data. In the "Objects" container list, you can view and search all container object data information. Click the container name in the list to slide out the details page and view the detailed information of the container object.

  • Object Processes DataFlux supports collecting object process data. In the "Objects" process list, you can view and search all process object data information. Click the process name in the list to slide out the details page and view the detailed information of the process object.

  • Object Others DataFlux supports collecting object other data, such as Alibaba Cloud ECS, etc. In the "Objects" other list, you can view and search all other object data information. Click the other object name in the list to slide out the details page and view the detailed information of the other object.

  • Logs DataFlux supports collecting various formats of log data, including system logs, Nginx logs, Apache logs, and other log data. It can quickly view and analyze all log data in the current workspace.

  • Link Tracing DataFlux supports viewing link services and link lists in "Link Tracing". In the service list, you can view or search all services receiving link data. In the link list, you can query link data, export link data, view link details, and perform full analysis of link performance through flame charts, span lists, etc. DataFlux also supports both light and dark theme displays. Here, dark theme is used as an example. Service List Link Topology Link List Flame Chart

  • Application Monitoring DataFlux supports collecting Web, Android, iOS, and mini-program application data, and provides application monitoring scenarios to quickly view and analyze application status. Taking Web application as an example: Web Application Explorer Web Application Details

February 2, 2021

DataFlux Studio

  • Added time series chart style area chart.
  • Added object hosts, containers, processes, added host honeycomb chart.
  • Optimized anomaly detection, added built-in detection library.
  • Optimized events, objects, logs, links, application explorers and page functions.
  • Optimized log flow chart query.
  • Optimized top list, dashboard, overview chart query, added expression query mode.
  • Optimized visual chart query, simple query and DQL query support mutual switching.
  • Optimized grouping logic, including scatter plots, bubble charts, maps, honeycomb charts, table charts.
  • Optimized filter conditions support real-time filtering.

DataKit

  • Added host object data collection.
  • Added container object data collection.
  • Added process object data collection.

January 14, 2021

DataFlux Studio

  • Added Explorer layout, optimized data structure, supports object/link/RUM/log and other module query/analysis.
  • Added mini-program application monitoring.
  • Added time series chart, chart type can choose line chart or bar chart.
  • Adjusted chart UI.
  • Adjusted log flow chart query and settings.
  • Optimized bar chart, top list, table chart related settings.
  • Threshold detection adjusted to DQL UI query mode.

Feedback

Is this page helpful? ×