Release Notes (2021)¶
This document records the update content descriptions for each online release of Guance, including DataKit, Guance best practices, Guance integration documentation, and Guance.
December 30, 2021¶
Optimized Binding to Built-in Views¶
“Guance” supports binding built-in views with different trace services, applications, log sources, projects, etc., by associating fields. According to the bound fields, you can view the associated built-in views (system views, user views) on the corresponding Explorer detail pages. Supported Explorers include scenario custom Explorers, infrastructure, logs, traces, user access, security checks, availability, etc.
Optimized Log Explorer¶
- In the Log Explorer, the left-side source list is expanded by default upon first login and can be manually collapsed; the Explorer will remember the last state by default.
- In the Log Explorer, clicking the settings button on the left allows you to manually add filter fields for quick filtering.
- In the log data list, you can use "hover" over the log content to expand and view the entire log content. Clicking the "copy" button copies the entire log content to the clipboard. When expanded, if possible, the system formats the log in JSON format; otherwise, it displays the log content normally.
Optimized Associated Display on Explorer Detail Pages¶
The associated display on the detail pages of the Log, Infrastructure, Trace, Security Check Explorers has been optimized. This includes related queries for HOSTs, Metrics, Traces, CONTAINERS, Pods, etc., supporting two types of associated displays: fixed display and non-fixed display (displayed based on whether the associated field is included).
Taking the Log Explorer detail page's association with HOSTs as an example, the associated fields are “host”. To view related hosts in the log details, the field “host” needs to match; otherwise, it won’t be possible to see related host pages in the log details. After matching the fields, under the associated hosts, you can view basic information about the host and its metric performance status.
-
Attribute View: Includes basic information about the host, integration running conditions, and if cloud host collection is enabled, you can also view cloud vendor information.
-
Metric View: You can view the performance metric views of the related host within the default 15 minutes. Click 「Open this View」 to go to the built-in view, where you can customize and modify the host view through cloning and save it as a user view, which can then be viewed by binding it to the log details page.
Optimized Snapshot Sharing¶
Snapshot sharing supports hiding the top bar on the share page. In the snapshot list, click the share button, and you can set advanced settings like "Hide Top Bar" in the pop-up dialog box.
Optimized Chart Query Result Showing Thousands Separator¶
Guance dashboard chart query results support automatically adding a thousands separator format. If a unit is set, the data format will follow the set unit, and after setting, you can preview the result in real-time.
Other Optimized Features¶
- Improved performance of host network topology queries, added target domain display in network topology nodes. That is, if the target host is not in the current workspace but the target domain exists and the port number of the target domain is less than 10,000, the target domain will be displayed in the topology map.
- Improved logic for pinning favorite dashboards and notes:
- Unfavorited dashboards are sorted in reverse chronological order based on creation time
- Favorited dashboards are sorted in reverse chronological order based on favoriting time
- Favorited dashboards are displayed above unfavorited dashboards
- Improved cascading query logic for view variables, i.e., when querying two linked variables, if the previous selection is
*
, the linked variable does not filter, showing all value options. - Added help documentation links to the collapsed navigation bar on the left.
December 16, 2021¶
Added Similarity Metric Queries for Time Series Graphs¶
In the analysis mode of time series graphs, you can select "chart queries" as trend lines/columns for metric queries and "view similar trend metrics". By selecting the absolute time range via boxing, you can query similar metric trends within the space.
Added Associated Data Statistics Display on Explorer Detail Pages¶
Added statistics display for related logs, traces, containers, etc., on Explorer detail pages. Supports directly obtaining statistics for related data in the tab title of the Explorer detail page, including infrastructure, logs, application performance, user access, security checks, etc.
Added Export Function for Explorer Charts¶
Guance supports exporting any chart from the "Guance" space Explorer to dashboards, notes, or clipboard.
Added Match Search in Explorer¶
Guance supports wildcard searches in the Explorer search bar using formats like "host:cc*" for fields such as hostname. This can be used for filtering log data of hosts whose names start similarly. Additionally, a "delete" button has been added to the right side of the Explorer search bar for one-click text deletion.
Optimized Log Explorer and Related Data Queries¶
- Through the "Containers" section below the log details page in "Guance", you can view all container (CONTAINER) data related to the log host within the last ten minutes.
- Logs details page supports filtering related link and container data using "trace_id" or "host".
- The content display on the log details page supports two viewing modes: JSON and Text, automatically displaying formats based on message type.
- Through the similar-text viewer, you can view detailed logs associated with Pattern clustering.
- Removed color tooltips for hover actions in the Log Explorer.
Optimized Mute Policies, Supporting Mute Settings for Monitors¶
Guance supports users managing all mute rules in the current space through "Mute Management" in the "Monitoring" feature. Guance provides three types of mute: HOST mute, monitor mute, group mute, allowing mute management for different HOSTs, monitors, groups, ensuring that no alert notifications are sent to any notification targets during the mute period.
Optimized Monitor Management¶
Duplicate monitors in the monitor list can affect users' quick location of monitoring objects. "Guance" supports choosing whether to create duplicate monitors from templates. Additionally, users can use the new "Batch Management" tool to customize export/deletion of monitors.
Optimized Query Display for View Variables¶
- Default values have been added to dashboard view variables, allowing previews of all variables in the current view and selecting initial variables to view by default on the dashboard.
- Supports adjusting the order of view variables via drag-and-drop using mouse hover (hover).
- Removed the "Settings" button for view variables, enabling "Sorting," "Hiding," and "Deleting" functions directly in the view variable list.
Optimized Lock Time Setting for Dashboards and Notes Charts¶
Guance supports adding a "More" option in the lock time setting for charts on dashboards and notes to customize the locked time range. After this optimization, the functionality of the lock time component and chart component remains consistent.
Optimized Help Center Page Entry¶
In Guance Help Center, you can quickly view product documentation, updates, blogs, live reviews, and other information. You can also join the official service group of Guance via QR code scanning.
December 2, 2021¶
Added Owner Member Role¶
Guance added the owner role for the current workspace, currently supporting the definition of four workspace member permissions, including "Owner," "Administrator," "Standard Member," and "Read-only Member." These constrain the management permissions, configuration permissions, operation permissions, and viewing permissions for different member categories respectively. The "Owner" role has the highest operational permissions and can designate "Administrators" for the current space and perform arbitrary management, configuration, operations, and data browsing. In Guance workspace's 「Management」-「Member Management」-「Edit」, you can edit and update member permissions.
Added Custom Explorer for Scenarios¶
"Guance" offers an Explorer in scenarios that can be quickly set up based on the log data range, supporting collaborative building of customized Explorers for scenarios by workspace members, meeting personalized viewing needs. Completed "Explorers" can be exported and shared with others, sharing Explorer templates.
Added Exporting User Views to Dashboards¶
"Guance" supports exporting user-created views from 「Built-in Views」 to a JSON file in the workspace, which can be used for importing into different workspaces for scenarios or built-in views.
Optimized Host Network Observability, Added Network Flow Data Viewing List¶
"Guance" supports displaying different visualized charts based on selected protocols in the 「Network」 section of the 「Host」 details page in Infrastructure, and customizing the display fields for network flow data lists.
By clicking 「View Network Flow Data」, you can check the network flow data for the past two days by default, including time, source IP/port, destination IP/port, source host, transmission direction, protocol, bytes sent, etc. You can customize the display fields or add filtering conditions to filter all string-type keyword fields.
Added sourcemap Function¶
When releasing applications in a production environment, to prevent code leakage and other security issues, files are usually converted and compressed during the packaging process. While these measures ensure code security, they also cause collected error stack information to be obfuscated, making it difficult to directly locate problems and complicating subsequent bug troubleshooting.
To address the above issues, "Guance" provides sourcemap functionality for web applications, supporting the deobfuscation of code to facilitate easier error localization and helping users resolve issues faster.
Added Security Inspection Data Analysis for Host Associations¶
"Guance" supports viewing security inspection data related to the host within the last day and performing keyword searches, multi-tag filtering, and data sorting on these security inspection data.
Added DataKit Collector Kubernetes Installation Guide¶
"Guance" supports adding a guide for installing the DataKit collector in a Kubernetes cluster under the workspace 「Integration」-「DataKit」.
Optimized Associated Trace Queries for User Access Monitoring¶
"Guance" supports adding attributes to the current filter or copying them (copying the tag to the local clipboard) on the trace details page and user access performance details page (user access monitoring VIEW Explorer).
Optimized Registration and Upgrade Process¶
Guance added direct selection of opening the Guance billing platform during account registration and optimized the upgrade process to the agile version after logging into the workspace,
Price Reduction Adjustment¶
Guance adopted the latest technology to optimize data storage costs, hence adjusted pricing downward, introducing a tiered billing model based on data storage strategies, including log data, APM Trace, RUM PV for these three billing items.
Additionally, to help enterprises observe IT infrastructure and application systems more comprehensively at more competitive prices, Guance launched three package options based on different stages of enterprise development: Startup Acceleration Package, Entrepreneur Development Package, and Enterprise Standard Package, along with stackable traffic packages for enterprises to use according to their own needs and package stacking.
Other Optimized Features¶
- Added LCP, FID, CLS, FCP related detection indicators to user access monitoring. In Guance workspace 「Monitoring」-Monitor, click 「+Create New Monitor」, choose 「User Access Indicator Detection」, and you can enter the configuration page for detection rules.
- Changed grouping in simple monitor queries to detection dimensions, which determine the dimension triggering the detection rule, i.e., the trigger object. "Guance" supports adding multiple detection dimensions, and if any detection dimension's indicator meets the alarm condition, an alarm is triggered.
- Optimized navigation menu display, presenting secondary menus flush against the function navigation name when the Guance workspace navigation bar is collapsed.
November 18, 2021¶
Added SSO Login¶
Guance supports users creating identity providers within workspaces. During login, users input their company email to obtain the corresponding SSO login link, clicking the SSO login link completes the verification login. In Guance workspace 「Management」-「SSO Management」-「Create Identity Provider」, you can set SSO single sign-on for employees.
Added SLO Charts to Dashboards and Notes¶
Added SLO charts to scene dashboards and notes. SLO charts can directly choose configured monitored SLOs for SLO data display, synchronously displaying SLO data results by selecting different SLO names.
Added Custom Group Fields for Infrastructure Grouping¶
In Guance infrastructure Explorer, click the settings button to the right of the group, and you can add custom group tags. After completion, you can choose to aggregate and display data by the added group tags.
Note: Only administrators can add custom group tags.
Added SLO Notification Silence Configuration¶
If an event is not very urgent but has high frequency of alert notifications, you can reduce notification frequency by setting notification silence. After configuring notification silence, events will continue to occur, but notifications will no longer be sent. Generated events will be stored in event management.
Optimized Chart Links¶
Guance supports adding built-in links and custom links to charts. Through links, you can jump from the current chart to the target page and pass data information by modifying the corresponding variable values in the link through template variables, completing data linkage. Chart links support opening from a new page, current page, or slide-over details page, enabling linked analysis.
Price Reduction Adjustment¶
- Daily DataKit price reduced to 3 yuan per unit
- Availability API testing adjusted to 1 yuan per 10,000 tests
Other Optimized Features¶
- Launched Free New Version in China Region 1 (Alibaba Cloud): The free experience version is divided into Alibaba Cloud login and AWS login. For Alibaba Cloud login, the timeline data retention strategy is 30 days, while other data retention strategies are 7 days. For AWS login, all data retention strategies are 1 day;
- Optimized SLO scoring logic: Disabled monitors will not be counted in the scoring range;
- Added dashboard tagging feature: When creating/modifying dashboard views, you can add custom tags to dashboards to help with quick filtering;
- Optimized log highlighting display: When searching logs in the search bar, only the matched keywords remain highlighted in the returned list;
- Optimized Explorer search: Added help documentation links in the Explorer search dropdown;
- Optimized Markdown formatting for text charts: Added support for unordered and ordered lists, optimizing display formatting.
November 4, 2021¶
Added Scenario Notes¶
In scenarios, you can create multiple notes for summaries and reports. These support inserting real-time visual charts for data analysis, inserting text documents for explanations, combining charts and documents for data analysis and summary reporting; support exporting infrastructure, log, application performance, user access, etc., Explorer data to notes for data analysis; support sharing notes with all workspace members, retaining abnormal data analysis to help with tracing, locating, and solving problems.
Upgraded Scenario Views to Dashboards¶
Guance upgraded scenario views to dashboards and adjusted the functionality and layout of dashboards. In scenarios, you can create multiple dashboards to build data insight scenarios, support searching dashboards via keywords, support adding visual charts for data analysis, and support filtering and finding corresponding dashboards quickly from "My Favorites," "Import Projects," "My Creations," and "Frequently Browsed."
Added Export to Dashboard and Notes¶
Infrastructure, log monitoring, application performance monitoring, availability monitoring, security inspections, etc., added export to dashboard and notes.
Upgraded Anomaly Detection Library to Monitoring¶
The original 「Anomaly Detection Library」 was upgraded to 「Monitoring」, with changes in naming as follows:
- Monitors (originally referred to as "Anomaly Detection Rules") provide various monitoring methods such as "Threshold Detection" and "Log Detection," allowing users to customize detection rules and trigger conditions and receive alerts immediately.
- Templates (originally referred to as "Built-in Detection Libraries"), Guance includes many built-in, ready-to-use availability monitoring templates, supporting one-click creation of Docker, Elasticsearch, Host, Redis monitors. After successfully creating a template, the corresponding official monitors are automatically added to the current workspace.
- Groups (originally referred to as "Custom Detection Libraries"), the grouping function supports you to create meaningful combinations of monitors, facilitating group management of various monitors.
Added Monitoring SLO Functionality, Support Export to Dashboard
¶
「Monitoring」 added SLO monitoring, supporting quality scoring of service providers (Providers) in the cloud era, comparing the corresponding SLI (Service Level Indicator) to meet target needs. At the same time, "Guance" also supports exporting SLO as a view to the dashboard for synchronized SLO monitoring on the dashboard.
Other Optimized Features¶
- Chart Optimization:
- Removed group selection from the overview chart;
- Adjusted style selection for time series charts, pie charts, and bar charts;
- Added tooltip information for time slices;
- Supported editing of chart JSON, linked with queries/settings, and validated input JSON for errors with error prompts;
- Styled optimization in analysis mode for time series charts;
- Added reminders for paid plans and billing, selecting settlement methods aliyun or aws, and the billing list directly shows links to the corresponding cloud service console for viewing bills.
October 21, 2021¶
Added Host Network Distribution Map¶
A new host network distribution map is added, providing infrastructure network observability. In 「Infrastructure」-「Host」, click the small icon in the upper-left corner to switch to viewing the host network distribution situation. In 「Network Distribution Map」, you can visually query the network traffic between hosts in the current workspace, quickly analyzing TCP latency, TCP fluctuations, TCP retransmission counts, TCP connection counts, and TCP close counts between different hosts.
Added User Access Monitoring Tracking Function¶
Added a 「Tracking」 function for user access monitoring. Supports users initiating tracking tasks via 「User Access Monitoring」 to monitor custom trace trajectories in real time. By pre-setting trace trajectories, you can filter trace data centrally, precisely query user access experiences, and promptly discover vulnerabilities, anomalies, and risks.
Added Scene Chart Json Format Query¶
In 「Scene」, when editing charts, each correct chart query corresponds to a JSON text. It supports mutual parsing between JSON texts and chart queries within the workbench, helping you understand chart drawing details.
Optimized DCA Desktop Client Application¶
Optimized UI display for the DCA desktop client, increasing host status.
- online: Indicates normal data reporting, allowing you to view DataKit’s operating status and configure collectors via DCA;
- unknown: Indicates remote management configuration is not enabled or not within the same local network;
- offline: Indicates the host has not reported data for more than 10 minutes, or the host name has been changed, resulting in the original host name being displayed as offline. Hosts that fail to report data normally will be removed from the list if they do not report data for more than 24 hours.
Optimized Event Host Linkage Queries and Detection Rule Queries¶
Optimized the events module, supporting linked queries for associated hosts, allowing real-time monitoring of logs, containers, processes, traces, and inspection events related to the event within the selected time component range; supports one-click viewing of the associated detection rules and querying other triggered events under the corresponding detection rule.
Adjusted Billing Method¶
- Based on node login method, choose billing settlement methods, including CloudCare accounts, Aliyun, AWS. Among them, CloudCare account settlement is a universal method, Aliyun login nodes can choose Aliyun account settlement (in progress), and AWS login nodes can choose AWS account settlement (already online).
- Removed Session count and Browser dial test count billing dimensions, removing Session count and Browser dial test count statistics and views in 「Paid Plans and Billing」, and added a security inspection count statistics chart.
Other Optimized Features¶
- Optimized scene chart addition function, removed the 「Add Chart」 button in 「Edit」 mode, all chart types available for addition are selectable at the top of the scene;
- Optimized scene chart grouping function, supports deleting groups only and keeping existing charts entering the default group;
- Added a regional ranking switch for China map/World map, default off;
- Optimized Explorer query method, merged search and filter bars, supports keyword search, label field filtering, and associated searches based on labels, fields, and text.
September 28, 2021¶
Added Host Network Performance Monitoring¶
Host network performance monitoring supports viewing network traffic between host services, containers, and any tagged infrastructure. It supports viewing network traffic and data connections between source hosts and destinations based on IP/ports, with real-time visualization to help businesses understand the operational status of their business systems' networks, quickly analyze, track, and locate fault problems, preventing or avoiding business issues caused by network performance degradation or interruption.
After successful collection of host network data, it will be reported to the Guance console, and in the 「Network」 section of the 「Host」 details page under 「Infrastructure」, you can view all network performance monitoring data information within the workspace.
Added Linked Chart Queries¶
In the scene view, you can view the DQL query statement, associated links, etc., of the chart by a single click, or perform associated queries for related logs, containers, processes, traces, etc., under different grouped query tags. When the chart query contains host (host) information, you can view the related host monitoring view. For example, clicking any chart will allow you to view the DQL query statement, set links, and associated queries for related logs, containers, processes, traces, and host monitoring views.
Added Rectangular Treemap¶
The rectangular treemap is used to visualize the proportion distribution of metric data under different groups. The color of the rectangular treemap represents the group's tag (Tag), and the area represents the metric data. When multiple metric queries are added, you can choose the main metric to display, and the size of the area is shown based on the selected metric data results, Top/Bottom, and quantity. For example, this chart uses a visual way to show the CPU usage ranking of hosts.
Added Associated Queries for Application Performance and User Access Monitoring¶
In the trace details of application performance monitoring, you can view associated (same trace_id) real-user access experience data through the 「Related View」 at the top of the page. This function not only helps you view proactively monitored application performance data but also helps you quickly perceive real users' access situations. More application performance monitoring can be referenced in the Trace Analysis documentation.
Added Absolute Time Range Query for Infrastructure¶
Infrastructure added 「Host」、「Container」、「Process」 query range absolute time, defaulting to displaying the last 24 hours of host data, the last ten minutes of container data, and the last ten minutes of process data, which can be refreshed manually. Hovering the mouse over an offline host allows you to view the offline handling prompt for the host.
Optimized Same-period Comparison Function for Charts¶
Optimized the same-period comparison function for charts, supporting comparisons of same-period data for time series charts, overview charts, etc. You can choose year-on-year (comparison of the same time point in adjacent time periods) or month-on-month (comparison of adjacent time periods). Based on the locked chart time, you can choose month-on-month, day-on-day, week-on-week, month-on-month, week-on-week, and year-on-year options.
Optimized SLIMIT for Chart Queries¶
Default SLIMIT is added to grouped query charts, used to limit the number of points or series returned by time series charts. One chart limits returning 10,000 points, and one query returns up to 10 time series.
Cloud Dial Test Adjusted to Availability Monitoring¶
Cloud dial tests are split into API dial tests and Browser dial tests, adjusted to availability monitoring. After adjustment, API dial tests are retained. Other functionalities such as overview analysis, Viewer analysis, self-built node features remain unchanged. For more details, refer to the Availability Monitoring documentation.
Price Reduction Adjustment¶
- Log price reduced from 1 yuan per 1 million entries to 0.5 yuan per 1 million entries
- User access monitoring price removed Session billing dimension, adjusted to 0.3 yuan per 1,000 PVs
For more details on pricing and data expiration strategies, refer to the Pay-as-you-go documentation.
September 9, 2021¶
DataFlux Renamed to "Guance"¶
In the cloud computing era, observability is increasingly adopted and valued by more and more enterprises. As an observability platform for the cloud age introduced by ZYUN Technology, DataFlux has officially been renamed "Guance" to make it easier for enterprises and users to understand. The website is "guance.com".
At the same time, we designed a new Logo for "Guance", using letters C and G as the basis to form a radar wave-like graphic, symbolizing detection and observation characteristics, fully aligning with the feature of observability, and embodying the core guarantee system that "Guance" builds for enterprises and users based on data.
Added Similar Text Log Viewer¶
"Guance" supports similarity calculation and analysis of the message
field in raw log data. Based on the selected time range in the upper-right corner, it fixes the current time period and retrieves 10,000 log data entries within that period for approximate analysis. It aggregates logs with high similarity, extracts and statistically analyzes common Patterns, helping to quickly discover abnormal logs and locate problems.
Pattern Clustering Details
Clicking the Pattern clustering list draws out the current full Pattern clustering list, with data lists sorted in reverse chronological order. It supports displaying up to 50 recent log data entries, which can be shown in 1-line, 3-line, or 10-line formats.
Added Container Jobs and Cron Jobs Viewer¶
"Guance" added viewers for container 「Jobs」 and 「Cron Jobs」. Select the corresponding viewer in the left-side object list under 「Infrastructure」-「Container」 to view the operational status and service capability-related metrics of Jobs and Cron Jobs in Kubernetes, thereby achieving real-time monitoring of Kubernetes clusters and deployed resources. Refer to the Container documentation for more details.
Added DCA Desktop Client Application¶
DCA, DataKit Control APP, is a desktop client application aimed at conveniently managing installed and configured collectors, supporting viewing integration lists, configuration file management, Pipeline management, and online viewing of integration configuration documentation. You can remotely connect to DataKit through DCA, make online changes to collectors, and apply the saved updates immediately.
In the "Guance" workspace, click 「Integration」-「DCA」 sequentially to download the installation package. After downloading, you can install it on your computer to remotely manage DataKit.
Added Mobile App¶
DataFlux Mobile can help you view logs and scene views from Guance on mobile devices anytime, anywhere, and receive all alert events notified via email, DingTalk bots, WeCom bots, Webhook, etc.
In the Guance workspace, you can download the DataFlux Mobile app by navigating to 「Integration」-「Mobile」.
Added Label Filtering Function for Metrics, Optimized Three Metric View Modes¶
The DataKit collector appends the tag host=<DataKit hostname>
to all collected data by default,
Labels are a collection of properties identifying the data point acquisition objects. Labels consist of label names and label values, and you can choose the corresponding metric view for the required label values on the 「Metrics」 page. As shown in the figure below: filter the host in the label bar and view its usage_system
, usage_total
, usage_user
, usage_iowait
metric views.
On the 「Metrics」 page, you can choose to switch metric view modes on the left side. Three view modes are supported: Tile Mode, Mixed Mode, List Mode, with Tile Mode selected by default.
Optimized Anomaly Detection¶
Added window (window) function for use in anomaly detection rule queries and chart queries, i.e., supports using selected time intervals as windows (record sets) and detection frequencies as offsets to re-execute statistical calculations for each record.
• When performing chart queries, supports adding window functions, i.e., using selected time intervals as windows (supports selecting 1 minute, 5 minutes, 15 minutes, 30 minutes, 1 hour, 3 hours), combined with aggregation functions to execute statistical calculations for each record.
• When configuring anomaly detection rule configurations, adds the window function as the chart display area for query results, returning real-time anomaly detection metric data used to trigger alerts.
Added percent() function for use in aggregation functions in anomaly rule queries and chart queries. Supports adding the percent() function in aggregation functions, including p50 (median value), p75 (value at 75% position), p90 (value at 90% position), p99 (value at 99% position).
Added label filtering for use in anomaly rule queries and chart queries, supporting selecting host label properties for display filtering. Before selecting labels in queries, you need to set label properties for the host in 「Infrastructure」-「Host」.
Optimized Unresolved Event Display Style¶
In the unresolved event list, you can preview the window function of the event for the last 6 hours:
• The dotted-line border effect indicates the time period affected by the anomaly event
• When the detection library rule type is threshold, log, application performance metrics, user access metrics detection, security inspection, anomalous process, or cloud dial test detection, you can view related anomaly detection metric data according to the block color corresponding to different alert levels, including critical, error, warning.
• When the detection library rule type is mutation, interval, or level, you can quickly identify the time point when the current event is triggered based on the vertical line in the chart.
You can view the status distribution, DQL function, and window function line chart for anomaly events on the event details page.
• Status distribution: Displays the event status (critical, important, warning, data interruption) within the selected time range (default displays the last 6 hours)
• DQL query statement: Real-time metric data returned by the custom query statement based on anomaly detection rules, default displays the last 6 hours of real-time metric data
• Window function: Based on anomaly detection rules, uses the selected time range as the window (record set), with the detection frequency as the offset, re-executing statistical calculations for each record, returning real-time anomaly detection metric data used to trigger alerts. Default displays the last 6 hours of real-time anomaly detection metric data
Optimized Scene View Analysis¶
In preview mode, double-clicking a chart enters analysis mode to view. In edit mode, double-clicking the chart enters edit mode for adding, deleting, modifying, and querying charts.
Optimized Built-in View Binding for Infrastructure¶
When selecting built-in views, duplicate-named views will be filtered. When system views and user views have the same name, user views will be prioritized, meaning only user views can be chosen. If you need to choose a system view, you must change or delete the duplicate-named user view.
Optimized Billing Methods, Added Cloud Dial Test Billing, Adjusted Data Expiration Strategies¶
- Added PV Count Statistics: Billing based on the lower cost between Session count and PV count
- Added Cloud Dial Test Task Count Fees: Opened self-built node management for cloud dial tests, supporting any workspace administrator to create new dial test nodes globally; Cloud dial test billing will be based on the number of calls made for cloud dial test tasks in the current workspace; Experience Edition workspaces can create up to 5 dial test tasks and only support "China Zone" dial test nodes; Agile Edition and higher versions support creating more dial test tasks and using more international dial test nodes.
August 26, 2021¶
DataFlux Studio¶
Added Chart Query Label Filtering¶
DataFlux supports selecting host label attributes for filtering display. Before selecting labels in chart queries, you need to set label attributes for the host in 「Infrastructure」-「Host」. As shown in the figure below, there are two hosts with the "production" label attribute, so data for two hosts is displayed.
Added Infrastructure Container Cluster, Replicate Set, Node Viewer¶
DataFlux added Cluster, Replicate Set, Node viewers for containers, supporting viewing the operational status and service capability-related metrics of Cluster, Replicate Set, Node in Kubernetes, thereby achieving real-time monitoring of Kubernetes clusters and deployed resources.
Added Quick Filter for User Access Monitoring VIEW Viewer¶
DataFlux supports quick filtering in the VIEW Viewer by clicking the settings button on the left, allowing custom addition of filter fields. After selecting the fields to add, click "Complete" to proceed.
Note: If the added filter field represents a time-related field, you can filter and view the corresponding page content by selecting a time range. For example, loading_time (page load time).
Added Navigation Menu Help Entry, Added Multiple Viewer Data Collection Guidance Entries¶
DataFlux added a help documentation entry in the navigation menu. When the Viewer does not upload data, DataFlux provides a quick access "How to Start Data Collection" entry to help you timely initiate real-time monitoring across different Viewers. This includes how to start log monitoring, how to start cloud dial tests, how to collect host objects, etc.
Optimized Log Viewer¶
DataFlux optimized the Log Viewer, using different colors to highlight different contents of logs, allowing you to quickly obtain key data information from logs, supporting both light and dark themes.
Log Content | Light Theme | Dark Theme |
---|---|---|
Date (log occurrence time) | Yellow | Pale yellow |
Keywords (HTTP protocol related, such as GET) | Green | Pale green |
Text (string with quotes) | Blue | Pale blue |
Default (text without any markers) | Black | Gray |
Numbers (log status codes, such as 404) | Purple | Pale purple |
Optimized Overview Statistics for Security Inspections¶
DataFlux supports customizing the view of the security inspection overview page. By clicking the 「Jump」 button, you can navigate to the corresponding built-in view on the overview page for viewing, editing, copying, and exporting.
Optimized Workspace Switching¶
DataFlux supports switching to other workspaces or creating new workspaces by clicking the 「Current Space」 name.
Optimized System Theme Color Switching¶
DataFlux added an "Auto" option for system theme colors, automatically switching theme colors based on computer appearance settings.
Other Optimized Features¶
- Optimized 「Service」 details in application performance monitoring, clicking the resource name in the service details page navigates directly to 「Trace」 to view the corresponding resource trace;
- Optimized 「Performance」 details in user access monitoring VIEW Viewer, clicking error data (data marked as "error" in the VIEW list) views the corresponding error details;
- Optimized label name and value display in the metric Viewer;
- Optimized X-axis time display for time series charts, defaulting to the last 15 minutes with segments by minute intervals; last 1 day, segments by hour intervals; last 7 days, segments by day intervals;
- Optimized expression queries, requiring consistent grouping labels if the expression query contains multiple query statements;
- Optimized host integration running status, adding DataKit version information for installed hosts in 「Infrastructure」-「Host」-「Integration Running Status」;
- Optimized snapshot sharing, supporting sharing snapshots with "anyone" who has the share link or encrypted sharing with those who have the link and key.
August 12, 2021¶
DataFlux Studio¶
Added 3 Permission Roles, Optimized Scene View Permissions¶
DataFlux currently supports defining three workspace member permissions, including "Administrator," "Standard Member," and "Read-only Member," constraining management permissions, operation permissions, and viewing permissions for different member categories respectively. "Administrators" can customize member permission scopes, such as limiting read-only members' accessible scene views, limiting standard members' editable scene views, etc.
In the DataFlux workspace's 「Management」-「Member Management」-「Modify」, you can edit and update member permissions.
Added Infrastructure Host Details View Connection Tracking and File Layout Optimization¶
DataFlux supports viewing host system information in the infrastructure host details page, including host name, operating system, processor, memory, network, disk, connection tracking count, file handle count, etc.
Added Scene View Snapshot Saving Function¶
In the DataFlux workspace, go to 「Scene」-「Node」, click the 「Settings」 button in the top navigation bar, and choose 「Save Snapshot」 to save the current view's displayed data content as a snapshot.
Additionally, DataFlux supports saving snapshots quickly via page function buttons and shortcut keys (Ctrl+ K). In the scene view and all viewers, using the shortcut key Ctrl+K will quickly pop up the 【Save Snapshot】 dialog box. Enter a name and click 【OK】 to add the current page as a new snapshot.
Note: If your backend program has other shortcut keys that conflict with the shortcut key Ctrl+K, you will not be able to use the "Save Snapshot" shortcut key function.
Added Chart and Snapshot Sharing Management¶
DataFlux supports unified management of charts and snapshots shared within the current space. After completing chart sharing in the 「Scene View」, you can view the chart sharing list within the current space through 「Management」-「Sharing Management」-「Chart Sharing」, and perform chart viewing, embedding code viewing, and canceling sharing.
Added User Email Replacement and Mobile Unbinding Function¶
In the DataFlux workspace, click the lower-left 「Account」-「Settings」-「Basic Settings」 to replace or unbind mobile and email.
Added Custom User Access Monitoring SDK Data Collection Content¶
DataFlux supports customizing user access monitoring SDK data collection content by setting user identifiers, custom sessions, and adding extra data TAGs, helping users locate and analyze data under specific scenarios by setting different types of identifiers.
Added Free Quota for Paid Edition Session, Task Scheduling, and Trace¶
The paid version of DataFlux adds 1 million log entries per day, 100 Session counts, 10,000 task scheduling counts, and 1 million Trace counts per DataKit daily.
Optimized Scene Chart Analysis Mode¶
DataFlux supports previewing object data interaction changes within four times the query cycle via the timeline below the time series chart in analysis mode, and selecting the time range displayed on the time series chart by dragging the timeline. For example: if the current time is 14:00 and the time range is selected as 【10:00-11:00】, then the timeline range is 【08:00-12:00】.
Note: A single query statement in the time series chart returns up to 10 time series, i.e., according to the results of the grouping (group by) conditions, data exceeding 10 time series are displayed only for the first 10 time series in order.
Optimized Infrastructure Host and Container Honeycomb Charts¶
Hovering the mouse over host and container objects allows you to view the names, CPU usage rates, and MEM usage rates of hosts and containers.
Optimized Log Source List Display Based on Viewer Time Range¶
DataFlux supports filtering log source lists based on viewer time ranges. For example, in the DataFlux workspace 「Log Viewer」, selecting a 15-minute time range in the upper-right corner displays only logs with data in the last 15 minutes in the left log list.
Optimized System Theme Color Switching¶
DataFlux supports switching system theme colors, divided into light and dark themes. Click the account in the lower-left corner and select the required theme under 「System Theme」 to switch directly.
Other Optimized Features¶
- Webhook custom notification sending content type changed from text format to JSON format.
- Added an alert silence option of 15 minutes for anomaly detection library alert settings, meaning no notifications will be sent for the same alert within a 15-minute time range.
- Adjusted the display style of unresolved event list pages, changing log alert event statistics from table charts to time series charts.
August 4, 2021¶
DataFlux Studio¶
Added Container Services and Deployments Viewer¶
DataFlux supports displaying the status of containers comprehensively from different dimensions through 「Container」, 「Pod」, 「Services」, 「Deployments」 viewers.
In the workspace 「Infrastructure」-「Container」, you can switch to 「Services」 via the data type filter bar in the upper-left corner to view detailed information of all retained Services in the space, including Service name, service type, Cluster IP, External IP, runtime duration, etc.
Switching to 「Deployments」 allows you to view detailed information of all retained Deployments in the space, including Deployment name, available replicas, upgraded replicas, readiness, runtime duration, etc.
July 29, 2021¶
DataFlux Studio¶
Added Global Label Function for Infrastructure¶
DataFlux added a global Label function for infrastructure, supporting users adding custom labels to infrastructure objects such as hosts, containers, and processes, and filtering and displaying infrastructure objects with the same label through added labels.
In the infrastructure object list, clicking the object name draws out the details page to add labels 「Label Attributes」 for the object.
After adding custom labels, you can filter and display objects with the same label in the object list via the added labels.
Added SMS Notification Object; Added SMS Fee Statistics for Paid Plans and Billing¶
DataFlux supports users managing notification objects for alerts, currently supporting adding DingTalk bots, WeCom bots, Lark bots, Webhook customizations, email groups, and SMS groups.
Note:
- SMS notification members must be invited to join the workspace in 「Management」 - 「Member Management」 before they can be selected.
- The free trial version of DataFlux does not support SMS notifications, while other versions charge 10 yuan for every 100 SMS messages, billed daily.
Added Slide Show Mode for Scene Views¶
DataFlux added slide show browsing for all node views in specified scenes, supporting customizing the slide show interval.
Added Workspace Token Replacement Function¶
DataFlux supports the current space administrator replacing the token within the space and customizing the expiration time of the current token. Entering 「Management」-「Basic Settings」-「Replace Token」, selecting the expiration time and confirming 「Replace」 generates a new token automatically, and the old token becomes invalid within the specified time.
Note:
- After replacing the token, the original token will become invalid within the specified time. Any proxy using this token will immediately stop data reporting, so please check the integration configuration promptly;
- Replacing the token triggers 「Operation Events」 and 「Notifications」.
Added Metric DQL Query Mode¶
DataFlux supports switching 「Search」 to 「DQL Query Mode」 in the workspace 「Metrics」, allowing direct input of DQL statements for queries. You can view corresponding tags and metrics on the left and visualized charts returned by the query statement on the right.
Optimized Multi-line Display for Log Viewer¶
DataFlux supports selecting "1 line", "3 lines", or "10 lines" in the "Settings" of the Log Viewer to view complete log contents.
Optimized Service and Trace Switching Views for Application Performance Monitoring, Optimized Response Time Line Chart¶
DataFlux supports retaining default filtering conditions (including: environment, version) and time range when switching between 「Service」 and 「Trace」 viewers. For example: in application performance monitoring 「Service」, select 「Environment」 as production_a
and 「Time Range」 as Last 30 Minutes
.
Switching to application performance monitoring 「Trace」 retains consistent filtering conditions with 「Service」.
Optimized Installation Guide for DataKit Collector¶
DataFlux supports customizing installation options when installing DataKit, such as cloud vendors, namespaces, global tags, etc.; DataKit offline installation supports directly downloading offline installation packages for different operating systems by clicking.
Optimized Flame Graph Synchronous and Asynchronous Call Display¶
DataFlux's flame graph for application performance monitoring supports viewing detailed performance data of synchronous and asynchronous call trace links. For example, through the flame graph, you can clearly see which requests were performed asynchronously, when they started, when they ended, and how much time was spent in total.
Optimized Field Alias Display for Viewer¶
DataFlux supports custom aliases for fields displayed in viewers, such as modifying field aliases in "Add Display Columns" for infrastructure host objects. After modification, you can view them in the viewer and details.
Optimized Self-built Node for Cloud Dial Test¶
DataFlux splits the geographical location of self-built nodes for cloud dial tests into two filter boxes: country and province (or city). If the country filter box selects China, the next filter box will filter out the list of provinces; if the country selects non-China, the next filter box will filter and display the list of cities. The filter box defaults to showing 20 items and supports fuzzy search.
Optimized DQL Query Functions¶
DataFlux DQL added various query functions such as exists, wildcard, with_labels, etc.
July 15, 2021¶
DataFlux Studio¶
Added Third-party Development Tool Integration for User Access¶
DataFlux added integration of mini-program development frameworks based on uniapp. By introducing SDK files, you can monitor the performance indicators, error logs, and resource request data of mini-programs and report them to the DataFlux platform.
Added Data Gap Anomaly Detection¶
DataFlux added data gap anomaly detection, triggering events based on configured custom detection cycles. Data gap events generated after configuring the detection rules are uniformly saved in the events.
After the data gap detection rule takes effect, the first detection of data gaps and continuous data gaps do not generate data gap alarm events; if data is detected and there is a data reporting gap within the configured custom detection cycle, a data gap alarm event is generated.
Added Built-in Detection Libraries for Elasticsearch, Redis, and Docker¶
DataFlux currently supports 4 built-in detection libraries: Elasticsearch, Redis, Host, and Docker. You don't need manual configuration; they are ready-to-use out of the box. Click the corresponding built-in detection library directly to learn more details.
- Redis Detection Library: Includes abnormal increase in the number of client connections waiting for blocking commands in Redis
- Host Monitoring: Includes multiple data metrics for host CPU, memory, disk, etc.
- Elasticsearch Detection Library: Includes ES JVM heap memory, thread pool, search query load anomalies, etc.
- Docker Monitoring: Includes container CPU, memory, and other data metrics
Added "All Trace Viewer" for Application Performance¶
DataFlux provides three trace screening viewing lists: "All Span," "Top Span," and "Traceid." Span represents a logical work unit within a given time period in a distributed system, and multiple spans form a trace trajectory.
- All Spans: Displays all collected span data within the selected time range
- Top Spans: Filters and displays all spans entering for the first time within the selected time range
- All Traces: Filters and displays all traces containing initial top-level spans within the selected time range
Added Topology Call Relationship Diagram for Trace Details
DataFlux added a corresponding service topology call relationship diagram in the application performance monitoring details page to view the call relationships between various services.
Added User Access Error Details Page¶
DataFlux supports viewing user access error details in the user access details page, where you can view error data information, error types, and error occurrence times during that user access.
Clicking the error information navigates to the corresponding error details page.
Added Security Inspection Overview Statistics¶
In 「Security Inspection」-「Overview」, you can filter hosts, security inspection levels, and security inspection categories to view the overview of security inspection events occurring on different hosts, including the quantity of security inspection events of different levels and visual chart analysis, and rankings of security inspection events of different categories and rules.
Added Handling Suggestions for Security Inspection Events¶
In the security inspection viewer, click the inspection event you want to view. In the drawn-out details page, you can view handling suggestions for the current security inspection event, including theoretical basis, risk items, audit methods, remedial measures, etc.
Added Secondary Navigation Bar¶
DataFlux added support for displaying secondary navigation via "mouse hover," thereby reducing the time users spend searching for target data.
Added Member Management Search Function¶
DataFlux supports searching for current workspace members quickly based on email or name in 「Member Management」.
Added Binding Built-in Views to Infrastructure Objects¶
DataFlux supports binding built-in views (system views, user views) to infrastructure object detail pages by current space administrators. As shown in the figure below, it binds the built-in system view "Disk Monitoring View."
Optimized Event List Display by Detection Library Type¶
DataFlux supports displaying corresponding event information in the event list based on different detection library types.
- When the detection library rule type is threshold, mutation, interval, level, application performance metrics, or user access metrics detection, it displays the baseline and corresponding information configured for the trigger conditions, including critical, error, warning.
- When the detection library rule type is log, security inspection, anomalous process, or cloud dial test detection, it queries all log data for anomaly events.
As shown in the figure below, there are three different detection library types, including threshold detection, security inspection detection, and process data detection.
Optimized Deletion of Metric Sets¶
DataFlux supports administrators deleting metric sets within the space. Enter 「Management」-「Basic Settings」, click 「Delete Metric Set」, input keywords to query and select the metric set name, and click 「Confirm」 to enter the deletion queue. Deleting a metric set generates system notification events, such as users creating delete metric set tasks, successful execution of delete metric set tasks, and failed execution of delete metric set tasks.
Optimized Built-in View Query Time Range¶
Changed the query time range for built-in views, including the 「Host Monitoring」 view and 「Process」 view on the details pages of infrastructure objects, traces, and logs.
- When viewing trace and log details, the 「Host Monitoring」 view by default displays performance data associated with the host within the selected time component range.
- When viewing infrastructure object (host, container) details, the 「Host Monitoring」 view by default displays host performance data within the last 24 hours.
- When viewing trace details, the 「Process」 view by default displays real-time process data within 10 minutes before and after the trace occurrence time point.
- When viewing log details, the 「Process」 view by default displays real-time process data within 10 minutes before and after the log generation time point.
Optimized Quick Filter Interaction for Viewers¶
If the current field value is too long in DataFlux quick filters, the excess part is displayed as ...
, and hovering the mouse over the current field value area expands the display according to the field value length.
July 1, 2021¶
DataFlux Studio¶
Added Unresolved Event Statistics Viewer¶
DataFlux added an unresolved event statistics viewer. In the DataFlux workspace's Events, by switching the viewer in the upper-left corner, you can view all unresolved events triggered continuously within the space, as well as data volume statistics and alarm information details for unresolved events under different alarm levels.
Added Chart Link Function¶
DataFlux added chart link functionality for time series charts, bar charts, overview charts, dashboards, pie charts, ranking charts, scatter plots, bubble charts, maps, and honeycomb charts. Chart links help you navigate from the current chart to the target page, supporting modification of corresponding variable values in the link via template variables to transmit data information, completing data linkage.
Chart links support obtaining template variables via queries, adding internal and external links, and opening in new pages, current pages, and sliding-out detail pages. Links can also have aliases added for display.
After adding the link, clicking the chart allows you to view clickable "Custom Links".
Added Application Performance Monitoring Bound Links and Built-in Views¶
DataFlux supports adding built-in views in individual service details, including built-in views and web addresses that can be linked. In the service list, click 「Settings」 to add one or more jump link addresses or built-in views and customize their names.
After adding the built-in view, you can view the corresponding built-in page by checking the service details.
Added PV/UV Overview Statistics for User Access Monitoring Scenarios¶
DataFlux added PV/UV overview statistics for user access monitoring in Web, iOS, Android, and Mini Program scenarios. You can quickly compare differences in PV/UV to understand actual user access numbers.
Added Notification Object: Lark Bot¶
DataFlux supports users managing notification objects for sending alerts, currently supporting adding emails, DingTalk bots, WeCom bots, Lark bots, and Webhook customizations.
Optimized Infrastructure Resource Catalog¶
Infrastructure data for objects other than hosts, containers, and processes can be reported to 「Infrastructure」-「Custom」 via DataFlux Func for custom reporting and analysis. Supports binding built-in views to resource catalog data and manually adding tags for filtering.
Reporting resource catalog data requires first installing and connecting DataKit and Func, then reporting data to DataKit via Func, and finally reporting data to the DataFlux workspace via DataKit.
Optimized Detection Rules for Anomaly Detection Library¶
DataFlux added recovery alert configuration for anomaly events; added support for regex matching styles in filtering conditions, optimizing =~
to match
, and adding not match
; added support for non-numeric data matching in trigger conditions, adding match
, not match
regex matching, and between
.
Optimized Associated Logs on Viewer Detail Pages¶
DataFlux supports filtering associated logs based on trace_id
in the side-sliding detail page of traces, viewing logs under the same trace_id
.
In trace details, clicking the 「Logs」 below the flame graph allows you to view associated logs based on the current trace-related host and trace ID (trace_id
field). Additionally, you can perform keyword searches and multi-tag filtering on logs. If you need to view more detailed log content, you can click the log content to navigate to the log details page or click the jump button to open the log page.
Added log association pages to viewers.
Optimized Viewer Search Styles¶
Adjusted search styles for creating new scenes, metrics, application performance monitoring, user access monitoring, anomaly detection libraries, host mute management, integrations, manage/built-in views, snapshots, etc.
Deployment Version Supports Modifying Data Retention Duration¶
Added change data storage strategy to basic information in workspace management.
June 17, 2021¶
DataFlux Studio¶
Added Log Analysis Mode¶
DataFlux supports grouping large amounts of raw data based on 1-3 tags through the log grouping function, reflecting the distribution characteristics and trends of grouped data at different times. DataFlux supports three ways to browse grouped data: line chart, area chart, bar chart.
Added Log Filtering¶
Added log filtering function (log blacklist), supporting setting filtering rules to filter out logs that meet the conditions, meaning that logs meeting the filtering criteria will no longer be reported to the DataFlux workspace, helping users save on log data storage costs.
In 「Logs」-「Filter Rules」-「Create New Rule」, select "log source" and add one or more data filtering conditions with a relationship of and(and) to filter collected log data. Logs meeting the filtering criteria will not be reported to the dashboard.
Added Saving Metric Browse Snapshots, Added Snapshot Link Copy Function¶
DataFlux supports creating quick-access data copies for metric viewers and saving them uniformly in the "Snapshot" directory; added generating shareable snapshot links, supporting sharing snapshot links with other members within the space to quickly reproduce instant copied data information and restore data to a certain time point and data display logic.
Added Pod Viewer for Containers¶
DataFlux added a Pod viewer under 「Infrastructure」-「Containers」. Through the filter in the upper-left corner, you can switch to 「Pod」 to view detailed information of all collected Pods, including Pod name, belonging node Node, one or more containers contained, start status, running status, and related metrics. You can also switch to the honeycomb chart for viewing.
Clicking the Pod in the honeycomb chart allows you to view the Pod's details, including Pod name, namespace, visual monitoring charts, etc.
Added Free Quota Warning Notifications¶
Added warning notifications for used quotas in the Free Plan and notifications when the quota is exhausted, supporting email and system notifications. Learn more about message notifications. System message notifications can be viewed in the DataFlux workspace 「Account」-「Notifications」.
Added Chart Sharing¶
DataFlux supports chart sharing, enabling insertion of charts into platforms outside DataFlux for visual data display and analysis. Shared charts will change according to changes in charts within DataFlux. Charts shared in scene views are stored uniformly in 「Integration」-「Embedded Sharing」.
Optimized Chart Queries and Chart Styles and Settings¶
Adjusted and optimized visualization chart queries, styles, and settings in scene views, adding features like hiding chart titles, time slice settings, overview chart data precision settings, ranking chart baseline settings, map/honeycomb chart color series filling settings, etc.
Optimized Integrations, Added DataKit and Function Installation Guides, Added Embedded Sharing¶
DataFlux supports viewing all supported collectors, DataKit installation instructions, DataKit upgrade commands, Function installation instructions, and embedded sharing chart codes in the integration section.
In the DataFlux workspace, click the 「Integration」 menu on the left to view all supported collectors. Clicking a collector shows its configuration details.
In the DataFlux workspace, click 「Integration」-「DataKit」 to obtain DataKit installation instructions, including Linux, Windows, MacOS three major operating systems, and offline installation methods. If DataKit is already installed but not the latest version, you can get the update script from here under "DataKit Upgrade".
In the DataFlux workspace, click 「Integration」-「Function」 to obtain Function installation instructions.
Optimized Host Details Page¶
In 「Infrastructure」-「Host」, clicking the host name in the list draws out the details page to view basic information about the host, processor, memory, network, hard disk, etc. Supports automatic conversion of data units and displays different host monitoring views based on different operating systems.
June 3, 2021¶
DataFlux Studio¶
Added Browser Log Collection¶
DataFlux supports actively sending different levels of log data (corresponding source: browser_log
metric type log data) to DataFlux via web browsers or JavaScript clients.
Added Session Viewer for User Access Monitoring¶
DataFlux added a Session Viewer to the User Access Monitoring Viewer, showing user access behavior monitoring data (session data) and session data details related to Web/Mini Program/Android/iOS applications. In the workspace, open 「User Access Monitoring」-「Select Any Application」-「Viewer」, and switch to the Session Viewer (Session). The Session Viewer statistics include the entire session's duration, number of pages, number of operations, number of errors, etc. Clicking a single session allows you to view the detailed records of the entire session. By viewing and analyzing session data from multiple dimensions, you can better understand real user access data and help improve application performance experience.
Added Browser Dial Test, Cloud Dial Test Anomaly Detection Templates, and Optimized Self-built Nodes¶
- Browser Dial Test
DataFlux added Browser Dial Tests. In DataFlux 「Cloud Dial Test」, click 「New」-「Browser Dial Test」 to create a new Browser Dial Test task, obtaining real-time user access experience data for web pages, including page load time, resource load time, etc.
- Anomaly Detection Template
「Cloud Dial Test Data Detection」 monitors cloud dial test data within the workspace. By setting threshold (boundary) ranges for specified data volumes generated by dial test tasks over a certain time period, alerts are triggered when the data volume reaches the threshold range. You can also customize alert levels, triggering different levels of alert events when the specified data volume reaches different threshold ranges.
- Self-built Nodes
DataFlux's cloud dial test self-built nodes added "Node Code", used to obtain the Code for node information, ensuring that the node Code is not duplicated within the current space.
Added Welcome Page on Registration¶
After registering and logging into a DataFlux account, a welcome page prompts you to view a DataFlux introduction video, scan a QR code to join the DataFlux WeChat and DingTalk service groups.
Supported DataFlux Func in Free Plan, Adjusted Billing Items¶
All versions of DataFlux fully support the DataFlux Func platform for custom functions. DataFlux Func is a script development, management, and execution platform based on Python similar to Serverless. At the same time, to reduce user costs for using DataFlux, DataFlux adjusted the billing logic for application performance monitoring and user access monitoring. After adjustment, application performance monitoring counts trace quantities, and user access monitoring counts daily session quantities.
Optimized Generated Metrics¶
Added generated metrics for user performance monitoring, user access monitoring, security inspections, optimized log-generated metrics. Generated metrics can be configured in three steps: data filtering, data querying, and generating metrics, including configuring dimensions, aggregation time, defining aggregation frequency, metric sets, and metric names. Examples are as follows:
- Application Performance Monitoring
- User Access Monitoring
- Security Inspections
Optimized Built-in Views, Added Clone Function¶
Built-in views are divided into "System Views" and "User Views", supporting cloning from "System Views" to "User Views", and allowing system views and user views to have the same name. Clicking "View" on the system view jumps to the system view details.
Clicking "Clone" opens the clone view dialog box, and clicking confirm creates the corresponding user view based on the current view template.
Successfully created, jumps to the created user view page.
If user views and system views have the same name and the system view is applied in the Explorer detail view, such as "Infrastructure - Host - Host Details" under the host view, the user view replaces the system view in the host view display.
Optimized View Variables¶
DataFlux supports adding global variables to views. When you want to dynamically filter charts in the view, you can choose to use view variables to achieve this. After configuring view variables, you can select whether to enable "Default Match All" via "Advanced Settings".
After enabling, you can view the comprehensive view of all hosts via 「*」, and manually select variables for filtering and viewing.
Optimized Viewers¶
- When single-column information is not fully displayed, hovering the mouse delays 1 second before showing all tooltip information;
- When searching key content in the viewer, the matched content in the list highlights;
- Tags on the details page support copy functions; host tags support [View Related Logs], [View Related Containers], [View Related Processes], [View Related Traces], [View Related Inspections];
- Statistical charts for log, application performance monitoring, user access monitoring, cloud dial test, security inspection viewers have been changed to state statistics;
- Added help links for starting user performance monitoring, user access monitoring, security inspections.
May 20, 2021¶
DataFlux Studio¶
Added Save Snapshots¶
DataFlux snapshots support creating snapshots for infrastructure, logs, events, application performance monitoring, user access monitoring, cloud dial tests, security inspections viewers. Through snapshots, you can quickly view and analyze data and anomalies at different times and tags.
Added Infrastructure Host Integration Interactions¶
The 「Integration Running Status」 of infrastructure hosts displays the running status of related collectors for the host. There are two running statuses: normally running integrations and erroneous integrations. Normally running integrations are displayed by default as 'light blue', and erroneous integrations are displayed by default as 'red' and allow you to view error information by clicking.
Simultaneously, by clicking the integration with a built-in view symbol (as shown in the figure below), you can "View Monitoring Views" and click to view related built-in views.
Added Self-built Nodes, Optimized Cloud Dial Test Node Display¶
「Cloud Dial Test」 supports custom data collection through the DataFlux Studio console. By creating a probe point, you can periodically monitor websites, domains, backend interfaces, etc., based on the HTTP protocol, analyzing site quality at any time. DataFlux covers 16 nodes globally, categorized by domestic, overseas, and self-built nodes for selection. It also supports self-built dial test nodes, ensuring service availability.
Added Process, Application Performance Monitoring, and User Access Monitoring Check Rules¶
「Process Anomaly Detection」 is used to monitor process data within the workspace, supporting setting alert triggers for one or more fields in process data. For example: you can set alerts based on the 'host' field being 'izaqbin' and the 'state' field being 'sleep' appearing a certain number of times.
「User Access Indicator Detection」 is used to monitor indicator data in the 「User Access Monitoring」 within the workspace. By setting threshold ranges, alerts are triggered when indicators reach the thresholds, supporting setting alerts for single indicators and customizing alert levels.
「Application Performance Indicator Detection」 is used to monitor indicator data in the 「Application Performance Monitoring」 within the workspace. By setting threshold ranges, alerts are triggered when indicators reach the thresholds, supporting setting alerts for single indicators and customizing alert levels.
Optimized Integration¶
After logging into the DataFlux workspace and entering the 「Integration」 page, you can view all supported collectors. You can also click the "Quick Get DataKit Installation Command" in the upper-right corner to directly obtain the DataKit installation command for installation. After installation, you can start more collectors to collect data metrics.
Optimized Log Metric Generation¶
Log metric generation added log source selection, allowing you to select log sources, filter logs, configure dimensions, aggregation frequencies, define metric sets, metric names, and aggregation rules to generate metrics. [Learn More About Log Metric Generation]
Optimized Log Viewer¶
After log data is collected into DataFlux, you can view all log sources and their contents in the 「Logs」 of the DataFlux Studio console. Supporting searching log sources to select log details to view.
Optimized User Access Monitoring Viewer¶
On the performance details page of user access monitoring, you can view front-end page performance, including page load time, content rendering time, interaction time, input delay, etc. As shown in the figure below, the LCP (largest content painting time) indicator reached 5.43 seconds, while the recommended time is within 2.5 seconds, indicating slow page loading speed requiring optimization. Supporting filtering and searching to help users quickly locate resources and content.
Switching to 「Trace」 allows you to view requests made to the back-end application, including the occurrence time, content, and duration of invoked resources. For example, you can view each AJAX request returned by the xhr object.
Switching to 「Errors」 allows you to quickly locate front-end errors through detailed information including error messages, error types, and occurrence times.
Optimized Built-in Views¶
Built-in views are divided into two types: system views and user views, applicable in scenarios.
System views provide integrated view templates provided by the system, supporting modifications, exports, and restorations. Users can directly 「Export」 system views or 「Modify」 system views. By 「Restoring」 system views, modified system views can be converted back to original system views. Restored views cannot be recovered.
User views are customized views saved as templates by users, supporting modifications, exports, and deletions. In 「Built-in Views」-「User Views」, clicking 「Create User View」 allows you to create user-defined view templates. Specific configurations can refer to Scenes and Views.
If user views and system views have the same name and the system view is applied in the Explorer detail view, such as "Infrastructure - Host - Host Details" under the host view, the user view replaces the system view in the host view display.
Optimized View Variables¶
DataFlux supports adding global variables to views. When you want to dynamically filter charts in the view, you can choose to use view variables to achieve this. After configuring view variables, you can select whether to enable "Default Match All" via "Advanced Settings".
After enabling, you can view the comprehensive view of all hosts via 「*」, and manually select variables for filtering and viewing.
Optimized Viewers¶
- When single-column information is not fully displayed, hovering the mouse delays 1 second before showing all tooltip information;
- When searching key content in the viewer, the matched content in the list highlights;
- Tags on the details page support copy functions; host tags support [View Related Logs], [View Related Containers], [View Related Processes], [View Related Traces], [View Related Inspections];
- Statistical charts for logs, application performance monitoring, user access monitoring, cloud dial tests, security inspections have been changed to state statistics;
- Added help links for starting user performance monitoring, user access monitoring, security inspections.
May 7, 2021¶
DataFlux Studio¶
Added Save Snapshots¶
DataFlux snapshots support creating snapshots for infrastructure, logs, events, application performance monitoring, user access monitoring, cloud dial tests, security inspections viewers. Through snapshots, you can quickly view and analyze data and anomalies at different times and tags.
Added Infrastructure Host Integration Interaction¶
The 「Integration Running Status」 of infrastructure hosts displays the running status of related collectors for the host. There are two running statuses: normally running integrations and erroneous integrations. Normally running integrations are displayed by default as 'light blue', and erroneous integrations are displayed by default as 'red' and allow you to view error information by clicking.
Simultaneously, by clicking the integration with a built-in view symbol (as shown in the figure below), you can "View Monitoring Views" and click to view related built-in views.
Added Self-built Nodes, Optimized Cloud Dial Test Node Display¶
「Cloud Dial Test」 supports custom data collection through the DataFlux Studio console. By creating a probe point, you can periodically monitor websites, domains, backend interfaces, etc., based on the HTTP protocol, analyzing site quality at any time. DataFlux covers 16 nodes globally, categorized by domestic, overseas, and self-built nodes for selection. It also supports self-built dial test nodes, ensuring service availability.
Added Process, Application Performance Monitoring, and User Access Monitoring Check Rules¶
「Process Anomaly Detection」 is used to monitor process data within the workspace, supporting setting alert triggers for one or more fields in process data. For example: you can set alerts based on the 'host' field being 'izaqbin' and the 'state' field being 'sleep' appearing a certain number of times.
¶
- Fixed frequent data collection by the Aliyun CMS collector causing some other collectors to freeze.
March 25, 2021¶
DataFlux Studio¶
Added Cloud Service Association¶
DataFlux supports centrally managing AccessKey information for cloud service accounts through 「Cloud Association」. After configuration, you can regularly synchronize cloud host information under the associated cloud account in [Infrastructure - Host], displaying all cloud host information, such as CPU, memory, network, disk, etc.
If you no longer need the association, you can delete the associated Alibaba Cloud RAM sub-account AK information. Deleting the association triggers an association invalidation notification event, which can be viewed in Events - Notification Events. For more details on cloud associations, refer to the Cloud Association documentation.
Added Operation Event Auditing¶
DataFlux supports auditing operation events within workspaces, storing operation events uniformly in Events - 「Operation Events」.
Added Host Muting¶
If you need to perform operations such as stress testing on a single cloud host, it will trigger frequent alerts. For such alert notifications, you can temporarily enable the host muting function. In 「Infrastructure」-「Host」, clicking the cloud host details allows you to choose the host muting duration.
All muted hosts can be viewed in 「Anomaly Detection Library」-「Host Muting Management」. Deleting a muted host allows you to receive corresponding host alert notifications again.
Added Email Groups¶
Added email group notification targets in [Notification Targets] management. Email groups can simultaneously add multiple members. After addition, selecting the email group in the anomaly detection library's "Alert Settings" sends alert emails to all added member emails simultaneously.
Optimized Anomaly Detection Library¶
DataFlux includes various built-in detection libraries, ready-to-use out of the box. It supports multiple built-in detection libraries such as hosts. Upon activation, you can receive related anomaly event alerts.
In 「Detection Library」, supports 「Import/Export Detection Libraries」. After configuring all detection rules, you can export the detection library via JSON files. The exported detection library can be imported into other detection libraries. During import, it verifies whether duplicate-named detection items exist, allowing options to 「Replace」 or 「Skip」.
Optimized Object Viewer and Details¶
- Added operating system small icon display for host objects
- Added container type small icon display for container objects
- Added 「Start Time」 display in the process slide-out details page
Optimized Data Storage Strategies, Paid Plans, and Billing¶
- Added change data storage strategies to basic information in workspace management
- Changed workspace management usage statistics to paid plans and billing
March 4, 2021¶
DataFlux Studio¶
Added View Variable Object Mapping¶
When using 「Object Mapping」, you must first define a view variable based on object class fields. After configuring object mapping in 「View Variables」, you need to use the mapped tag field as a group in 「Chart Query」 and enable 「Field Mapping」 in 「Chart Settings」. Then, you can view the set object mapping display in the view chart.
Optimized Event Grouping¶
DataFlux supports grouping 「Anomaly Events」 according to 「Detection Items」 to aggregate related events. Clicking 「Aggregated Events」 allows you to quickly view associated event lists, different host anomaly event status distributions, and metric monitoring. Anomaly event status distributions include critical
, error
, warning
, ok
, and Mute
, corresponding to five status colors. The bottom color is green, filling backward from the event occurrence time point with corresponding status colors. The fill range corresponds to the detection frequency. When the group contains host
and host muting is set, the muted time range displays as gray.
Optimized Detection Item Queries¶
In the 「Anomaly Detection Library」, the 「Detection Rules」's 「Detection Indicators」 only allow adding one query by default, supporting switching between Simple Query and 「Expression Query」. Uniformly changed the 「Trigger Conditions」 M1
in the 「Detection Rules」 to Result
; supports adding AS
aliases to queries.
DataKit (v1.1.4-rc2)¶
- Fixed the issue of frequent data collection by the Aliyun monitoring data collector (aliyuncms) causing some other collectors to freeze.
March 25, 2021¶
DataFlux Studio¶
Added Cloud Service Association¶
DataFlux supports centrally managing cloud service account AccessKey information through 「Cloud Association」. After configuration, you can periodically synchronize cloud host information under the associated cloud account in [Infrastructure - Host], displaying all cloud host information, such as CPU, memory, network, disk, etc.
If you no longer need the association, you can delete the associated Alibaba Cloud RAM sub-account AK information. Deleting the association triggers an association invalidation notification event, which can be viewed in Events - Notification Events. For more details on cloud associations, refer to the Cloud Association documentation.
Added Operation Event Auditing¶
DataFlux supports auditing operation events within workspaces, storing operation events uniformly in Events - 「Operation Events」.
Added Host Muting¶
If you need to perform operations such as stress testing on a single cloud host, it will trigger frequent alerts. For such alert notifications, you can temporarily enable the host muting function. In 「Infrastructure」-「Host」, clicking the cloud host details allows you to choose the host muting time.
All muted hosts can be viewed in 「Anomaly Detection Library」-「Host Muting Management」. Deleting a muted host allows you to receive corresponding host alert notifications again.
Added Notification Target: WeCom Bot¶
DataFlux supports users managing notification targets for sending alerts, currently supporting adding emails, DingTalk bots, WeCom bots, and Webhook customizations.
Optimized Event Viewer, Added Event Group Aggregation Viewing¶
In the event list viewer, through grouping functionality, DataFlux supports quickly aggregating and statistically analyzing related events based on grouped labels. For example, in 「Anomaly Events」, grouping by "Detection Item" allows you to quickly obtain all events triggered by that detection item. By switching the aggregated event list, you can view the details of these events.
Optimized Trace Service Filtering and Topology Charts¶
In the 「Application Performance Monitoring」's 「Service」 list, supports switching the list to topology chart mode to view the call relationships between various services. Supports filtering and displaying through different performance indicators and customizing trace service performance indicator color ranges.
Other Optimized Features¶
- Optimized log, event, infrastructure, application performance monitoring, user access detection viewers' search and export, added exporting to views. Optimized label filtering, associated query analysis on log, event, trace, infrastructure-container, infrastructure-process detail pages.
- Optimized time components, optimized simple query and expression query switching for single-query charts (overview charts, dashboards, ranking charts) in scene views.
- Optimized offline host display: when the host is offline, CPU usage rate, memory usage rate, and load are not displayed, values are filled with " - ".
- Optimized anomaly detection library detection rules, added level detection, interval detection, and mutation detection.
DataKit (v1.1.4-rc1)¶
- Added more information to the message field in the process collector for full-text search convenience.
- Supported custom tags in the host object collector for easier cloud attribute synchronization.
DataKit (v1.1.4-rc0)¶
- Added file collector, dial test collector, and HTTP packet collector.
- Internally supported ActiveMQ/Kafka/RabbitMQ/gin (Gin HTTP access logs)/Zap (third-party log framework) log cutting.
- Enriched statistical information on the [http://localhost:9529/stats] page, adding information such as collection frequency (n/min), data volume per collection, etc.
- Increased certain cache space in DataKit itself (invalid upon restart), avoiding data loss due to occasional network issues.
- Improved Pipeline date conversion functions, enhancing accuracy. Additionally, added more Pipeline functions (parse_duration()/parse_date()).
- Added more business fields to trace data (project/env/version/http_method/http_status_code).
- Various detail improvements to other collectors.
March 4, 2021¶
DataFlux Studio¶
1. Optimized Time Series Charts, Bar Charts, Ranking Charts, Log Stream Charts, Alert Statistics Charts, Object List Charts¶
- Time Series Charts: Added 「Display Density」, including 「Low: 60 points」, 「Low: 180 points」, 「Medium: 360 points」, 「High: 720 points」. Added 「Stacked Display」, only supporting bar charts, default off.
- Ranking Charts: Chart display optimization
- Bar Charts: Chart display optimization
- Log Stream Charts: Added search box, supporting searching message content
- Alert Statistics Charts: Query display optimization
- Object List Charts: Query display optimization
2. Added Automatic Unit Conversion¶
- Default query result values automatically convert units following 「Scientific Notation K, M, B」, retaining two decimal places (1 thousand = 1K, 1 million = 1M, 1 billion = 1B)
- Renminbi automatically converts units 「Yuan, Ten Thousand Yuan, Hundred Million Yuan」, retaining two decimal places
- After setting units, scientific notation is canceled, displaying according to user-set units
3. Added Object Management¶
DataFlux supports users managing notification targets for alerts, currently supporting adding DingTalk bot notification configurations and Webhook custom notification configurations.
4. Adjusted Event, Log, Trace, Application Monitoring, Object Host, Object Container, Object Process, Other Object Viewers, Supporting Exporting CSV Documents, Maximum Support Exporting Recent 5000 Data Entries.¶
- Events
DataFlux supports collecting all event data generated based on the 「Anomaly Detection Library」. Users can view and search all event data within the workspace on the Events page.
- Object Hosts
DataFlux supports collecting object host data. In the host list under 「Object」, you can view and search all host object data information.
Clicking the host name in the list draws out the details page to view detailed information about the host object, including basic information, CPU, memory, network, hard disk, etc.
- Object Containers
DataFlux supports collecting object container data. In the container list under 「Object」, you can view and search all container object data information. Clicking the container name in the list draws out the details page to view detailed information about the container object.
- Object Processes
DataFlux supports collecting object process data. In the process list under 「Object」, you can view and search all process object data information. Clicking the process name in the list draws out the details page to view detailed information about the process object.
- Other Objects
DataFlux supports collecting other object data, such as Alibaba Cloud ECS, etc. In the other list under 「Object」, you can view and search all other object data information. Clicking the other object name in the list draws out the details page to view detailed information about the other object.
- Logs
DataFlux supports collecting various formats of log data, including system logs, Nginx logs, Apache logs, etc., allowing quick viewing and analysis of all log data in the current workspace.
- Trace Monitoring
DataFlux supports viewing trace services and traces in 「Trace Monitoring」. In the service list, you can view or search all services receiving trace data. In the trace list, you can query trace data, export trace data, view trace details, and perform comprehensive analysis of trace performance through flame charts and span lists.
DataFlux supports both light and dark theme displays. Here is an example using the dark theme.
Service List
Trace Topology
Trace List
Flame Chart
- Application Monitoring
DataFlux supports collecting Web, Android, iOS, and Mini Program application data and provides application monitoring scenarios, allowing quick viewing and analysis of application conditions. Taking Web applications as an example:
Web Browser
Web Application Details
February 2, 2021¶
DataFlux Studio¶
- Added area chart style for time series charts
- Added object hosts, containers, processes, added host honeycomb charts
- Optimized anomaly detection, added built-in detection libraries
- Optimized events, objects, logs, traces, monitors, viewers, and page functionalities
- Optimized log stream chart queries
- Optimized ranking, dashboard, overview chart queries, added expression query mode.
- Optimized visual chart queries, allowing switching between simple queries and DQL queries
- Optimized grouping logic, including scatter plots, bubble charts, maps, honeycomb charts, table charts
- Optimized filtering conditions to support real-time filtering
DataKit¶
- Added host object data collection
- Added container object data collection
- Added process object data collection
January 14, 2021¶
DataFlux Studio¶
- Added Explorer layout, optimized data structure, supporting querying/analysis for objects/traces/RUM/logs modules
- Added Mini Program application monitoring
- Added time series charts, allowing chart types to be chosen as line charts or bar charts
- Adjusted chart UI
- Adjusted log stream chart queries and settings
- Optimized bar chart, ranking chart, table chart related settings
- Threshold detection adjusted to DQL UI query mode