Release Notes (2020)¶

This document records the release notes for each update of Guance, including DataKit, Guance best practices, Guance integration documentation, and Guance.

December 29, 2020¶

DataFlux Studio¶

• Added built-in views
• Added new chart type: Image
• Added support for stacked and percentage stacked bar charts
• Added unit conversion: Traffic, Bandwidth, Timestamp
• Optimized the DataWay page, added option to enable host monitoring
• Optimized DQL queries, DQL functions, and DQL CLI
• Optimized chart query methods, supporting normal queries, expression queries, and DQL queries
• Optimized view variable management, supporting DQL queries
• Optimized preview/edit view, added chart interaction
• Optimized scenario creation, supporting creation based on view templates
• Optimized object monitoring views, supporting binding views
• Relationship graphs now support saving as charts and optimized drawing interaction
• Optimized click interaction on the node tree

DataKit¶

• Added data collection support for WeChat Mini Program Application Monitoring
• Added default configuration file for enabling host monitoring

December 15, 2020¶

DataFlux Studio¶

• Added RUM (Real User Monitoring) application module, supporting Web, iOS, and Android application monitoring
• Added DQL query, supporting remote CLI
• Added composite chart (up to 5 charts combined)
• Added Top List chart, supporting both bar chart and table chart display modes
• Added chart links, allowing line charts, pie charts, bar charts, and table charts to jump from the current page to a target page
• Added chart title description
• Added Top 5 ranking and ascending/descending sorting for map chart indicators
• Optimized scatter plot, added bubble chart, supporting bubble size to represent data magnitude

DataKit¶

• Added data collection support for Web, iOS, and Android application monitoring

November 26, 2020¶

DataFlux Studio¶

• Adjusted billing model to pay-as-you-go
• Added usage statistics module for viewing version information and billing item usage
• Added object addition feature, supporting resource catalog attributes and tags
• Added relationship graph functionality for objects
• Added quick filtering for logs, events, and traces, supporting custom tags
• Added billing-related landing pages on the official website

DataKit¶

• Added object collection for Alibaba Cloud, Tencent Cloud, Huawei Cloud, and Docker containers

November 5, 2020¶

DataFlux Studio¶

• Time widget added relative time selection and data refresh interval adjustment
• Support for setting different time ranges for different views
• Added tag filtering for objects and traces
• Added 2 new chart types: Object List and Alert Statistics
• Log stream chart added unit settings and column display options
• Added metric query duplication
• Anomaly detection added log detection
• Optimized event page data loading speed
• Optimized several interactions and UI for improved user experience

October 15, 2020¶

DataFlux Studio¶

• Optimized detail pages for logs, events, and objects for clearer and more intuitive information display
• Added search functionality for view variables
• Added chart duplication feature, supporting copy and paste to any scenario
• Adjusted object data structure
• Fixed several style issues, improved chart loading speed

DataKit¶

• Optimized CSV/Excel metric collection
• Added metric collection for Huawei Cloud CloudMonitor and TiDB

September 22, 2020¶

DataFlux Studio¶

• Added "Dark Mode" theme

September 17, 2020¶

DataFlux Studio¶

• Added monitoring view functionality for objects, applicable to IT infrastructure monitoring, etc.
• Added object-type view variables

September 8, 2020¶

DataFlux Studio¶

• Added metric search functionality
• Added breadcrumb functionality
• Optimized several interactions and UI for improved user experience
• Fixed discovered bugs

August 20, 2020¶

DataFlux Studio¶

• Optimized scenario creation workflow, adjusted node tree display
• Optimized create node workflow and page UI
• View templates added collection source configuration, supporting import of template files to automatically generate collection source configuration
• Added 1 new chart type: Honeycomb Chart
• Fixed discovered bugs

August 6, 2020¶

DataFlux Studio¶

• Distributed Tracing added service topology map to display call relationships between services
• Logs, Objects, and Traces added list settings, supporting custom display columns and column names; added display settings, supporting hiding "filter bar" and "statistics chart"
• Added ticket functionality, allowing users to submit tickets for questions and suggestions

DataKit¶

• Added 15 data sources: ASP.NET, .NET CLR, Druid, Envoy, Ansible, Kibana, Microsoft Exchange Server, ModBus, MQTT Consumer, Nfsstat, Puppet Agent, systemd_utils, Kong monitoring metrics, WeChat Mini Program operation metrics, and AWS CloudWatch log and event collection

July 16, 2020¶

DataFlux Studio¶

• Adjusted left navigation menu, added Logs, Distributed Tracing, and Objects modules
• Added log collection and management, supporting "extract tag" and "generate metrics" from log data
• Added Distributed Tracing for trace data analysis and management, applicable to Application Performance (APM) monitoring
• Added object collection and analysis, supporting manual object addition
• "Key Events" renamed to Events, restructured business logic, adjusted display interface, supporting aggregation of related events; supporting display of related metric line charts triggered by events; supporting creating new events from charts in analysis mode
• Scenarios added 4 new view categories: Log View, Event View, Trace View, and Object View, supporting saving logs, events, traces, and objects to corresponding views for query and analysis
• Added tag correlation analysis, supporting querying related metrics, logs, objects, events, and traces by tag
• "Metric Browser" renamed to "Metrics", added tile mode for viewing multiple metric line charts simultaneously
• "Triggers" renamed to "Anomaly Detection", adjusted "Basic Detection" to "Threshold Detection", "Advanced Detection" to "Custom Detection Function", and added 4 new anomaly detection types: Event Detection, Mutation Detection, Interval Detection, and Water Level Detection
• Removed "Trigger History" module; alert history data is now directly saved to Events
• Added a new chart type: Log Stream Chart, supporting adding log data tables to views
• Under metric query UI mode, added InfluxDB built-in transformation functions
• Added 5 new view templates: etcd monitoring view, CoreDNS monitoring view, Traefik monitoring view, Hadoop monitoring view, RaspberryPi monitoring view
• Other optimizations and adjustments
• Optimized navigation menu collapse/expand style
• Integration page added verified markers, including "Verified" and "Beta" statuses
• Optimized sorting of derivative nodes, default ascending by first letter of node name
• Optimized UI design of several interfaces, added hover effect for view charts
• SaaS user registration added company/team information

DataFlux Admin Console¶

• Adjusted DataWay gateway usage logic; DataWay installation moved to admin console, available to all workspace members
• Support for setting different data retention durations for different types

DataKit¶

• DataKit added 15 new data sources: Containerd, GitLab, Yarn, Harbor, Tailf, Rsyslog, Jira, AWS CloudWatch Log, RaspberryPi, Go runtime metrics, Neo4j monitoring metrics, directory monitoring metrics, Alibaba Cloud security metrics, Baidu Index, and host object data collection
• DataKit supports trace data collection via Zipkin and Jaeger
• Adjusted DataKit installation commands, supporting more platforms and offline installation
• Adjusted DataKit configuration; some collectors have field changes; it is not recommended to copy the original conf.d directory to new collectors
• Reorganized DataKit configuration file directory structure and default installation path

DataWay¶

• DataWay added interfaces for writing log data and object data, and adjusted the event data write interface

May 7, 2020¶

DataFlux Studio¶

• Added scatter plot chart to display data distribution
• Metric Browser added "configure automatic aggregation rule" for Measurements, supporting automatic downsampling of collected metrics according to set rules
• Removed dataset mode from views, optimized metric browser interaction, and added table mode when browsing metrics
• Optimized integration component display
• Fixed bug where users were automatically logged out after a period of time, and bug where tag filter conditions were not saved in some cases when creating new queries, and several other bugs, improving product usability

DataFlux Admin Console¶

• Workspace added data warehouse configuration, supporting specifying measurements to sync data to the data warehouse
• Workspace supports enabling/disabling automatic metric aggregation for measurements

DataFlux f(x)¶

• "DataFlux.f(x) Quick Start Guide" updated to the latest version
• Authorization link GET access supports flat form (?kwargs_key=value) and simplified form (?key=value)

April 29, 2020¶

Data Collection¶

• DataKit added 6 new data sources: Oracle monitoring metrics, CoreDNS, uWSGI, Traefik, Solr, and Alibaba Cloud RDS slow query logs. Added monitoring metric collection for Express projects.
• Added four new view templates: Self-built MySQL Monitoring, Alibaba Cloud PolarDB MySQL Monitoring, Alibaba Cloud PolarDB PostgreSQL Monitoring, and Alibaba Cloud RDS for PostgreSQL Monitoring; optimized two view templates: MySQL Monitoring, PostgreSQL Monitoring

April 21, 2020¶

DataFlux Studio¶

• Added a new chart type: Custom Chart, supporting drawing charts based on custom background images
• Basic charts added "auto-align time interval" feature. The system calculates the time interval based on the selected time range and aggregation interval, then dynamically adjusts the query upward to the nearest preset auto-align time interval (including 1 minute, 5 minutes, 10 minutes, 30 minutes, 1 hour, etc.)
• Charts added "fit to view" feature. When there is only one chart in the view, selecting "Fill View" makes the chart automatically fit the entire view window
• When adding nodes, derivative nodes support specifying measurements and node icons
• Adjusted the classification and names of the left navigation bar
• Basic settings support viewing the version of the current workspace
• Adjusted the DataWay interface for the SaaS version
• Fixed several bugs, improving system stability

DataKit (v1.0.0-109-g13b2ced)¶

• Added data collection support for six new data sources: timezone, StatsD, SSH, lighttpd, etcd, and Alibaba Cloud pricing
• Supports CentOS 6.7 and Ubuntu 14.04 and above
• DataKit supports cascading and Lua script processing

April 14, 2020¶

DataFlux Studio¶

• DataFlux online added product version restrictions; all new workspaces are set to Free Plan by default (500 time series, 1-day data retention, 1 trigger rule, 7-day trigger rule validity)
• Restructured and optimized the underlying logic of triggers; trigger rules now support custom messages for data gap alerts; added advanced detection, supporting custom development of advanced detection functions on the DataFlux f(x) platform
• Optimized chart loading speed and SQL queries
• Fixed chart alias and color display errors
• Fixed issue where measurements could not be deleted
• Fixed issue where data could not be queried if the measurement name contained a dot

DataFlux Admin Console¶

• Adjusted "add workspace" logic, removed "select existing database" and "set data permissions" features; all new workspaces now "automatically create a new database"
• New workspaces support setting "data retention duration" and "trigger history retention duration"

DataFlux.f(x)¶

• Added online update functionality for official script library; official scripts no longer need to be exported and imported
• Official script library added simple detection support (requires installing the latest official library)
• Editor uses strict monospace font to ensure 1 Chinese character equals 2 English characters in width
• Fixed known issues and improved system stability

DataWay (V1.0.1-401-ga917b59)¶

• Lua module added connection pooling for MongoDB/Redis/MySQL (enabled by default)
• Removed correctness validation for metrics such as $flow_ and $alert
• DataWay added data upload security authentication mechanism; all deployed DataWay instances must be upgraded, otherwise they cannot continue to be used

April 3, 2020¶

• Line chart, bar chart, and pie chart "Settings" support setting colors, and support setting colors for different groups after grouping
• Chart settings support custom metric units
• Fixed Safari compatibility issues
• Optimized sidebar navigation

March 31, 2020¶

DataFlux Studio¶

• Help Center added search functionality, supporting quick keyword location
• Scenario nodes added sharing functionality, supporting viewing sharing history, editing sharing, canceling sharing, and viewing QR codes
• Scenario nodes added search functionality, supporting viewing search history
• Adjusted baseline functionality, moving the "baseline" from "Query" and "Anomaly Detection" to the settings of "Line Chart" and "Bar Chart"
• Metric Browser supports viewing the "data type" of a Measurement
• Added 37 new view templates for IT operations scenarios
• Adjusted chart query rules and UI style, supporting selection of multiple fields from one Measurement, and supporting nested aggregation functions
• Optimized scenario node tree, recording expanded state selection; fixed selection issues
• Optimized left navigation bar, supporting collapse display to increase chart visualization area
• Optimized "Custom SQL" in metric queries, supporting return point count and time limits
• Optimized slow query issues for tag keys, tag values, measurements, and field keys

DataFlux Mobile¶

• Added "Service Agreement" confirmation; users must agree before using the app
• Added node import feature, supporting direct viewing of node charts

Data Collection¶

• Added data collection support for four new data sources: PostgreSQL, MongoDB, network packet capture (pcap), and Alibaba Cloud CDN

March 16, 2020¶

DataFlux Studio¶

• Supports users leaving a workspace; supports admins dissolving a workspace
• In chart analysis mode, line charts support adjusting query conditions; key events support time-aggregated display; key events support list display
• Added year-over-year comparison for summary charts, line charts, and bar charts
• Added two new chart types: China Map and World Map
• In new charts, aliases support formatted display, including "show only group name", "show only metric name", etc.
• When creating child nodes, allow not inheriting parent node tag filters
• Supports locking the chart time range, independent of the global time range
• Optimized table chart, supporting grouped display in query mode
• DataFlux now supports Warehouse functionality
• Fixed incorrect data queries in pie chart, table chart aggregate mode, gauge, and bar chart grouped mode
• Optimized integration component filtering method
• Fixed bug where trigger rules sometimes did not generate trigger history

DataFlux Admin Console¶

• Adding a workspace supports both "automatically create a new database" and "select an existing database"
• Optimized default workspace name, unified as [username]'s workspace

DataFlux.f(x)¶

• "Simple Debug Panel" now supports browsing the workspace list via built-in DataWay
• "Script List" added "Quick View Panel", allowing viewing another script code while editing a script
• "DataWay Operation Object" now supports specifying a token for data writing
• "Script Editor" no longer allows multiple windows to edit the same script
• "@DFF.API" now supports the category='check' option, used to mark detection functions
• Fixed known issues and improved system stability

Data Collection¶

• Added 5 new DataWay APIs: /v1/config, /v1/reload, /v1/lua, /v1/lua/list, /v1/license
• Optimized /v1/write/metrics API functionality
• DataWay upgraded to support merging old and new configuration files, including parameters that can be passed via command line: remote_host, collect_second, lua_worker, access_key, secret_key
• DataWay installation added two new parameter supports: DW_ENABLE_CONFIG_API and DW_CONFIG_PASSWORD, for enabling the config API and setting the config API password. Once the config API is enabled, installation will fail if no password is specified.
• Adjusted DataWay cache cleanup behavior: limited single cleanup count, increased cleanup frequency
• DataWay Lua script related changes
• All Lua files are now placed uniformly under < DataWay-安装目录 >/lua; all Lua configurations from older versions must be manually moved to this directory, and route configurations may need to be updated accordingly
• Absolute path Lua configurations are no longer supported; if configured in older versions, manual handling is required
• DataKit allows specifying a token via the X-Token header to send data to a public DataWay

March 13, 2020¶

Data Collection¶

• Added DataWay Android SDK, DataWay iOS SDK, DataWay Python SDK, DataWay JavaScript SDK

March 11, 2020¶

Data Collection¶

• DataKit (v1.0.0-44-gaa4a656) data sources added support for Azure Monitor, Zabbix, Prometheus Exporter, NSQ monitoring metrics, network port scanning, Kubernetes monitoring data, GitHub, and Java middleware (Cassandra, Hadoop-HDFS, Java JVM, JBoss, Kafka, Tomcat, Weblogic, BitBucket) monitoring metric collection
• DataWay added support for Logstash and Prometheus data collectors
• Released CSV data collector
• Added support for Oracle data collection

March 5, 2020¶

DataFlux Studio¶

• Measurements support deletion

DataFlux Admin Console¶

• Admin console added password strength requirements for admin accounts and member accounts

February 28, 2020¶

DataFlux Studio¶

• Chart query added custom function mode, supporting development of custom query functions using DataFlux.f(x)
• View variables added sorting functionality, supporting sorting and previewing variable values when creating and modifying view variables
• Optimized SQL mode, supporting direct mapping of UI mode to SQL mode, added execute button and error feedback
• Optimized function interaction experience in chart queries, added function description tooltips
• Optimized summary chart value selection, supporting selection of aggregation algorithm
• Fixed issue where table chart displayed incorrectly when null data existed
• Fixed issue where bar chart top sorting was incorrect

DataFlux Admin Console¶

• Admin console accounts added role functionality, divided into admin and developer roles; developers can only log in to the DataFlux.f(x) platform

DataFlux.f(x)¶

• Added data source structure browser
• Code editor supports switching between published, editing, and DIFF views
• UI/API enhanced asynchronous call support
• Added query function support (category=query)
• Updated built-in official script package
• Fixed known issues and improved system stability

Data Collection¶

• DataKit added Alibaba Cloud Log Service (SLS) data collection, Alibaba Cloud ActionTrail data collection, Kafka Consumer data collection, route tracing metric collection, Kubernetes monitoring metrics, Kube Inventory monitoring metric collection, and HTTP performance metric collection
• Added Oracle data collection
• DataWay compatible with DataX data synchronization tool

February 20, 2020¶

DataFlux Studio¶

• Added command component, supporting creation and configuration of command components in charts
• Optimized the interaction method for adding charts and chart classification, including basic charts and advanced charts
• Added PNG image export functionality for charts
• Summary chart added mapping functionality
• Integration added scenario category filtering
• When adding or modifying scenarios, support setting the data range for the scenario
• Optimized chart data display, all float type data retains two decimal places
• Support for adding custom icons to nodes
• Trigger actions in trigger rules added support for action function triggers
• Added DataFlux EBA Agent Builder, which can package and generate dedicated collectors for enterprise behavior analysis scenarios
• System deployment supports configuring Continuous Query rules to downsample metrics

DataKit¶

• Added support for agent configuration
• Fixed related bugs in Alibaba Cloud cost collection
• Optimized collection and reporting performance

February 15, 2020¶

DataFlux Studio¶

• Fixed occasional white screen issue in views
• Fixed issue where prediction function could not accept custom parameters in analysis mode
• Fixed issue where bar chart could not be displayed in SQL mode
• Fixed issue where scenario QR codes generated in private deployment versions could not be scanned and added on mobile devices
• Fixed issues with table chart: previous/next page buttons not clickable, returning to first page after timed refresh, duplicate data being hidden, and pagination bar not being displayed when data is empty after pagination

DataFlux.f(x)¶

• Script editor added read-only mode; editing is only possible after clicking "Start Editing"
• Fixed issue where API interface failed when script function returned non-standard JSON
• Fixed copy and display issues, added page tips

February 10, 2020¶

• Added scenario templates; creating scenarios supports template-based creation, and created scenarios can be exported as templates
• Scenarios support sharing; after sharing, they can be viewed on mobile devices via the DataFlux app or mini program
• Views support adding flowcharts, videos, and real-time video streams
• Added chart combination functionality: bar chart combined with line chart, summary chart combined with line chart
• Chart query transformation functions support selecting custom transformation functions developed via DataFlux f(x)
• Released DataFlux mobile app; after publishing a scenario, it can be viewed by scanning a QR code in the app
• Bar chart supports TopList display mode
• Optimized view display page, supporting dynamic loading of charts
• Optimized DataWay management interface monitoring views
• Released DataWay API documentation and SDK
• Measurements support multiple RP (retention policy)

January 10, 2020¶

• Added Telegraf collector configuration documentation
• DataKit collector added support for snmp, syslog, exec, varnish, NSQ Consumer collection source configurations
• Optimized page loading performance when query returns too many data points
• Optimized configuration logic when setting trigger rules for string-type metrics: removed unsupported aggregation functions for strings; optimized trigger condition settings for string count aggregation
• Fixed issue where pinning a scenario to the dashboard resulted in incorrect navigation
• Fixed misalignment issue when multiple queries exist in a bar chart
• Fixed issue where data was missing after deleting a custom expression when creating a new query