Changelog¶
1.22.2 (2026/11/10)¶
New Features¶
- Added optional tail sampling spill. When the memory budget is insufficient, groups pending decision are spilled to disk. This supports resource limits, usage and error metrics, and Sinker routing by point labels (!234, !235).
- Added spill configuration, capacity estimation, Kubernetes storage deployment, and troubleshooting guides (!236).
Bug Fixes¶
- Fixed an issue in legacy raw tail sampling where, after recompression of small packets, data could be received successfully but not delivered downstream (!235).
- Fixed an issue in tail sampling spill multi-target routing where continuous retries to failed targets blocked other targets in the same group; successful targets are not sent repeatedly (#138).
1.22.1 (2026/11/10)¶
New Features¶
- Added metrics for tail sampling stage duration, send resident bytes and budget, and bytes before and after compression (#135).
Feature Optimization¶
- Optimized tail sampling batch sending and buffer reuse, reducing duplicate allocations and copies during decompression, batching, and sending (#133).
- Reduced request processing overhead when HTTP access logs are disabled, applicable to the
gin_log: ""configuration (#133).
Compatibility Notes¶
- Downstream requests for direct tail sampling and Sinker routing now default to zstd instead of gzip, with no automatic negotiation or fallback. Before upgrading, ensure that all downstream KodoX Point Workers support zstd; cache replay preserves the original encoding and request body (#133).
1.22.0 (2026/11/09)¶
Bug Fixes¶
- Upgraded Go to 1.26.10 to fix a certificate trust security issue in crypto/x509 (CVE-2026-111943 / GO-2026-6809) (#130).
Feature Optimization¶
- Optimized tracing tail sampling performance when processing multiple shards of the same trace (#129).
Compatibility Notes¶
- The base OS of the main image was upgraded from Ubuntu 22.04 to 24.04, supporting linux/amd64 and linux/arm64 (#131).
1.21.0 (2026/10/14)¶
New Features¶
- Added OpenLineage data ingestion, gzip forwarding, and Sinker routing (#112, #122).
- Added the OCSF receiving endpoint, which supports batch data and writes to the
ocsfLog Index (#120).
Bug Fixes¶
- Fixed an issue where invalid aggregation parameters could cause process exceptions (#117).
- Fixed an issue where stalled uploads occupied concurrency slots for extended periods and proxy response status codes were incorrect (#118).
Feature Optimization¶
- Improved HTTP timeout control to prevent stalled requests from occupying resources for extended periods (#118).
- Standardized token redaction in logs, retaining the first 8 and last 4 characters and fully hiding short tokens (#121).
Compatibility Notes¶
- Downstream 4xx responses retain their original status codes. If the error body cannot be parsed,
{}is returned instead of500(#124).
1.20.0 (2026/09/10)¶
New Features¶
- Added an HTTP concurrency limit before reading the Body, reusing
kodo_queue.workers(DW_KODO_QUEUE_WORKERS, default256). After the limit is reached, at most128additional requests can wait, for up to1s. The limit remains effective after the Kodo dispatch queue is disabled, and requests without a Body are not counted (#115). - Added the
dataway_http_concurrency_in_flight,dataway_http_concurrency_waiting,dataway_http_concurrency_requests_total, anddataway_http_concurrency_wait_secondsmetrics to observe in-flight requests, waiting requests, admission results, and wait duration (#115).
Bug Fixes¶
- Fixed a compatibility issue with Langfuse SDK 4.x when
/api/public/ingestionsucceeds upstream but the response body is empty: returnsuccesses/errorsJSON by event ID in the request batch and preserve existing JSON response content from upstream (#116).
Compatibility Notes¶
- When the HTTP concurrency wait queue is full or the wait times out, registered DK-compatible endpoints return
503and other endpoints return429, all withRetry-After: 1; these rejected requests do not enter the disk cache (#115). - When an HTTP/1.x request is rejected by the concurrency limit, terminate reading the remaining Body and close the connection after returning the response, avoiding waiting for unfinished uploads; clients need to establish a new connection when retrying (#115).
1.19.0 (2026/08/27)¶
New Features¶
- Added the
/v2/tail_samplingzstd reporting endpoint and compatibility with zstd packets sent by DataKit 2.10 via/v1/tail_samplingwithout a compression negotiation header (#110). - Added bounded batch sending and failure retries for tail-sampled retained data, changed memory overflow spill-to-disk to asynchronous processing, and improved restart recovery and shutdown cleanup mechanisms (#110).
Feature Optimization¶
- Tail sampling extracts
trace_idafter decompression and recalculates predicates; it uses gzip compression when batch sending downstream, reducing request count and transfer overhead (#110). - Improved recording coverage for runtime metrics such as tail sampling queue submission results, wait duration, disk depth, and batch status (#110).
Security and Observability¶
- Set token labels in metrics to
redacted, no longer retaining original token fragments, to avoid credential leaks and high-cardinality time series (#90). - Redacted errors from aggregation, tail sampling, and multi-backend forwarding to no longer expose backend URLs, response content, or authentication information (#90).
1.18.0 (2026/08/12)¶
Feature Optimization¶
- Optimized the
trace_idextraction path for tail sampling to avoid repeated payload traversal whenRawGroupIdis available while keeping sampling behavior unchanged (#108). - Upgraded
cliutilsto automatically initialize capacity metrics after the disk cache is opened, and improved close and file lock release behavior (#111).
Bug Fixes¶
- Unified token validation and propagation semantics for special Aggregate and Tail Sampling APIs (#88).
- Isolated the invalid token negative cache by token, write route, and main/headless mode to avoid cross-scenario false blocking (#106).
- Fixed concurrent access issues in the failure cache and tail sampling disk degradation paths, avoiding data races during disk queue initialization, reads, and shutdown (#111).
- Isolated pprof and Prometheus HTTP mux; pprof is disabled by default, and when enabled, it can only listen on loopback, with limits on diagnostic request duration and concurrency (#89).
- Redacted tokens, credentials, global tags, and rule content in request forwarding, cache replay, aggregation, tail sampling, and Sinker logs (#104).
- Fixed system Workspace URL determination and added regression verification for token injection in normal and cascading modes (#104).
- Corrected main HTTP Server request header timeout, Gin mode, IP blocklist configuration examples, and API wrapper error handling (#104).
Security and Observability¶
- Pinned the Go and golangci-lint download packages used by CI, verified SHA-256 by architecture, and removed the direct execution path for remote installation scripts (#101).
- Updated the default DataWay dashboard, adding queue, tail sampling, Sinker cache, and runtime monitoring (#85).
1.17.0 (2026/07/14)¶
New Features¶
- Added OTLP HTTP/protobuf endpoints:
/otel/v1/traces,/otel/v1/metrics, and/otel/v1/logs(#81). - OTLP requests support query
token,X-Token, and Bearer token, and are integrated with the Kodo queue and disk cache (#81). - OTLP requests support pass-through of Kodo status codes, body, and required headers (#81).
Feature Optimization¶
- Optimized Sinker routing for aggregation and tail sampling so that a failure in one group no longer affects other groups (#71).
- The invalid token negative cache applies only to eligible write requests (#77).
- Optimized aggregation memory usage and concurrency stability.
- Optimized tail sampling backlog handling; when the memory queue is full, data can be temporarily stored to disk.
- Improved stability in disk cache exception scenarios.
Bug Fixes¶
- Fixed race and capacity issues in the Sinker rule cache (#93).
Observability¶
- Added file descriptor metrics
dataway_open_files_by_typeand TCP metricsdataway_tcp_connections. - Added tail sampling, send queue, worker, and disk staging metrics.
1.16.1 (2026/07/09)¶
Bug Fixes¶
/v1/write/rumrejects empty bodies and returns400anddataway.emptyBody; non-empty bodies are still forwarded as-is (#82).- When the Kodo queue is full, cacheable requests are written to disk; when the cache is unavailable,
503is returned (#83). - Return "cached" only after a successful write, and improved recording of cache failure reasons (#83).
1.16.0 (2026/07/02)¶
New Features¶
- Added IPv6 and dual-stack deployment support; the IP allowlist supports IPv4, IPv6, and CIDR (#79).
- Added
bind_ipv6/DW_BIND_IPV6andtrusted_proxies/DW_TRUSTED_PROXIESconfigurations (#78, #79).
Feature Optimization¶
- When the Kodo success response is empty, no longer return an empty body and
Content-Type(#78). - Return CORS headers only for requests with
Origin, and addX-Content-Type-Options: nosniffto forwarded responses (#78).
1.15.0 (2026/05/27)¶
New Features¶
- Added a bounded Kodo queue covering write, upload, and Firehose requests.
- The queue is enabled by default and supports
DW_KODO_QUEUE_*environment variables. - When the queue is full, cacheable requests are cached. When the cache is unavailable,
503is returned. - Added an invalid token negative cache, supporting
DW_TOKEN_NEGATIVE_CACHE_*environment variables. - The negative cache has a default TTL of
5mand stores up to1000tokens.
Feature Optimization¶
- Added token and payload validation to the aggregation endpoint.
- Aggregation sending now uses a background queue, supports 3 retries, and adds
Content-MD5. - The aggregation proxy reuses the reverse proxy and validates related configuration.
- Cache replay uses the current
remote_host, and the default cleanup interval is changed to1s. - Updated Sinker documentation. Starting from DataKit 2.0.0, non-point write APIs generally do not require
__dataway_apirules.
Observability¶
- Added Kodo queue metrics:
dataway_kodo_queue_wait_seconds,dataway_kodo_queue_depth,dataway_kodo_queue_bytes,dataway_kodo_queue_enqueued_total,dataway_kodo_queue_full_total, anddataway_kodo_queue_dispatch_total. - Added invalid token negative cache metrics:
dataway_token_negative_cache_added_totalanddataway_token_negative_cache_blocked_total. - Added aggregation/tail sampling metrics:
dataway_http_api_body_size_bytes_total,dataway_http_aggr_point_total,dataway_http_tail_sampling_trace_total,dataway_http_tail_sampling_span_total, anddataway_http_tail_sampling_packet_send_total. - Added disk cache lock contention metrics:
diskcache_lock_wait_secondsanddiskcache_lock_contention_total. - Fixed the
sinkedlabel of latency metrics in Sinker scenarios.
1.14.0 (2026/04/17)¶
- Added a Session Replay image processing endpoint.
1.13.0 (2026/04/10)¶
- Added support for bypass aggregation and sampling in DataWay (#67).
1.12.3 (2026/03/26)¶
- Added v2 Sinker header format to resolve HTTP standard violations caused by abnormal values in routing headers (#68).
1.12.2 (2026/03/17)¶
- Updated the SSL certificates in the DataWay image to avoid failure to recognize Openaway SSL certificates in cascading scenarios (#69).
1.12.1 (2025/12/25)¶
- Fixed an issue in version 1.12.0 where Sinker routing could not copy requests (#66).
1.12.0 (2025/12/19)¶
New Features¶
-
Optimized the Sinker caching mechanism (#34).
- The new sinker caching mechanism caches by a 16-byte request hash signature instead of caching the complete request signature. It uses dual hashing to avoid hash collisions as much as possible; the theoretical collision probability is n/2^128, where n is the number of cached keys.
- Added TTL and capacity mechanisms to the Sinker cache to further limit memory usage: TTL cleans up inactive cache entries, and capacity ensures that the number of keys in the cache does not exceed the specified limit. The TTL and capacity limits further reduce the hash collision probability described above.
-
Added configuration to optimize DataWay HTTP header responses and reduce DataWay public network traffic (#65).
1.11.2 (2025/11/25)¶
- Optimized DataWay forwarding of Kodo HTTP requests (#64).
1.11.1 (2025/11/15)¶
- Fixed invalid
__internal__token issue (#63).
1.11.0 (2025/11/12)¶
New Features¶
- Upgraded the DataWay base image to Ubuntu 22.04 (#62).
- DataWay supports blocking invalid traffic (#61).
Bug Fixes¶
- Fixed an issue where Session Replay and profile routing could not copy data to different Workspaces (#59).
1.10.0 (2025/09/03)¶
New Features¶
- Added Langfuse data ingestion (#57).
1.9.0 (2025/04/25)¶
New Features¶
- Added
X-Pkg-Idto data upload requests in DataWay for data tracing (#53). - Improved application logs; more error codes returned by Kodo are passed through to DataKit (#52).
- Added environment variables for API self-pull (#55).
- Updated disk cache code to diagnose
invalid argumenterrors that occur in some cases (#56).
1.8.0 (2025/02/19)¶
New Features¶
- When Sinker mode is enabled, DataWay supports processing data reporting requests sent by clients (such as DataKit/Function) at the same time (#50).
1.7.0 (2024/12/18)¶
New Features¶
- Added the AWS Firehose ingestion entry point (#47).
- Added support for passing the token via
X-Token(#43).
Feature Optimization¶
- Adjusted default parameters for internal recycling to optimize memory usage (#49).
- Optimized the default YAML in Kubernetes deployment and adjusted its cache disk mount configuration (#48).
- Adjusted and added some internal metrics and updated the DataWay default dashboard.
1.6.2 (2024/12/03)¶
Feature Optimization¶
- Added a connection configuration entry point at the HTTP layer (#46).
1.6.1 (2024/11/19)¶
Feature Optimization¶
- Added an option to disable the 404 page (#44).
1.6.0 (2024/09/19)¶
Feature Optimization¶
- Optimized environment variable configuration related to disk cache.
- Exposed more self metrics and optimized memory usage at the HTTP forwarding layer (#39).
- DataWay supports configuring its own SNI (#42).
- Added a time synchronization endpoint to help DataKit obtain a more accurate Unix timestamp (#40).
- Sinker:
- Filter conditions support
nilchecks, i.e., determining whether a specific field exists (#41). - Added default rule settings, so requests that do not match existing routing rules are routed to the Workspace corresponding to the default rule (#30).
- Filter conditions support
1.5.0 (2024/07/05)¶
Feature Optimization¶
- Optimized the installation script (#33).
Compatibility Adjustment¶
- The new version removes direct Sinker configuration support in Host installation mode. This feature will be supported later through a new approach.
1.4.1 (2024/06/19)¶
- Fixed missing
DW_ENABLE_TLSin Kubernetes installation mode. - Changed the build image address.
1.4.0 (2024/05/15)¶
- Added DataKit metering endpoint (#29).
- Fixed potential data loss in cache (#31).
- Added support for direct HTTP TLS certificate configuration on the DataWay side (#32).
- Exposed more metrics.
1.3.9 (2024/03/28)¶
- When Sink drops requests, added HTTP
406 Not Acceptableerror responses to facilitate troubleshooting (#82).
1.3.8 (2024/01/24)¶
- Exposed more metrics.
- Significantly improved Sinker rule matching performance (#26).
1.3.7 (2023/12/14)¶
- Exposed more metrics.
- Optimized the disk cache cleanup policy based on the principle of consuming as soon as possible:
- Avoid actively dropping data due to storage overrun.
- Send cached data as early as possible.
- Added observability for dropping oversized requests (metrics/logs).
- Adjusted DataWay's own Pod YAML and improved its metric and log collection.
1.3.6 (2023/10/24)¶
- Added DataWay
sinkmanagement command.
1.3.5 (2023/09/13)¶
- Added support for installing DataWay in Kubernetes by injecting environment variables.
- Newly released dataway.yaml.
1.3.4 (2023/09/01)¶
- Upgraded Golang to 1.9.
- Fixed the no-data issue with inner token.
- Adjusted the following default configurations:
- Default API rate limit increased to 100K.
- Default maximum Body increased to 64MB.
- Added HTTP timeout setting (DataWay to central service), default 30s.
- Added configuration to ignore TLS 509 errors.
1.3.3 (2023/09/01)¶
- Fixed
curlissue in image packaging.
1.3.2 (2023/08/30)¶
- Added the
curlcommand to the image to facilitate viewing self metrics.
1.3.1 (2023/08/28)¶
- Added Sinker functionality for URLs.
1.3.0 (2023/08/24)¶
- Refactored the entire DataWay implementation and added Sinker functionality.
1.2.12 (2023/08/28)¶
- Refactored the CI release process and implemented dual publishing of the install script.
1.2.8 (2022/09/18)¶
-
/v1/wirteendpoint updates:- Removed redundant
/v1/write/xxxAPI and added/v1/write/:categoryendpoint. - Write data no longer performs line protocol decoding, but Body is still read for signing.
- Removed redundant
-
Disk cache:
- When sending to the central service fails during cache cleanup, added a consumption pause policy so that the next cache is not cleaned until the current cached request is sent successfully.
- Updated the pbcache proto file.