Update History¶

1.15.0(2026/05/27)¶

New Features¶

• Added a bounded Kodo dispatch queue for controlling concurrency and back pressure before write/upload requests are sent to Kodo or the next Dataway:
  • Applies to /v1/write/*, /v1/upload/*, and /v1/input/firehose; disk-cache replay requests still use the original synchronous send path so replay traffic does not occupy the queue again
  • Enabled by default with 256 workers, 1024 waiting slots, a 1GB total queued body-byte limit, and a 100ms enqueue timeout
  • Added DW_KODO_QUEUE_ENABLED, DW_KODO_QUEUE_WORKERS, DW_KODO_QUEUE_SIZE, DW_KODO_QUEUE_MAX_BYTES, and DW_KODO_QUEUE_ENQUEUE_TIMEOUT; DW_KODO_QUEUE_MAX_BYTES accepts values such as 1GB and 1024MB
  • Dataway returns 503 when the queue or queued body-byte limit remains full until the enqueue timeout expires
• Added a token negative cache:
  • Temporarily caches Kodo kodo.tokenNotFound and kodo.invalidClientToken responses to reduce repeated Kodo lookups for known invalid tokens
  • Enabled by default with a 5m TTL and up to 1000 cached tokens; configurable through DW_TOKEN_NEGATIVE_CACHE_ENABLED, DW_TOKEN_NEGATIVE_CACHE_TTL, and DW_TOKEN_NEGATIVE_CACHE_MAX_KEYS

Improvements¶

• Hardened aggregate and tail-sampling paths:
  • Aggregate requests now validate tokens and can write the checked token back to both the request query and X-Token
  • Aggregate payloads now reject nil batches, nil points, nil aggregation algorithms, and invalid windows
  • Aggregate sends now use background workers with a queue and 3 retries; aggregate and tail-sampling payloads sent to Kodo now include Content-MD5
  • Aggregate proxy mode now reuses reverse proxy instances and validates backend endpoints, pick keys, and empty targets explicitly
• Improved disk-cache replay:
  • Replay now keeps only the cached path/query and uses the current remote_host scheme and host, preventing stale cached hosts from affecting cascaded or reconfigured deployments
  • Default disk-cache cleanup interval changed from 30s to 1s
  • Added coverage for EOF file switching and reader progress while writers append concurrently
• Updated Sinker documentation: DataKit 2.0.0 and later no longer attach Sinker headers to non-point-write APIs, so these APIs usually no longer need __dataway_api special sink rules; compatibility guidance for older DataKit versions remains documented

Observability¶

• Added Kodo queue metrics: dataway_kodo_queue_wait_seconds, dataway_kodo_queue_depth, dataway_kodo_queue_bytes, dataway_kodo_queue_enqueued_total, dataway_kodo_queue_full_total, and dataway_kodo_queue_dispatch_total
• Added token negative-cache metrics: dataway_token_negative_cache_added_total and dataway_token_negative_cache_blocked_total
• Added aggregate and tail-sampling metrics: dataway_http_api_body_size_bytes_total, dataway_http_aggr_point_total, dataway_http_tail_sampling_trace_total, dataway_http_tail_sampling_span_total, and dataway_http_tail_sampling_packet_send_total
• Added disk-cache lock contention metrics: diskcache_lock_wait_seconds and diskcache_lock_contention_total
• Fixed the sinked label value on Dataway HTTP API latency metrics in Sinker scenarios

1.14.0(2026/04/17)¶

• Add session replay image APIs

1.13.0(2026/04/10)¶

New Features¶

• Add aggregate and sampling for sidecar mode dataway (#67)

1.12.3(2026/03/26)¶

• Sinker header added v2 format (#68)

1.12.2(2026/03/17)¶

• Update DataWay image SSL certificates (#69)

1.12.1(2025/12/25)¶

• Fix sinker failed to copy request in version 1.12.0 (#66)

1.12.0(2025/12/19)¶

New Features¶

• Optimized Sinker caching mechanism (#34)

  • The new mechanism uses 16-bytes hash of requests for caching instead of complete request info. It employs dual hashing to minimize hash collisions, with a theoretical collision probability of n/2^128, where n is the number of cached keys
  • Added TTL and capacity mechanism to manage Sinker caching: TTL mechanism cleans inactive entries, while capacity mechanism ensures the number of keys in cache does not exceed specified limits. Both TTL and capacity restrictions further reduce the probability of hash collisions mentioned above

• Added option to optimize Dataway HTTP response header (#65)

Bug Fixes¶

• Fixed session replay and profile sharding issues preventing data replication to different spaces (#59)

1.11.2(2025/11/25)¶

• Optmize HTTP request send from Dataway to Kodo (#64)

1.11.1(2025/11/15)¶

• Fixed __internal__ token invalidation issue (#63)

1.11.0(2025/11/12)¶

New Features¶

• Upgraded DataWay base image to Ubuntu 22.04 (#62)
• DataWay now supports blocking invalid traffic (#62)

Bug Fixes¶

• Fixed /v1/upload traffic routing issue (#59)

1.10.0(2025/09/03)¶

New Feature¶

• Add support for Langfuse(#57)

1.9.0 (April 25, 2025)¶

New Features¶

• Dataway adds X-Pkg-Id to data upload requests for data tracking (#53).
• Improved program logging: more Kodo error codes will be passed through to DataKit (#52).
• New API for self-managed pulling of environment variables (#55).
• Updated disk cache code to locate invalid argument errors that occur in some cases (#56).

1.8.0(2025/02/19)¶

New Features¶

• When Dataway is in Sinker mode, it supports handling requests sent by clients (such as DataKit/Function) with or without sink header (#50)

1.7.0 (2024/12/18)¶

New Features¶

• Added AWS Firehose integration entry point (#47).
• Supported HTTP header token via X-Token (#43).

Performance Improvements¶

• Update the default configure parameters for buffer usage to optimize memory usage (#49).
• Optimized the default yaml in Kubernetes deployment, update cache disk mount configuration (#48).
• Update and added some internal metrics, updated the default Dataway dashboards.

1.6.2(2024/12/03)¶

Feature Enhancements¶

• Add configures for HTTP connection(#46)

1.6.1(2024/11/19)¶

Feature Enhancements¶

• Add option to disable 404 page(#44)

1.6.0 (2024/09/19)¶

Feature Enhancements¶

• Optimized environment variable configurations related to disk caching.
• Increased exposure of more internal metrics and optimized memory usage at the HTTP forwarding layer (#39).
• Dataway now supports configuring its own SNI (#42).
• Add NTP API for DataKit to get correct Unix timestamp (#40)
• Sinker:
  • Filtering conditions now support nil checks, which means determining whether a specific field exists (#41).
  • Added default rule settings, meaning that requests not matching any existing sinker rules will be directed to the workspace corresponding to this default rule (#30).

1.5.0(2024/07/05)¶

Feature Enhancements¶

• Optimize install script under host installation(#33)

Breaking Changes¶

• Removed Sinker configurations in host-install command. We should manually set sinker related configurations after installing.

1.4.1(2024/06/19)¶

• Fix DW_ENABLE_TLS missing under Kubernetes
• Update docker image

1.4.0(2024/05/15)¶

• Add a new DataKit metering API (#29).
• Fix the issue where data might be lost in the cache (#31).
• Support direct configuration of HTTP TLS certificates on the Dataway side (#32).
• Increase the exposure of additional metrics.

1.3.9(2024/03/28)¶

• When the Sink discards a request, it should return an HTTP 406 Not Acceptable error to facilitate troubleshooting (#82).

1.3.8(2024/01/24)¶

• Export more Prometheus metrics
• Significantly enhance the performance of Sinker rule matching (#26)

1.3.7(2023/12/14)¶

• Export more Prometheus metrics
• Optimize disk cache cleaning strategy, adhere to the principle of collecting and consuming data as soon as possible:
  • Prevent data from being actively discarded due to storage limits being exceeded.
  • At the same time, ensure that cached data is sent out as early as possible.
• Increase observability related to discarding of oversized requests (Metrics/Logs).
• Adjust the Dataway's own POD yaml to improve its metrics and log collection.

1.3.6(2023/10/24)¶

• Added Dataway sink command

1.3.5(2023/09/13)¶

• Dataway installation via environment variable injection is supported under Kubernetes
• Update dataway.yaml

1.3.4(2023/09/01)¶

• Upgrade Golang to 1.9
• Fixed the issue that the inner token has no data
• Adjust the following configuration in the default configuration:
  • Default API throttling increased to 100K
  • The default maximum Body is increased to 64MB
  • Added HTTP timeout setting (Dataway to Center), default 30s
  • Added configuration to ignore TLS 509 errors

1.3.3(2023/09/01)¶

• Fixed image packaging 'curl' issue

1.3.2(2023/08/30)¶

• Added 'curl' command to the image to make it easier to view its own indicators

1.3.1(2023/08/28)¶

• Added Sinker functionality for URLs

1.3.0(2023/08/24)¶

• Refactoring the entire Dataway implementation to add sinker functionality

1.2.12(2023/08/28)¶

• Refactor the CI release process to achieve dual release of install scripts

1.2.8(2022/09/18)¶

• /v1/wirte interface update:

  • Remove redundant /v1/write/xxx APIs and add /v1/write/:category interface
  • Write data is no longer line protocol decoded, but still body read for signature

• Disk cache:

  • When the sending center fails during cache cleanup, increase the consumption pause policy until the current cached request is sent successfully
  • Update the pbcache proto file