Loads Kafka and gRPC streams into ClickHouse in large batched inserts.
-
Updated
Nov 21, 2026 - Rust
Loads Kafka and gRPC streams into ClickHouse in large batched inserts.
The template a DFE deployment's own git repo is seeded from.
Runs a Vector pipeline as a supervised transform stage in a DFE deployment.
Writes a DFE deployment's streams to compressed files in a local path or object store.
The whole DFE stack on one host with Docker Compose, from pinned release images or local builds.
Defines the core ClickHouse objects and Kafka bootstrap topics a DFE deployment creates.
The DFE control plane: an API and CLI that turn operator changes into git commits a deployment runs from.
Pulls data from cloud and SaaS APIs, databases and files into a DFE deployment.
Runs your VRL transforms over a DFE deployment's event stream, in-process.
Turns Beats and Elastic Agent events into DFE-normalised events.
The DFE console, where an operator sets up and runs a deployment.
DFE fork of HyperDX extended with ported and merged DFE 2.1 Discovery UI code, JSON type capabilities and dfe-ui integration. rerere for added functionality, security fixes post-applied.
The front door of a DFE deployment: takes events in the protocols agents already speak and hands them on as JSON.
Helm charts, Argo CD ApplicationSets and OpenTofu that stand up and test a DFE deployment on Kubernetes.
To associate your repository with the data-fusion-engine topic, visit your repo's landing page and select "manage topics."