DeltaBus

A Lakehouse-Native Event Bus Pattern for Data Platforms

Most Lakehouse platforms bolt on a separate message queue. Another cluster, another billing line, and events that live outside your Unity Catalog boundary. Databricks already has everything a production event bus needs.

Read the white paper

Key takeaways

Governed by default

Events stay inside Unity Catalog. No bus-shaped hole in your audit trail.

CDF as the event log

Checkpoint-based reads give at-least-once delivery and queryable history.

Sub-10-second delivery

ZeroBus SDK confirms fast, and buffers in memory when the bus is unreachable.

~$50 a month

Against $2,000 to $15,000 for a comparable managed queue service.

About this white paper

The bus you already own

DeltaBus is the messaging backbone powering every component of Auraa: tenant provisioning, ingestion orchestration, data quality, and agent-to-agent communication. It runs in production on Databricks with no external dependencies, using Delta's append-only storage, Change Data Feed, and Unity Catalog governance.

Written by the team that built and operates the platform
Includes the trade-offs we accepted, not only the wins

Read the architecture before you build it.

Twenty minutes of reading that can save a migration. No form, no gate, straight to the PDF.

Read the white paper Book a demo

External message buses introduce a separate cluster to manage, a separate billing line, and events that flow completely outside the Unity Catalog governance boundary. For platform operations workloads (tenant provisioning, ingestion orchestration, data quality notifications, agent-to-agent communication), all of those events need to be governed, auditable, and queryable alongside the data they affect. Routing them through an external bus breaks that continuity. The Lakehouse already has append-only storage, Change Data Feed, and Unity Catalog governance: everything a production event bus needs.

DeltaBus enables Change Data Feed on a Delta table and treats it as an event log. Publishers append structured event records. Subscribers poll using checkpoint-based CDF reads, tracking their last-processed Delta version. This gives at-least-once delivery, checkpoint-based deduplication to achieve effectively-once processing semantics, and a permanent queryable history of every event. All of this runs inside the existing Unity Catalog governance boundary with no external coordination service.

The ZeroBus SDK is the publishing interface used by Auraa's platform components. It delivers sub-10-second confirmed event delivery when the bus is available. When DeltaBus is not reachable, ZeroBus automatically falls back to in-memory buffering and replays on reconnection. This fallback is transparent to the caller: the publishing component does not need to handle connectivity failure logic.

DeltaBus setup takes approximately 5 minutes, creating a CDF-enabled Delta table. A comparable managed Kafka cluster or Azure Event Hubs configuration takes 2-6 hours. Operating cost for DeltaBus is approximately $50/month in Delta storage. Comparable managed queue services for the same platform operations workloads cost $2,000-15,000/month. There is no idle compute cost: DeltaBus uses serverless compute that scales to zero when subscribers are not polling.

The whitepaper identifies three trade-offs, each deliberate for platform operations: 5-10 second event latency (unsuitable for real-time financial transactions, appropriate for tenant provisioning and ingestion coordination), per-topic ordering (events within a single Delta table are ordered; cross-topic ordering requires application-level sequencing), and single-workspace scope (DeltaBus operates within one Databricks workspace; cross-workspace messaging still requires an external bus). For Auraa's platform operations workloads, none of these are disqualifying.