Auraa and Genie Code
Better Together: An Agentic Data Platform Meets Databricks’ AI Coding Agent
Genie Code writes SQL and Python, builds pipelines, and debugs failures. But governance, multi-tenant isolation, audit trails, and reproducible pipelines are platform responsibilities, not agent responsibilities.
Key takeaways
Active governance
Drift detection across tenant catalogs, reconciled against declared policy.
166 ops, 3 tool slots
The three-meta-tool pattern, with authorization on every invocation.
No curation tax
Catalog metadata is populated at provisioning, not left as manual work.
Auditable by design
Metadata-driven specs in Delta, not a log of code executions.
About this white paper
Where Genie Code stops, Auraa starts
This paper documents how the two compose. Genie Code handles the code. Auraa handles the configuration, governance, and organizational context that makes that code trustworthy at scale, including the curation tax, the three-meta-tool pattern, and active drift detection across tenant catalogs.
- Written by the team that built and operates the platform
- Includes the trade-offs we accepted, not only the wins
Read the architecture before you build it.
Twenty minutes of reading that can save a migration. No form, no gate, straight to the PDF.
Questions
Questions architects ask us
The curation tax is the performance penalty Genie Code pays when the Unity Catalog metadata it relies on for context is incomplete, stale, or inconsistently documented. Genie Code uses catalog metadata (table descriptions, column annotations, relationship documentation) to understand what data exists and what it means. When that metadata is missing or wrong, the quality of generated code degrades proportionally. Auraa eliminates the curation tax structurally at provisioning time: when a new tenant or data source is onboarded, Auraa's agents populate the required catalog metadata as part of the provisioning workflow, rather than leaving it as a manual task for data engineers.
Generated code (SQL, Python, Spark) is an output. When compliance teams need to audit a pipeline, they must read and interpret the code to understand what transformation was applied, under what quality rules, to which tenant's data. Metadata-driven pipeline specs stored in Delta Lake express those decisions as structured, versioned records: what source, what transformation contract, which quality thresholds, which tenant scope. The audit trail is the metadata itself, not a log of code executions. This gives compliance teams a reproducible, human-readable record without requiring them to interpret generated code.
MCP (Model Context Protocol) agents like Genie Code consume tool slots from a fixed budget per session. Exposing 166 platform operations as 166 individual MCP tools would exhaust that budget immediately. The three-meta-tool pattern packages all 166 operations into 3 meta-tools: one that discovers which operations exist and what they require, one that invokes a named operation, and one that checks authorization before invocation. Genie Code uses the discovery tool to find the operation it needs, confirms authorization, then invokes it, accessing the full Auraa platform capability without consuming tool budget on individual operation definitions.
Workflow prompts and domain skills are structured knowledge packages that Auraa delivers to any MCP-compatible agent, including Genie Code. Workflow prompts encode Auraa's platform knowledge as reusable prompt templates, for example the correct sequence of operations to onboard a new data source, or the standard quality validation workflow for a silver layer promotion. Domain skills encode data engineering best practices specific to Auraa's architecture. Both transfer platform knowledge to the agent without consuming MCP tool budget, because they are delivered as prompt context rather than as tool definitions.
Active governance means Auraa continuously verifies that the permissions actually applied in Unity Catalog match the policy declared in Auraa's grant policy registries. Drift detection runs across tenant catalogs and flags any divergence: a permission that was granted outside Auraa's provisioning workflow, a catalog structure that does not match the declared isolation strategy, or a Unity Catalog role that has accumulated permissions beyond its declared scope. The reconciliation workflow then identifies the corrective actions needed to realign actual permissions with declared policy.