CADENZA Compiles NL Queries Into Operator DAGs That Beat SOTA 165x in Latency

165.7x faster and 310.3x cheaper — with a quality bump of +0.49 — is what CADENZA achieves over state-of-the-art semantic query processing engines on the SemBench benchmark.

Why Existing SQPE Optimizers Leave Performance on the Table

Semantic query processing engines bolt model inference onto relational queries. That means each semantic operator — think entity extraction, classification, embedding — runs an ML model over unstructured data. Model inference dominates latency and cost, and outputs are stochastic and backend-dependent.

Current optimizers treat each semantic operator as an opaque black box. They don't expose intermediate task outputs as relational optimization objects, so you can't filter, reorder, route, threshold, or jointly tune them. That's a huge blind spot when you're trying to balance quality, latency, and cost across a query plan.

CADENZA's Plan-Space Compilation

CADENZA compiles each semantic operator instance — a template bound to a natural‑language intent — into an intent‑specific plan space of typed task DAGs. The key innovation is task‑extended relational algebra (TxRA), a conservative extension of relational algebra with task‑specific operators.

The logical planner synthesizes seed TxRA plans, applies structural rewrites (safety‑checked via operator dependencies), and enumerates semantics‑guided alternatives from alternative‑generation templates. The physical planner then compiles each task‑specific operator into a router over heterogeneous backends. It jointly tunes routing cutpoints, backend parameters, and relational thresholds using Bayesian optimization.

Real Benchmark Numbers That Hurt Competing Approaches

On SemBench, CADENZA improved scenario‑level averages across quality, latency, and cost by +0.49, 165.7x, and 310.3x respectively relative to the previous state of the art. That's not a marginal gain — it's crossing the threshold where semantic query processing becomes practical for latency‑sensitive and cost‑conscious production workloads.

Turning a query that took minutes and cost dollars into seconds and cents changes what problems you even think about solving with semantic operators.

Source: CADENZA: Compiling Natural-Language Intent into Task-Specific Operator DAGs for Semantic Query Processing
Domain: arxiv.org