Agentic AI and big data: what your platform needs before agents are useful

The pitch for agentic AI on enterprise data is genuinely appealing: instead of a chatbot that answers one question at a time, you get something that plans, queries, checks its own work, calls a second tool, and comes back with a finished piece of analysis — or actually does the thing. Reconcile these two systems. Find out why yesterday’s pipeline was late and open a ticket. Pull the accounts at churn risk and draft the outreach list.

The demos work. What determines whether the same thing survives contact with your actual data estate has very little to do with which model you picked, and almost everything to do with what’s underneath it. Here’s the honest list.

What “agentic” actually changes

A single-shot LLM feature — summarize this, answer that — fails visibly. It returns something wrong, a human reads it, and the human moves on. An agent fails differently: it takes a wrong step early, then builds four more steps on top of it, and the output is a confident, internally consistent artifact that’s wrong in a way nobody can see without redoing the work.

Two properties drive most of the difficulty:

Multi-step means compounding error. If each step in a chain is 90% reliable, a five-step task is roughly 60% reliable end to end. That math is why agents that look brilliant on a rehearsed demo feel flaky on real work — the demo was three steps on clean data, the real task is eight steps on messy data.

Tool use means real consequences. The moment an agent can write to a table, open a ticket, send a message, or trigger a job, “the model hallucinated” stops being an annoyance and becomes an incident. Read-only agents and acting agents are different risk products and should be scoped, reviewed and rolled out differently.

The data platform is the bottleneck, not the model

Every agent that touches your data is doing one of two things: retrieving from documents, or querying tables. On the table side — which is where most enterprise value sits — the agent’s ceiling is set by your semantic layer, not by model quality.

If your warehouse has three definitions of “active customer,” inconsistent grain between your orders and sessions tables, and columns named flag_2 with no description, an agent will do exactly what a new analyst with no context would do: pick something plausible and be wrong. The difference is that the new analyst asks someone by Wednesday, and the agent never does.

Concretely, before agents are worth piloting on your data:

• Curated tables, not raw ones. Point agents at governed, well-modeled gold tables with documented columns — not at the bronze layer, not at production replicas. Narrow, clean scope is the single highest-leverage thing you control.
• Metadata that carries meaning. Table and column descriptions in a catalog (Unity Catalog, if you’re on Databricks) aren’t documentation hygiene here — they’re the mechanism by which the agent knows what it’s looking at.
• One definition per metric. Agreed definitions for revenue, churn, active user, encoded once as views or metric definitions. If humans argue about it, the agent will simply pick a side and present it as fact.
• Known freshness. An agent that doesn’t know a table lags by six hours will happily report yesterday’s number as today’s.

None of this is agent-specific work. It’s the same modeling and governance discipline that makes BI trustworthy — the difference is that agents make skipping it expensive faster, because they scale confident wrong answers to more people.

Tools beat raw access

The instinct with a capable model is to hand it broad SQL access and let it figure things out. In practice, the systems that hold up do the opposite: they expose a small number of narrow, well-described, deterministic tools, and let the model decide which to call rather than how to compute.

get_revenue_by_region(start_date, end_date, region?) is a tool an agent uses correctly almost every time. “Here’s a SQL endpoint against 400 tables, good luck” is a tool that works in the demo and produces silent join errors in month two. Every piece of business logic you move out of the model’s improvisation and into a tested function is a step you no longer have to evaluate probabilistically.

The design rules that matter: keep the tool count small enough that selection is unambiguous, make each description precise about what it returns and at what grain, return structured data with explicit units and date ranges, and make errors informative — an agent recovers well from “no rows for that region; valid regions are X, Y, Z” and badly from a stack trace.

Identity, permissions and audit

This is where agentic projects most often stall in security review, and the concern is legitimate. If an agent queries data using one service principal with broad access, then every user of that agent effectively inherits that access — your row-level and column-level controls are gone, laundered through a chatbot.

What a defensible setup looks like: the agent runs queries under the identity of the requesting user, so existing catalog permissions apply unchanged; every tool call is logged with inputs, outputs and requester; write actions are scoped to purpose-built service accounts with the narrowest possible grants; and anything consequential — money moving, records changing, messages leaving the building — either runs through human approval or is confined to a reversible sandbox. Decide which actions require a human before the pilot, not after the first bad one.

Cost and latency at big-data scale

Agentic patterns multiply everything. One user question can become a dozen model calls and half a dozen queries as the agent explores, checks and retries. On a lakehouse, a careless exploratory query against a multi-terabyte table isn’t a rounding error — it’s real compute, and it can be triggered in a loop.

The controls that matter are unglamorous: point agents at pre-aggregated gold tables rather than letting them scan raw history; enforce query limits, row caps and timeouts at the warehouse layer, not in the prompt; cap the number of steps per task; cache aggressively, since agents re-ask the same sub-questions constantly; and route cheap steps to a small model, reserving the expensive one for planning and synthesis. Put a per-task cost budget on the pilot and alert when it’s exceeded. You want to learn the shape of that number in week two, not from a quarterly bill.

You are evaluating trajectories, not answers

Evaluation for a single-shot feature is a graded set of questions with known-good answers. That doesn’t transfer cleanly to agents, because two runs can reach the same correct number by different routes, and one of those routes may be luck.

So evaluate both: final-answer correctness against a domain expert’s spot check, and the trajectory — did it call sensible tools in a sensible order, and did it stop when it should have? Track step count and cost per task alongside accuracy, because the failure mode you’re most likely to hit isn’t a wrong answer, it’s an agent taking fourteen steps to produce a right one at ten times the expected cost. Keep the eval set versioned and rerun it on every prompt or tool change, or you’re shipping on vibes.

Where this pays off first

The best first agentic use cases are internal, reversible, and sit on data you already trust. Data engineering toil is a strong candidate: triaging a failed pipeline by reading logs, checking upstream freshness, and drafting a summary with a suggested cause — a task where the agent gathers context and a human decides. Analyst support is another: multi-step ad hoc questions that today queue behind a person, on a curated domain like support tickets or subscription billing.

What we’d avoid for a first project: anything customer-facing, anything that writes to a system of record without review, and anything spanning domains where the data model is still contested. Those aren’t permanent exclusions — they’re just a bad place to learn what your platform can’t yet support.

A sane sequence

Pick one domain with clean, well-understood tables. Fix the metadata and metric definitions in that domain first — that’s usually the real project, and it’s worth doing whether or not the agent ships. Build three to five narrow tools instead of granting broad access. Run under user identity with full logging. Set step, row and cost ceilings. Write twenty real tasks with known-good answers and grade both output and trajectory. Then, and only then, widen the scope.

If your catalog and semantic layer are already in good shape, that’s a matter of weeks. If they aren’t, the agent pilot quietly becomes a data platform cleanup — which is worth knowing before you commit to a launch date, not after.

That platform work is most of what makes agentic AI real, and it’s what our data engineering and AI team does for clients on Databricks and AWS — governed tables, tool layers, evaluation harnesses, and the cost controls that keep the whole thing affordable. If you’re scoping an agentic project and want a straight read on whether your data is ready for it, talk to us.