Building Reliable Data Analytics Agents: Lessons from the KDD Cup

NVIDIA's KGMON team placed second in the KDD Cup 2026 Data Agents competition, where agents had to answer plain-language questions across databases, CSV and JSON files, documents, PDFs and briefing videos. The catch was that every team had to use a small, fixed language model, so the only real lever was the harness around it.

That constraint mirrors a lot of real companies, which want analytics agents on smaller or open models they can afford and control. The team's write-up is a practical case that reliability comes from structure, not from giving the model more freedom.

We think this is one of the more useful agent posts of the week precisely because it is unglamorous. The team folded CSV and JSON into a single SQLite database, briefed the agent on tables, join keys and data-quality traps before it started, and limited it to a handful of tools for schema, SQL, document lookup and writing the answer. Middleware fixed malformed calls so one bad step did not sink a run. Document text went through a separate, zero-temperature helper so raw prose did not flood the main context.

That is the opposite of the 'give it every tool' instinct, and we think it is right. Analytics errors are usually quiet: a wrong join, the wrong row grain, a missed rule buried in a PDF. A narrow interface makes those mistakes rarer and easier to find.

The team is also candid about the downsides, which we appreciate. Repeated attempts improved reliability but cost tokens, latency and compute. Self-improvement loops risked overfitting the benchmark, from hardcoding examples into prompts to piling up contradictory instructions. Their answer was held-out tasks, prompt audits and human approval before changes stick. That is a governance pattern worth borrowing well beyond competitions.

Our caution: this is a benchmark result with a fixed model, no internet access and exact value scoring, and the authors say not every choice should be copied into production. I'm watching whether enterprise data teams adopt the same discipline, especially execution traces that let a person see where an answer first went wrong. An analytics agent that is confidently wrong at scale is worse than no agent at all.

KDD Cup is the annual data mining competition attached to the ACM SIGKDD conference. The post says a presentation archive with recorded sessions and slides from eight featured teams is available as a design reference.