Businesses look to NL2SQL software to query exploding data volumes

Software tools that use large language models to extract instant insights from databases instead of relying on code-writing specialists are poised to help businesses tap more information on the fly.

Rather than leaning on SQL-writing business analysts to distill database tables into reports and dashboards, some organizations are tapping “natural language to SQL” software that turns conversational typed or spoken business questions into SQL code for database queries via generative AI. The tools are helping users analyze reams of information about customers, products, and sales—and then deliver immediate answers to staff.

Oracle, Microsoft, Google, IBM, Amazon Web Services, Snowflake, Databricks, and Teradata have developed their own versions of so-called NL2SQL software, and it’s become an academic research topic as well.

Generative AI has already transformed the way computer users interact with unstructured data, such as documents, PDFs, emails, and social media posts, by making them more readable by large language models and available for searches that understand users’ meaning beyond keywords. Now it’s poised to do the same for relational database tables, housing businesses’ valuable information on sales, customers, employees, and operations. With NL2SQL-generated queries, managers and their teams can more easily see, for example, what’s driving revenue in a specific region, spot high-spending baby boomers in San Francisco, or divine why usage of an app is ebbing.

“Customers are going to need to plan and execute these programs carefully.”

Companies considering adopting the technology will need to pay close attention to how it’s implemented. Cloud computing costs can rise as thousands of employees ask LLMs daily questions that churn out database queries. As business users demand insights across markets, customers, and supply chains, IT departments will need to integrate data silos. And companies need scalable data analytics platforms to run queries across millions of customers and billions of transactions.

“This technology is going to be used on a large scale, and the number of people actually touching the data is going to go up by a large number,” says Richard Winter, CEO of consultancy WinterCorp, which helps businesses evaluate databases for their most demanding workloads. “It’s not going to be instant magic that solves every problem. Customers are going to need to plan and execute these programs carefully.”

Accuracy and efficiency boost

Rank-and-file employees can trip up LLMs by submitting vague or confusing questions. At large companies with thousands of database tables, AI models don’t always pick the right ones for a job. Complicated AI-generated queries and statistical functions that involve joining large database tables may require reading data from disk, slowing performance and making it difficult to derive correct answers. The result may be more nontechnical users generating complex queries that require gathering data from diverse sources.

NL2SQL vendors and their customers are taking steps to boost accuracy and efficiency. They’re sending more metadata to LLMs to help them better understand relationships among database tables and the performance metrics companies value. They’re adding features so users can provide AI models with real examples of how automatically generated SQL could have yielded a more accurate result.

Oracle and other vendors are setting limits on the amount of data flowing to and from LLMs, so that heavy users don’t break AI inference budgets—the costs model providers charge for data uploaded from prompts or sent back as answers. Coming this year are NL2SQL AI agents that can take a second look at answers that aren’t likely to be accurate, then generate helper code to determine data points the model may have missed the first time.

From BI to AI

NL2SQL is the latest evolution in a decades-long quest to make corporate data more accessible to executives and office workers, spanning 1980s spreadsheets through the formatted reports and colorful dashboards of the business intelligence era in the 2000s.

Today, enterprise information is spread across relational databases, data warehouses, and nonrelational systems, including data lakes in the cloud. NL2SQL and other AI queries will lead to an explosion of users with direct access to enterprise data—maybe 10 or more times the number as during the BI era, Winter says. The pressure on business units and IT will only grow as LLM-powered AI agents work in the background to fetch data from enterprise applications on their own.

“Databases are an enormous success, and there are many of them behind every process,” says Carsten Binnig, a professor of computer science at the Technical University of Darmstadt in Germany, who researches how to make text-to-SQL systems more accurate. “With databases we can process petabytes of data in a very short time. Text-to-SQL gives us a way to enable access for everyone—including nontechnical users.” But there are still challenges, such as dealing with complex queries, data, and schemas that are typical in enterprises, Binnig says.

A system designed by his group at TU Darmstadt called HLR-SQL uses reasoning models, a type of LLM trained to plan its output in logical steps, to compose SQL queries incrementally for databases that include many tables. Another, called CAESURA, can query not only tables, but also images and text—useful, for example, in medical scenarios where queries span tabular patient information and textual doctors’ reports.

Massachusetts Institute of Technology, University of California, Berkeley, and Stanford University are also developing systems that blend tables with other modalities.

Finer query control

To be sure, SQL-generating AI models aren’t yet ready for managers to dictate streams of questions to their phones and get instantaneous updates. AI systems can stumble on ambiguously named columns, tables, and dynamic data views, and they tend to return an answer even if they haven’t found the right one, says Brad Shimmin, an analyst at IT consultancy Futurum. “That is the biggest challenge with these tools: How do you know the answer isn’t just valid SQL, but more importantly, that it’s correct?” he says.

The BIRD-SQL benchmark for LLM-generated SQL, with a data set covering three dozen professional domains, shows the top five AI models on its leaderboard achieving 76% to 82% accuracy, compared with 93% for code written by engineers and technical students. The top models on Yale’s Spider benchmark list are 84% to 91% accurate.

To improve precision, businesses are employing techniques that give them finer control over queries. They’re sending metadata to models to describe business terminology absent from tables, as well as to help LLMs understand large databases’ schema and better join tables to answer questions.

Metadata can also show which KPIs a business cares about, pulling information pertinent to a user’s question at inference time using retrieval-augmented generation (RAG). That lets NL2SQL software tailor responses according to whether a user is, for example, asking about shipping or customer management, says Sanket Jain, an architect for Oracle Autonomous AI Database Select AI, a software tool that lets customers of Oracle Autonomous AI Database turn colloquially written queries into SQL code.

Technical users can adjust AI models’ outputs from a prompt window, uploading descriptions of how to produce better answers, or even examples of correct SQL for joining tables or other operations. These can be stored as vectors, or arrays of numbers representing concepts, for use in future queries.

The approach avoids the cost of trying to change the underlying model, and it controls the amount of information sent to LLMs, which keeps API costs in check, Jain says. “That’s very important to most users—the cost of AI,” he notes.

“That’s very important to most users—the cost of AI.”

Oracle plans to provide Select AI customers with the ability to limit the hourly number of tokens, or word fragments that an LLM processes, by employee profile, to throttle heavy users.

AI agents—LLM-powered, flexible software assistants in enterprise software—are also enhancing NL2SQL tools. For example, agents could detect when a query yields an improbable answer, or one with no results, given vague or missing data. An AI agent could then write SQL code to interpret values in a table—for example, turning international numeric codes for countries into country names—in order to produce a correct answer in a few more seconds.

IT organizations can build similar capabilities with agentic frameworks, such as LangGraph and CrewAI. Oracle also released Select AI Agent, a framework for building and managing agents from within the Oracle Autonomous AI Database using PL/SQL and Python.

Billions of transactions

As businesses consider adding NL2SQL to their arsenals, or expanding their purview from data scientists to managers, they’ll need to assess whether their database landscapes and governance policies are up to par.

At large retailers, telcos, and banks with tens of millions of customers and billions of transactions, running queries on market or purchasing patterns could mean all the matched records don’t fit into CPUs’ cache memory, forcing a system to repeatedly fetch data from disk and leading to performance problems, consultant Winter says. “Queries with big sets on both sides of the join are a problem,” he says. “If you have to go to disk once to answer a question, that’s not a big deal. If you need to go out a million times, it is.”

Some businesses with especially large data needs are adding cloud storage layers to their data lakes in so-called open table formats, such as Apache Iceberg, which brings improved query performance and data management capabilities. Storing data in open tables can make it easier for companies to lay the groundwork for adding query engines optimized for AI and NL2SQL workloads in coming years, without needing to convert their existing data.

Businesses are also setting controls on the information AI-generated database queries can show to different users. For instance, Oracle Select AI can limit the tables users can query based on their organizational role and seniority.