翻訳待ち：Putting Task Expertise into RL Achieves Performance on Text-to-SQL

AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。

Many industries rely on relational databases that are queried with SQL. Most SQL is machine-written, but humans alone likely write billions of custom SQL queries each monthBased on our internal estimates and publicly available data, such as from Snowflake filings. prompted by business questions. They are quite good at it — humans score 92.96% on BIRD, a realistic benchmark for translating natural-language questions into SQL. However, AI performance on text-to-SQL has lagged behind. LLM scores on the BIRD leaderboard improved from just below 70% in 2024 to 82% today. Frontier models like GPT-5.6 Sol Ultra and Claude Fable 5 can score in the mid-80s, albeit at a cost that is prohibitive for high-volume applications. This isn’t for lack of training data: SQL is widely represented in the internet content used in LLM pretraining. The challenge for AI is in navigating the ambiguous questions and highly-contextual schema that characterize real-world examples. A common approach for improving AI performance on tasks people understand well is building agentic scaffolding. Systems such as OpenHands, AI co-scientist, and MetaGPT decompose a task into stages, each with its own prompt or model call. Text-to-SQL scaffolds follow the same pattern. A schema-linking stage narrows thousands of columns down to a candidate set.Real-world, enterprise data systems (including databases, data warehouses, and data lakehouses) contain up to millions of columns. Answering business questions often involves understanding which columns to use. Academic benchmarks are simpler. A generation stage samples queries. A self-correction stage repairs execution errors. A selection stage votes among the survivors. Every component is a separate call, and the orchestration is typically tuned for a given benchmark.