2026-05-25原文

Knowledge Distillation for Low-Resource Open-source Text-to-SQL Model

This paper proposes a knowledge-aware Text-to-SQL framework that constructs a task-specific knowledge base including schema semantics, abbreviations, business logic, and query patterns, and injects them into both training and inference. Experiments on seven benchmarks demonstrate substantial performance improvements for both open-source and closed-source large language models, especially in low-resource domain-specific settings.

Article intelligence

EngineersAdvanced

Key points

Addresses challenges in low-resource Text-to-SQL with opaque schema and implicit business logic.
Proposes a knowledge-aware framework that builds a knowledge base for data synthesis and inference.
Achieves significant improvements on seven benchmarks for both open and closed LLMs.
Enhances generalization, robustness, and adaptability in domain-specific scenarios.

Why it matters

This matters because addresses challenges in low-resource Text-to-SQL with opaque schema and implicit business logic.

Technical impact

May affect model selection, inference cost, product capability, and evaluation benchmarks.

[2605.22843] Knowledge Distillation for Low-Resource Open-source Text-to-SQL Model

[Submitted on 13 May 2026]

Title:Knowledge Distillation for Low-Resource Open-source Text-to-SQL Model

View a PDF of the paper titled Knowledge Distillation for Low-Resource Open-source Text-to-SQL Model, by Tianhao Qiu and 1 other authors

View PDF HTML (experimental)

Abstract:Text-to-SQL converts natural language questions into executable SQL queries, enabling non-technical users to access relational databases for analytics and intelligent data services. In real-world scenarios, performance is often constrained by low-resource settings, where high-quality annotated \texttt{} pairs are scarce, particularly for domain-specific databases. Additional challenges include opaque schema definitions, abbreviations, and implicit business logic that are not explicitly encoded in the schema. Existing data synthesis and prompting techniques improve coverage but often fail to produce task-specific, semantically grounded examples aligned with database constraints. To address these challenges, we propose a knowledge-aware Text-to-SQL framework that constructs task-specific knowledge base including schema semantics, abbreviations, business logic, and query patterns, and injects them into both training and inference. This framework generates diverse, contextually grounded synthetic training data and enhances inference through targeted knowledge retrieval. Experiments on seven benchmarks, covering both general and domain-specific datasets, demonstrate that our approach substantially improves the performance of open-source and closed-source large language models in Text-to-SQL tasks, especially in low-resource domain-specific settings, enhancing generalization, robustness, and adaptability.

Comments: 17ages, 5 figures

Subjects:

Computation and Language (cs.CL); Information Retrieval (cs.IR)

MSC classes: 68T50, 68P15

ACM classes: H.2.3; I.2.7; I.2.6

Cite as: arXiv:2605.22843 [cs.CL]

(or arXiv:2605.22843v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2605.22843

arXiv-issued DOI via DataCite

Submission history

From: Tianhao Qiu [view email] [v1] Wed, 13 May 2026 07:54:13 UTC (1,519 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Knowledge Distillation for Low-Resource Open-source Text-to-SQL Model, by Tianhao Qiu and 1 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CL

new | recent | 2026-05

Change to browse by:

cs cs.IR

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)