Abstract

Despite the significant advancements in Text-to-SQL (Text2SQL) facilitated bylarge language models (LLMs), the latest state-of-the-art techniques are stilltrapped in the in-context learning of closed-source LLMs (e.g., GPT-4), whichlimits their applicability in open scenarios. To address this challenge, wepropose a novel RObust mUltitask Tuning and collaboration mEthod (ROUTE) toimprove the comprehensive capabilities of open-source LLMs for Text2SQL,thereby providing a more practical solution. Our approach begins withmulti-task supervised fine-tuning (SFT) using various synthetic training datarelated to SQL generation. Unlike existing SFT-based Text2SQL methods, weintroduced several additional SFT tasks, including schema linking, noisecorrection, and continuation writing. Engaging in a variety of SQL generationtasks enhances the model's understanding of SQL syntax and improves its abilityto generate high-quality SQL queries. Additionally, inspired by thecollaborative modes of LLM agents, we introduce a Multitask CollaborationPrompting (MCP) strategy. This strategy leverages collaboration across severalSQL-related tasks to reduce hallucinations during SQL generation, therebymaximizing the potential of enhancing Text2SQL performance through explicitmultitask capabilities. Extensive experiments and in-depth analyses have beenperformed on eight open-source LLMs and five widely-used benchmarks. Theresults demonstrate that our proposal outperforms the latest Text2SQL methodsand yields leading performance.