Abstract

Large Language Models (LLMs) have emerged as a promising solution forconverting natural language queries into SQL commands, enabling seamlessdatabase interaction. However, these Text-to-SQL (Text2SQL) systems faceinherent limitations, hallucinations, outdated knowledge, and untraceablereasoning. To address these challenges, the integration of retrieval-augmentedgeneration (RAG) with Text2SQL models has gained traction. RAG serves as aretrieval mechanism, providing essential contextual information, such as tableschemas and metadata, to enhance the query generation process. Despite theirpotential, RAG + Text2SQL systems are susceptible to the quality and size ofretrieved documents. While richer document content can improve schema relevanceand retrieval accuracy, it also introduces noise, increasing the risk ofhallucinations and reducing query fidelity as the prompt size of the Text2SQLmodel increases. This research investigates the nuanced trade-off betweendocument size and quality, aiming to strike a balance that optimizes systemperformance. Key thresholds are identified where performance degradationoccurs, along with actionable strategies to mitigate these challenges.Additionally, we explore the phenomenon of hallucinations in Text2SQL models,emphasizing the critical role of curated document presentation in minimizingerrors. Our findings provide a roadmap for enhancing the robustness of RAG +Text2SQL systems, offering practical insights for real-world applications.