Abstract

Existing Retrieval-Augmented Generation (RAG) systems face challenges inenterprise settings due to limited retrieval scope and data security risks.When relevant internal documents are unavailable, the system struggles togenerate accurate and complete responses. Additionally, using closed-sourceLarge Language Models (LLMs) raises concerns about exposing proprietaryinformation. To address these issues, we propose the Secure Multifaceted-RAG(SecMulti-RAG) framework, which retrieves not only from internal documents butalso from two supplementary sources: pre-generated expert knowledge foranticipated queries and on-demand external LLM-generated knowledge. To mitigatesecurity risks, we adopt a local open-source generator and selectively utilizeexternal LLMs only when prompts are deemed safe by a filtering mechanism. Thisapproach enhances completeness, prevents data leakage, and reduces costs. Inour evaluation on a report generation task in the automotive industry,SecMulti-RAG significantly outperforms traditional RAG - achieving 79.3 to 91.9percent win rates across correctness, richness, and helpfulness in LLM-basedevaluation, and 56.3 to 70.4 percent in human evaluation. This highlightsSecMulti-RAG as a practical and secure solution for enterprise RAG.