Abstract

Retrieval-augmented generation (RAG) enhances large language models (LLMs) byretrieving relevant documents from external sources and incorporating them intothe context. While it improves reliability by providing factual texts, itsignificantly increases inference costs as context length grows and introduceschallenging issue of RAG hallucination, primarily caused by the lack ofcorresponding parametric knowledge in LLMs. An efficient solution is to enhancethe knowledge of LLMs at test-time. Parametric RAG (PRAG) addresses this byembedding document into LLMs parameters to perform test-time knowledgeenhancement, effectively reducing inference costs through offline training.However, its high training and storage costs, along with limited generalizationability, significantly restrict its practical adoption. To address thesechallenges, we propose Dynamic Parametric RAG (DyPRAG), a novel framework thatleverages a lightweight parameter translator model to efficiently convertdocuments into parametric knowledge. DyPRAG not only reduces inference,training, and storage costs but also dynamically generates parametricknowledge, seamlessly enhancing the knowledge of LLMs and resolving knowledgeconflicts in a plug-and-play manner at test-time. Extensive experiments onmultiple datasets demonstrate the effectiveness and generalization capabilitiesof DyPRAG, offering a powerful and practical RAG paradigm which enablessuperior knowledge fusion and mitigates RAG hallucination in real-worldapplications. Our code is available at https://github.com/Trae1ounG/DyPRAG.