Abstract

Memory, additional information beyond the training of large language models(LLMs), is crucial to various real-world applications, such as personalassistant. The two mainstream solutions to incorporate memory into thegeneration process are long-context LLMs and retrieval-augmented generation(RAG). In this paper, we first systematically compare these two types ofsolutions on three renovated/new datasets and show that (1) long-contextsolutions, although more expensive, shall be easier to capture the big pictureand better answer queries which require considering the memory as a whole; and(2) when the queries concern specific information, RAG solutions shall be morecompetitive especially when the keywords can be explicitly matched. Therefore,we propose a novel method RAG-Tuned-LLM which fine-tunes a relative small(e.g., 7B) LLM using the data generated following the RAG principles, so it cancombine the advantages of both solutions. Extensive experiments on threedatasets demonstrate that RAG-Tuned-LLM can beat long-context LLMs and RAGmethods across a wide range of query types.