Abstract

Retrieve-augmented generation (RAG) frameworks have emerged as a promisingsolution to multi-hop question answering(QA) tasks since it enables largelanguage models (LLMs) to incorporate external knowledge and mitigate theirinherent knowledge deficiencies. Despite this progress, existing RAGframeworks, which usually follows the retrieve-then-read paradigm, oftenstruggle with multi-hop QA with temporal information since it has difficultyretrieving and synthesizing accurate time-related information. To address thechallenge, this paper proposes a novel framework called review-then-refine,which aims to enhance LLM performance in multi-hop QA scenarios with temporalinformation. Our approach begins with a review phase, where decomposedsub-queries are dynamically rewritten with temporal information, allowing forsubsequent adaptive retrieval and reasoning process. In addition, we implementadaptive retrieval mechanism to minimize unnecessary retrievals, thus reducingthe potential for hallucinations. In the subsequent refine phase, the LLMsynthesizes the retrieved information from each sub-query along with itsinternal knowledge to formulate a coherent answer. Extensive experimentalresults across multiple datasets demonstrate the effectiveness of our proposedframework, highlighting its potential to significantly improve multi-hop QAcapabilities in LLMs.