Abstract

Although people are impressed by the content generation skills of largelanguage models, the use of LLMs, such as ChatGPT, is limited by the domaingrounding of the content. The correctness and groundedness of the generatedcontent need to be based on a verified context, such as results fromRetrieval-Augmented Generation (RAG). One important issue when adapting LLMs toa customized domain is that the generated responses are often incomplete, orthe additions are not verified and may even be hallucinated. Prior studies onhallucination detection have focused on evaluation metrics, which are noteasily adaptable to dynamic domains and can be vulnerable to attacks likejail-breaking. In this work, we propose 1) a post-processing algorithm thatleverages knowledge triplets in RAG context to correct hallucinations and 2) adual-decoder model that fuses RAG context to guide the generation process.