Abstract

We introduce MCTS-RAG, a novel approach that enhances the reasoningcapabilities of small language models on knowledge-intensive tasks byleveraging retrieval-augmented generation (RAG) to provide relevant context andMonte Carlo Tree Search (MCTS) to refine reasoning paths. MCTS-RAG dynamicallyintegrates retrieval and reasoning through an iterative decision-makingprocess. Unlike standard RAG methods, which typically retrieve informationindependently from reasoning and thus integrate knowledge suboptimally, orconventional MCTS reasoning, which depends solely on internal model knowledgewithout external facts, MCTS-RAG combines structured reasoning with adaptiveretrieval. This integrated approach enhances decision-making, reduceshallucinations, and ensures improved factual accuracy and response consistency.The experimental results on multiple reasoning and knowledge-intensive datasetsdatasets (i.e., ComplexWebQA, GPQA, and FoolMeTwice) show that our methodenables small-scale LMs to achieve performance comparable to frontier LLMs likeGPT-4o by effectively scaling inference-time compute, setting a new standardfor reasoning in small-scale models.