ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning

  • 2025-03-25 09:00:58
  • Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Fan Yang, Zenan Zhou, Weipeng Chen, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen
  • 0

Abstract

Large Language Models (LLMs) have shown remarkable capabilities in reasoning,exemplified by the success of OpenAI-o1 and DeepSeek-R1. However, integratingreasoning with external search processes remains challenging, especially forcomplex multi-hop questions requiring multiple retrieval steps. We proposeReSearch, a novel framework that trains LLMs to Reason with Search viareinforcement learning without using any supervised data on reasoning steps.Our approach treats search operations as integral components of the reasoningchain, where when and how to perform searches is guided by text-based thinking,and search results subsequently influence further reasoning. We train ReSearchon Qwen2.5-7B(-Instruct) and Qwen2.5-32B(-Instruct) models and conductextensive experiments. Despite being trained on only one dataset, our modelsdemonstrate strong generalizability across various benchmarks. Analysis revealsthat ReSearch naturally elicits advanced reasoning capabilities such asreflection and self-correction during the reinforcement learning process.