Abstract

Text semantic segmentation involves partitioning a document into multipleparagraphs with continuous semantics based on the subject matter, contextualinformation, and document structure. Traditional approaches have typicallyrelied on preprocessing documents into segments to address input lengthconstraints, resulting in the loss of critical semantic information acrosssegments. To address this, we present CrossFormer, a transformer-based modelfeaturing a novel cross-segment fusion module that dynamically models latentsemantic dependencies across document segments, substantially elevatingsegmentation accuracy. Additionally, CrossFormer can replace rule-based chunkmethods within the Retrieval-Augmented Generation (RAG) system, producing moresemantically coherent chunks that enhance its efficacy. Comprehensiveevaluations confirm CrossFormer's state-of-the-art performance on public textsemantic segmentation datasets, alongside considerable gains on RAG benchmarks.