Abstract

Modern orchards are planted in structured rows with distinct panel divisionsto improve management. Accurate and efficient joint segmentation of point cloudfrom Panel to Tree and Branch (P2TB) is essential for robotic operations.However, most current segmentation methods focus on single instancesegmentation and depend on a sequence of deep networks to perform joint tasks.This strategy hinders the use of hierarchical information embedded in the data,leading to both error accumulation and increased costs for annotation andcomputation, which limits its scalability for real-world applications. In thisstudy, we proposed a novel approach that incorporated a Real2Sim L-TreeGen fortraining data generation and a joint model (J-P2TB) designed for the P2TB task.The J-P2TB model, trained on the generated simulation dataset, was used forjoint segmentation of real-world panel point clouds via zero-shot learning.Compared to representative methods, our model outperformed them in mostsegmentation metrics while using 40% fewer learnable parameters. This Sim2Realresult highlighted the efficacy of L-TreeGen in model training and theperformance of J-P2TB for joint segmentation, demonstrating its strongaccuracy, efficiency, and generalizability for real-world applications. Theseimprovements would not only greatly benefit the development of robots forautomated orchard operations but also advance digital twin technology.