Abstract

Teleoperation is essential for autonomous robot learning, especially inmanipulation tasks that require human demonstrations or corrections. However,most existing systems only offer unilateral robot control and lack the abilityto synchronize the robot's status with the teleoperation hardware, preventingreal-time, flexible intervention. In this work, we introduce HACTS(Human-As-Copilot Teleoperation System), a novel system that establishesbilateral, real-time joint synchronization between a robot arm andteleoperation hardware. This simple yet effective feedback mechanism, akin to asteering wheel in autonomous vehicles, enables the human copilot to interveneseamlessly while collecting action-correction data for future learning.Implemented using 3D-printed components and low-cost, off-the-shelf motors,HACTS is both accessible and scalable. Our experiments show that HACTSsignificantly enhances performance in imitation learning (IL) and reinforcementlearning (RL) tasks, boosting IL recovery capabilities and data efficiency, andfacilitating human-in-the-loop RL. HACTS paves the way for more effective andinteractive human-robot collaboration and data-collection, advancing thecapabilities of robot manipulation.