基于强化学习的臂手协同灵巧旋拧操作研究

Research on arm-hand coordinated dexterous screwdriving operation based on reinforcement learning

  • 摘要: 针对灵巧手握持螺丝刀旋拧时下压力与旋拧动作强非线性耦合、仿真迁移困难等问题,研究了基于强化学习的臂手协同灵巧旋拧操作策略及零样本迁移方法。区别于现有灵巧操作多聚焦自由空间物体旋转,针对有持续接触约束的工具旋拧任务,提出了臂手协同的师生蒸馏框架。搭建了由七自由度力控机械臂、三指灵巧手及触觉传感器组成的硬件平台。将旋拧任务建模为马尔可夫决策过程,设计了复合奖励函数,采用近端策略优化(proximal policy optimization, PPO)算法在仿真中训练教师策略,并通过行为克隆蒸馏得到仅依赖本体感知与触觉特征的学生策略。仿真训练表明,学生策略迭代约100次后收敛。零样本迁移实验中,在±1 mm位置偏差下,20次旋拧测试成功率达95%。该研究为实现灵巧手工具操作的零样本迁移提供了可行技术路径。

     

    Abstract: To address the issues of strong nonlinear coupling between axial pressing force and rotational motion, as well as the difficulty of sim-to-real transfer in screwdriving operations using a dexterous hand, a reinforcement learning based arm-hand coordinated dexterous screwdriving strategy and a zero-shot transfer method were studied. Distinguishing from existing dexterous manipulation works that mostly focus on object rotation in free space, this paper addresses the tool screwdriving task with sustained contact constraints and proposes an arm-hand coordinated teacher-student distillation framework. An arm-hand collaborative hardware platform consisting of a 7-DOF force-controlled robotic arm, a three-finger dexterous hand, and tactile sensors was established. The screwdriving task was formulated as a Markov decision process, and a composite reward function was designed. The teacher policy was trained in simulation using the PPO algorithm, and the student policy relying only on proprioception and tactile features was distilled via behavioral cloning. Simulation training results show that the student policy converges within about 100 iterations. In zero-shot real-robot transfer experiments with ±1 mm initial position deviation, the success rate reached 95% over 20 trials. This study provides a feasible technical pathway for zero-shot sim-to-real transfer of dexterous tool manipulation.

     

/

返回文章
返回