Abstract: To meet the constant force polishing needs of robots, this paper designs a floating polishing actuator and conducts research on polishing control algorithms. The structural design of the floating polishing actuator was carried out, and the force analysis and dynamic modeling of the floating polishing actuator system were carried out. On the basis of traditional PID control algorithms, the DDPG deep reinforcement learning algorithm is used to tune the PID control parameters. Conduct experimental verification of the constant force performance of the floating polishing actuator, and the experimental results show that the floating polishing actuator designed in this paper can meet the requirements of constant force control. By using the DDPG deep reinforcement learning algorithm to tune PID control parameters, the tedious parameter tuning steps are reduced, and it has better constant force control performance.