Abstract: To achieve precise control of the tooth surface residual stress in internal gear shaping, a thermo-mechanical coupled finite element model of equivalent oblique cutting is established based on Abaqus. By adopting the single-stroke cutting assumption, the complex reciprocating motion is equivalent to a continuous feed speed, which reduces the computational cost of the simulation. Taking cutting speed, feed speed and cutting depth as independent variables, response surface methodology (RSM) is employed to construct prediction models for residual compressive stress S₁ and residual tensile stress S₂. The high accuracy and reliability of the models are verified by analysis of variance (ANOVA). By integrating the non-dominated sorting genetic algorithm (NSGA-II), multi-objective optimization is performed for residual stress, thereby obtaining the non-dominated optimal solution set. The results show that the coefficient of determination R² of the prediction models are greater than 0.96. The optimized parameter combination can significantly increase the compressive stress and reduce the tensile stress. The average absolute relative errors between simulation and experiment are 8.55% and 7.43%, respectively.