Abstract: To achieve high-energy-efficient and high-quality machine tool service combinations in cloud manufacturing, a two-stage decision model based on a supply-demand knowledge graph was developed. A knowledge graph considering energy consumption attributes and the subjective and objective demands for part processing was first constructed. Secondly, a calculation model for machine tool service time, cost, energy consumption, and quality indicators was established, using three strategies for energy consumption based on power test data, specific cutting energy, and rated power. A two-stage matching model was built to meet customer needs, with initial service selection in the first stage through knowledge graph searches and implicit relationship reasoning. The second stage used a Markov decision process to optimize energy-saving service combinations, solved with the Actor_Critic algorithm from reinforcement learning. Experiments with a machine tool service resource simulation pool and box body machining cases showed that the Actor_Critic algorithm had better convergence than deep Q-learning (DQN), policy gradient method (PGM) and deep deterministic policy gradient (DDPG), enabling rapid matching of cost-effective, energy-saving, and high-quality service combinations in cloud manufacturing.