Abstract: At present, in the field of aluminum surface defect detection, the commonly used detection models have problems such as low detection accuracy, weak real-time performance, and large number of parameters. In view of the above problems, the target detection model YOLOv8 has been improved. Firstly, the model uses the self-developed dynamic deformable convolution module to replace the original last layer C2f module. Secondly, the detection head in the real-time detection transformer (RT-DETR) model is transplanted into the new model, and the idea of the decoder in Transformer is used to omit the model post-processing steps. Finally, normalized Wasserstein distance (NWD) loss and wise-intersection over union version 3 (WIoUv3) loss are combined as the regression loss function of the improved model, so that the model dynamically selects anchor boxes and solves the problem of inaccurate label assignment caused by the different sensitivity of IoU to different defect types of different sizes. Compared with the baseline model you only look once version 8 (YOLOv8), the average accuracy of the improved model is improved by 3.8%, and the speed reaches 92 f/s. At the same time, it also has strong robustness on the steel surface defect detection dataset and the solar panel surface defect detection dataset. The new model has great advantages in real-time detection and actual deployment.