Abstract: In view of the current problems of low detection accuracy of aluminum surface defect detection algorithms in practical engineering applications and being not lightweight enough and difficult to deploy, an aluminum surface defect detection method based on improved YOLOv5s is proposed. This algorithm is based on the classic YOLOv5s model, and integrates the ShufflenNetV2-Block algorithm into the backbone network to reduce the computational complexity of the model; then adds an SE attention mechanism to focus attention on defect-related areas and better distinguish between categories. The difference between them improves classification performance and detection efficiency; finally, the loss function is optimized and SIoU (S-intersection over union) is used to replace CIoU to improve network positioning accuracy. The results show that the detection accuracy of pinhole and spot defects is improved by 8.3% and 8.4% respectively compared with the original YOLOv5, and the mAP value is increased by 6.4%, which improves the defect detection accuracy and reduces the size and memory of the model, making it more convenient. Mobile terminal deployment effectively improves the problem of missed detection during the manufacturing process.