Abstract:
Grating-based displacement signal processing is a key enabler of high-accuracy motion in ultra-precision machine tools and other high-end manufacturing and metrology equipment. In engineering applications, the sine/cosine observation chain of grating encoders often suffers from phase bias, quadrature (orthogonality) errors, and noise with uncertain statistics that vary with operating conditions. To address these issues, an improved neural-network-aided Kalman filtering method, termed KalmanNet-LSTM, was proposed by integrating a long short-term memory (LSTM) network into the (Kalman filter,KF) . In this method, the regulation rule for the time-varying Kalman gain was learned through a data-driven approach. Phase-related features extracted from the grating sine/cosine signals, together with statistical information such as innovation, were constructed as network inputs to enhance the modeling capability for temporally correlated disturbances and varying operating conditions. Experimental results on a UMAC-based platform demonstrate that KalmanNet-LSTM achieves a root mean square (RMS) displacement error of 0.418 nm, representing an 82.1% reduction compared with the conventional Kalman filter. Furthermore, when validated against a capacitive displacement sensor as ground truth, the RMS error over representative motion segments ranges from approximately 0.378 nm to 0.421 nm. These results demonstrate that KalmanNet-LSTM provides improved robustness and estimation stability under structural measurement-chain errors and condition-dependent noise uncertainty, thereby enhancing the performance of ultra-precision motion feedback systems.