Deep Learning vs. Machine Learning for Predictive Maintenance under Limited Industrial Data: A Cross-Machine Validation Case Study

Predictive maintenance (PdM) has become a strategic factor in modern manufacturing systems. While deep learning (DL) architectures such as Long Short-Term Memory (LSTM) models have demonstrated strong performance on large-scale industrial datasets, their effectiveness in PdM under limited data conditions remains unclear, creating an important gap in the literature. This study addresses that gap by investigating whether LSTM provides advantages over classical machine learning (ML) model, Random Forest (RF), in small-sample industrial environments. The study contributes to the literature by systematically evaluating cross-machine generalization under limited failure data conditions and offers managerial implications for ML and DL adoption in PdM. The empirical analysis focuses on five CNC machines and considers practical data challenges common in production systems: missing observations, irregular sampling intervals, and maintenance-induced downtime. A Leave-One-Machine-Out cross-validation strategy was implemented to evaluate models’ performance. The findings show that, despite the temporal nature of the problem, the RF model outperformed LSTM across most evaluation metrics, achieving higher AUC and recall while exhibiting more stable cross-machine generalization. The results suggest that high predictive performance can be achieved without complex recurrent architectures when appropriate feature windowing and ensemble learning are employed. At the same time, LSTM achieved higher precision, indicating fewer false alarms. This highlights a managerial trade-off: RF may be preferable when maximizing fault detection is critical, whereas LSTM may remain attractive when minimizing unnecessary maintenance interventions is a priority. Overall, the study suggests that classical ML can provide a robust and computationally efficient alternative to DL in low-data industrial environments.