CNC Condition Monitoring
Group-Aware Validation for CNC Machine-Learning Models
Split vibration data by machine, date or production group to test generalization and prevent misleading scores.
Randomly splitting neighboring CNC files can put nearly identical conditions in training and test sets. The score then measures memory more than generalization.
What counts as a group
Machine, date, production batch, tool or continuous acquisition session may form a group. The right choice depends on the claim.
Leave-one-machine-out
Train on two machines and test on the third to ask whether the method transfers across machines.
Time-based testing
Train on earlier periods and test later data to expose drift. Do not let future statistics enter preprocessing.
Honest reporting
A lower grouped score is often more useful because it reveals the work needed before deployment.
A split that tests the real promise
To claim cross-machine generalization, train on M01 and M02, then test once on M03. Do not tune repeatedly on M03 and continue calling it unseen.
For time robustness, train on earlier periods and evaluate later periods. All preprocessing, including scaling and feature selection, must be fit without future data.
| Claim | Appropriate held-out group |
| Works on new files | Acquisition session |
| Works over time | Later date block |
| Works on new machines | Whole machine |
Common mistakes to avoid
- Randomly splitting adjacent files.
- Fitting normalization before the split.
- Tuning on the final test group.
Frequently asked questions
Why are grouped scores lower?
They remove easy similarities and expose real distribution shift.
Can cross-validation be grouped?
Yes, use folds that keep each group together.
What if there are only three machines?
Leave-one-machine-out results are useful but uncertainty remains large.
Practical workflow for this method
Write the deployment claim before choosing the split. If the goal is future performance on the same machines, hold out later time blocks. If the goal is installation on a new machine, hold out a complete machine. One random split cannot answer both questions.
Apply grouping to every transformation. Feature selection, imputation, normalization and threshold tuning must be learned without the held-out group. Even a simple scaler can leak information when fitted before the split. Use a reproducible pipeline that fits each step only on training data.
Report results for every held-out group as well as their average. With three machines, leave-one-machine-out testing produces three meaningful stories. A wide spread between them is evidence that transfer remains difficult, not a result to hide.
About the data used in this guide
The charts use a small teaching sample selected from machines M01, M02 and M03, primarily operations OP01 and OP02. The source records tri-axial acceleration at 2 kHz and labels available examples as good or bad. Label coverage is uneven across machine-operation groups, so missing groups are not treated as healthy evidence. These figures are transparent worked examples, not population estimates or universal fault thresholds.
Dataset: CNC Machining Data, CC BY 4.0. Recommended citation: Tnani, Mohamed-Ali; Feil, Michael; Diepold, Klaus. Smart Data Collection System for Brownfield CNC Milling Machines: A New Benchmark Dataset for Data-Driven Machine Monitoring. Procedia CIRP 107 (2022), 131–136. Research paper.
We explain what the selected data supports and avoid naming a mechanical fault when the dataset only provides a good/bad process label. A machine should be inspected by a qualified person before maintenance or safety decisions are made.