CNC Condition Monitoring
How to Split CNC Vibration Data Without Leakage
Learn why random file splitting can exaggerate CNC monitoring accuracy and how machine-, session- and time-grouped tests provide stronger evidence.
A model can look accurate simply because nearly identical records from one machine or session appear in both training and test sets. Grouped splitting asks a harder and more useful question: does the method work on genuinely unseen context?
Where leakage enters
Overlapping windows, repeated cycles and files from the same acquisition session share structure. Random row splitting scatters that structure across both sides, letting a model recognize context instead of learning condition.
Choose a group that matches deployment
Hold out sessions to test future sessions, machines to test fleet transfer, and later dates to test future behavior. Each split supports a different claim.
Stratification is not enough
Balancing good and bad labels does not prevent identity leakage. Preserve label coverage where possible, but never break a natural group merely to make class counts look tidy.
Use nested decisions
Feature selection, scaling and threshold tuning must happen inside training data. The final held-out group should remain untouched until the complete pipeline is frozen.
Design the split from the intended claim
For a machine-specific monitor, train on earlier healthy and labeled cycles, tune on a later block and test on the latest untouched block.
For a fleet method, hold out an entire machine. If performance falls, report the limitation rather than quietly mixing its records into training.
| Deployment claim | Stronger test |
| New windows in same session | Record split |
| Future production | Time-block split |
| New machine | Machine-group split |
| New tool or program | Tool/program holdout |
Common mistakes to avoid
- Splitting after creating overlapping windows.
- Fitting normalization on all records.
- Trying many pipelines against the final test set.
Frequently asked questions
Is random splitting ever valid?
Yes, when rows are truly independent and that matches deployment, which vibration windows often do not.
What if a held-out group has one label?
It may still test baseline transfer, but it cannot estimate every classification metric.
Should the test set be balanced?
Not artificially; prevalence affects operational interpretation.
Practical workflow for this method
Store a split manifest listing every source file and group. Reproducibility requires more than a random seed.
Compare random and grouped results openly. A large gap is useful evidence about context dependence, not a reason to hide the grouped test.
About the data used in this guide
The charts use a small teaching sample selected from machines M01, M02 and M03, across the available OP01–OP06 files. The source records tri-axial acceleration at 2 kHz and labels available examples as good or bad. Label coverage is uneven across machine-operation groups, so missing groups are not treated as healthy evidence. These figures are transparent worked examples, not population estimates or universal fault thresholds.
Dataset: CNC Machining Data, CC BY 4.0. Recommended citation: Tnani, Mohamed-Ali; Feil, Michael; Diepold, Klaus. Smart Data Collection System for Brownfield CNC Milling Machines: A New Benchmark Dataset for Data-Driven Machine Monitoring. Procedia CIRP 107 (2022), 131–136. Research paper.
We explain what the selected data supports and avoid naming a mechanical fault when the dataset only provides a good/bad process label. A machine should be inspected by a qualified person before maintenance or safety decisions are made.