Split a dataset and check for leakage before training
Practice the sequence you already learned for “Split a dataset and check for leakage before training” until the order feels automatic. Drill ML-ops evaluation order from applied-ML training. The model is not trained until the split unit, time cutoff, and leakage checks are done.
- Define the split unit (user, claim, time) so rows from one unit do not land in both sets.
- Cut train / validation / test, using time if the problem is temporal.
- Search for leakage: target in features, future timestamps, or ID hashes.
- Fit only on the training set.
- Report metrics on the held-out test set once.
- Do not tune on the test set.
- Do not shuffle a time series randomly.