Suppose each label is the return over the next ten days. A training sample dated 1 March carries information up to 11 March. If the test fold begins on 5 March, that training row overlaps the test period and the model has partly seen the answer.
Purging removes every training row whose label window intersects the test window. In the example you would drop training rows dated from 24 February onward before testing on the fold beginning 5 March.
The effect is not cosmetic. On labels with long horizons, purging can cut a reported classification accuracy from something like 0.61 to 0.53, which is the difference between an exciting result and a coin flip. Pair it with an embargo-period for the leakage that runs the other way.
Related: embargo-period, cross-validation, data-leakage