Loss against the selected target distribution.
Target and prediction, class by class
training targetmodel predictiongradient p − y
ycorrect = 0.90 · y + 0.10 / 5Smoothing spreads ε across all classes; frameworks may differ in the exact convention, so verify the implementation.
Model probability on the labeled class.
Magnitude of p − y across logits.
Uncertainty explicitly encoded by the target.
One-hot targets never stop asking for more
Even after the correct class wins, cross-entropy continues pushing its logit upward toward probability one. Label smoothing gives the optimum a finite target confidence.
Gradients change for every class
For softmax cross-entropy, each logit gradient is prediction minus target. Redistributing target mass weakens the winner's negative gradient and gives alternatives a small positive target.
Calibration is an outcome to measure
Smoothing can reduce overconfidence, but it can also alter representation geometry and harm some distillation or uncertainty workflows. Evaluate accuracy, log loss, calibration, and downstream use.
Tune it as a regularizer, not a ritual.
Check the convention
Some implementations spread ε across all classes; others allocate it only to incorrect classes. That changes the exact target and optimum confidence.
Audit noisy labels
Smoothing does not identify incorrect annotations. Compare with data cleaning, robust losses, reweighting, and cohort-specific error analysis.
Report the full recipe
Record ε, class count, loss implementation, augmentation, mixup, distillation, and calibration procedure so the training objective is reproducible.