Overfitting
Overfitting happens when a model memorizes the training data instead of learning general patterns. It scores well on training data but poorly on new data.
12 min•By Priygop Team•Updated 2026
What Overfitting Looks Like
A student who memorizes every answer in the textbook word for word may score 100% on practice problems. But on the actual exam with slightly different questions, they fail.
Overfitting in AI is the same problem:
- Training accuracy: very high (the model memorized the data)
- Validation accuracy: much lower (the model cannot generalize)
Signs of overfitting in training:
- Training loss keeps decreasing
- Validation loss starts increasing (or stops improving)
- The gap between training and validation performance grows wider
Diagram
Loading diagram…
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
How to Fix Overfitting
- Get more training data: the most effective fix. More data makes it harder to memorize
- Use dropout: randomly disable some neurons during training to prevent co-dependency
- Add weight regularization (L2): penalize large weights to keep the model simple
- Use early stopping: stop training when validation loss stops improving
- Reduce model complexity: use fewer layers or fewer neurons
- Use data augmentation: artificially create more training examples by rotating, flipping, or cropping images
Detecting Overfitting
Detecting Overfitting
# Detecting overfitting by comparing train and validation loss
# Simulated training history (illustrative)
history = [
# (epoch, train_loss, val_loss)
(1, 1.50, 1.45),
(5, 0.90, 0.88),
(10, 0.55, 0.57),
(15, 0.30, 0.45), # gap starts widening
(20, 0.18, 0.58), # clear overfitting
(25, 0.10, 0.72), # severe overfitting
]
print("Training History: Detecting Overfitting")
print(f"{'Epoch':>6} {'Train Loss':>10} {'Val Loss':>10} {'Status':>20}")
print("-" * 55)
for epoch, train_loss, val_loss in history:
gap = val_loss - train_loss
if gap < 0.05:
status = "Good (learning)"
elif gap < 0.15:
status = "Warning (watch)"
else:
status = "OVERFITTING"
print(f"{epoch:>6} {train_loss:>10.2f} {val_loss:>10.2f} {status:>20}")
print()
print("Best model: epoch where validation loss is lowest (epoch 10 here).")
print("Save the model checkpoint at this point.")Key Takeaways
- Overfitting happens when a model memorizes the training data instead of learning general patterns.
- Get more training data: the most effective fix. More data makes it harder to memorize
- Use dropout: randomly disable some neurons during training to prevent co-dependency
- Add weight regularization (L2): penalize large weights to keep the model simple