Training a Deep Learning Model
Training a deep learning model means repeatedly showing it examples and adjusting its weights to reduce errors. Understanding the training process helps you diagnose problems and improve results.
The Training Loop
Training a deep learning model follows this cycle:
1. Prepare data: split into training and validation sets
2. Initialize weights: start with small random values
3. For each epoch (pass through the data):
a. Process training data in small batches
b. Forward pass: make predictions
c. Calculate loss: measure errors
d. Backward pass: calculate gradients
e. Update weights: apply gradient descent
f. After the epoch: evaluate on validation data
4. Stop when validation performance stops improving
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
Training Illustrated with Metrics
# Simulating training progress (illustrative data)
# In real deep learning, these numbers come from actual training
# Here we simulate what training curves typically look like
training_metrics = [
# (epoch, train_loss, val_loss, train_acc, val_acc)
(1, 2.30, 2.28, 0.15, 0.17),
(2, 1.85, 1.80, 0.35, 0.38),
(5, 1.20, 1.15, 0.58, 0.61),
(10, 0.75, 0.72, 0.75, 0.77),
(20, 0.35, 0.40, 0.89, 0.87),
(30, 0.20, 0.38, 0.94, 0.88),
(40, 0.12, 0.45, 0.97, 0.86), # overfitting starts here
]
print("Training Progress:")
print("-" * 70)
print(f"{'Epoch':>6} {'Train Loss':>10} {'Val Loss':>10} {'Train Acc':>10} {'Val Acc':>10}")
print("-" * 70)
for epoch, tl, vl, ta, va in training_metrics:
gap = vl - tl
flag = " <-- overfitting!" if gap > 0.15 else ""
print(f"{epoch:>6} {tl:>10.2f} {vl:>10.2f} {ta:>10.0%} {va:>10.0%}{flag}")
print()
print("Notice: early epochs show rapid improvement.")
print("After epoch 30, val_loss increases even as train_loss decreases.")
print("This is overfitting: the model memorizes training data.")
print("The best model is typically saved at the lowest validation loss.")