Beginner-Friendly Topic
Take your time - it's perfectly normal to re-read this topic 2-3 times. Try the interactive code editor below to run code yourself. Use the Q&A section to check your understanding before moving on. You've got this! 🚀
How a Neural Network Learns
A neural network learns by repeatedly making predictions, measuring errors (loss), and adjusting its weights to reduce those errors. This process is called training.
The Training Process
Training a neural network has four steps that repeat thousands of times:
- 1Forward pass: feed data through the network to get a prediction
- 2Calculate loss: measure how wrong the prediction is
- 3Backward pass (backpropagation): figure out which weights caused the error
- 4Update weights: nudge the weights in the direction that reduces loss
This cycle repeats for every batch of training examples, over many passes through the entire dataset (called epochs).
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
Gradient Descent: Adjusting Weights
The technique for adjusting weights is called gradient descent.
Imagine you are in foggy mountains and need to find the lowest valley. You cannot see far, so you feel the slope under your feet and take a small step in the downhill direction.
Gradient descent is the same idea. The network calculates which direction would reduce the loss (the 'downhill' direction) and takes a small step by adjusting the weights. After thousands of steps, it finds the weight values that produce the lowest possible loss.
Simple Learning Loop in Code
# Simplified neural network learning loop
# Shows the concept of training without complex math
import random
random.seed(42)
# Simple task: learn that output = 2 * input
training_data = [(1, 2), (2, 4), (3, 6), (4, 8), (5, 10)]
# Start with a random weight
weight = random.uniform(-1, 1)
learning_rate = 0.01
print(f"Starting weight: {weight:.4f}")
print(f"Target weight: 2.0000")
print()
# Training loop
for epoch in range(100):
total_loss = 0
for x, target in training_data:
# Forward pass: make prediction
prediction = weight * x
# Calculate error
error = target - prediction
total_loss += error ** 2
# Update weight (gradient descent, simplified)
weight += learning_rate * error * x
avg_loss = total_loss / len(training_data)
if epoch % 20 == 0 or epoch == 99:
print(f"Epoch {epoch+1:3d}: weight={weight:.4f}, loss={avg_loss:.6f}")
print()
print(f"Final learned weight: {weight:.4f}")
print(f"Expected weight: 2.0000")
print()
print(f"Prediction for x=6: {weight * 6:.2f} (expected: 12)")
print(f"Prediction for x=7: {weight * 7:.2f} (expected: 14)")Try It Yourself
Tip
Tip
The learning rate controls how large each step is during gradient descent. Too large: the network jumps around and may never converge. Too small: the network learns very slowly. Finding the right learning rate is one of the most important hyperparameter choices in training.