Batch Size
Batch size is the number of training examples processed together in one step. It affects training speed, memory usage, and the quality of learning.
What is Batch Size?
Instead of calculating the gradient (weight update direction) after every single example, deep learning processes a batch of examples together.
Small batch (e.g., 16 or 32): updates weights frequently with noisy estimates. Slower per epoch but can reach good solutions.
Large batch (e.g., 256 or 512): updates weights less frequently but with more accurate estimates. Faster per epoch but may need more memory.
The most common batch sizes are 32, 64, and 128. These are chosen as powers of 2 because they work efficiently with GPU hardware.
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
Batch Size Trade-offs
- Smaller batch: more updates per epoch, noisier gradients, sometimes finds better solutions
- Larger batch: fewer updates per epoch, smoother gradients, faster training per step
- Larger batch needs more GPU memory (can cause out-of-memory errors)
- A batch size of 32 is a safe default for most problems
- If training is slow, try larger batches. If accuracy is poor, try smaller batches
Tip
Tip
If you get an 'out of memory' error when training a deep learning model, reducing the batch size is the first thing to try. Cutting batch size in half roughly halves the memory usage.
Key Takeaways
- Batch size is the number of training examples processed together in one step.
- Smaller batch: more updates per epoch, noisier gradients, sometimes finds better solutions
- Larger batch: fewer updates per epoch, smoother gradients, faster training per step
- Larger batch needs more GPU memory (can cause out-of-memory errors)