What is Deep Learning?
Deep learning is a type of machine learning that uses neural networks with many layers. The term 'deep' refers to the depth of these layers. Deep learning has powered the most impressive AI breakthroughs of the past decade.
What Makes it Deep
In Module 4, we looked at a simple neural network with one or two layers. Deep learning means using many more layers, often dozens or even hundreds.
Each additional layer allows the network to learn more abstract and complex representations:
- Shallow layer: detects simple patterns (edges in images, common words in text)
- Middle layers: combine simple patterns into more complex ones (shapes, phrases)
- Deep layers: recognize high-level concepts (faces, emotions, meaning)
This hierarchy of learned representations is what makes deep learning so powerful.
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
What Deep Learning Can Do
- Image recognition: identify objects, people, and scenes in photos with near-human accuracy
- Speech recognition: convert spoken words to text (used in voice assistants)
- Language translation: translate between over 100 languages in real time
- Text generation: write articles, code, and conversations (ChatGPT uses this)
- Image generation: create photorealistic images from text descriptions (Midjourney, DALL-E)
- Medical imaging: detect cancer and other diseases in X-rays and MRI scans
- Protein structure prediction: AlphaFold predicted the structure of almost every known protein
Deep vs Shallow Network Comparison
# Illustrating shallow vs deep network structure
def describe_network(name, layer_sizes):
print(f"{name}:")
total_params = 0
for i in range(len(layer_sizes) - 1):
params = layer_sizes[i] * layer_sizes[i+1]
total_params += params
print(f" Layer {i+1}: {layer_sizes[i]} -> {layer_sizes[i+1]} ({params} connections)")
print(f" Total connections: {total_params}")
print()
# Shallow network (1 hidden layer)
describe_network("Shallow Network", [784, 128, 10])
# Deep network (many hidden layers)
describe_network("Deep Network", [784, 512, 256, 128, 64, 10])
print("The deep network has more layers to learn hierarchical features.")
print("More layers -> more expressive power -> harder tasks.")
print()
print("But: more layers also means:")
print(" - Needs more data to train properly")
print(" - Requires more computing power (GPUs)")
print(" - Harder to train without modern techniques")Quick Q&A
Key Takeaways
- Deep learning is a type of machine learning that uses neural networks with many layers.
- Image recognition: identify objects, people, and scenes in photos with near-human accuracy
- Speech recognition: convert spoken words to text (used in voice assistants)
- Language translation: translate between over 100 languages in real time