Beginner-Friendly Topic
Take your time - it's perfectly normal to re-read this topic 2-3 times. Try the interactive code editor below to run code yourself. Use the Q&A section to check your understanding before moving on. You've got this! 🚀
The Perceptron - Building Block of All Neural Networks
A perceptron is the simplest neural network - one neuron. It takes inputs, multiplies each by a weight, sums them up, adds a bias, and passes through an activation function. Every modern transformer with billions of parameters is built from millions of these.
Perceptron from Scratch
import torch
import torch.nn as nn
import numpy as np
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# SINGLE NEURON (Perceptron) - implemented manually
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
class Perceptron:
"""A single neuron: output = activation(W·X + b)"""
def __init__(self, n_inputs: int):
# Random initialization - critical to break symmetry
self.W = np.random.randn(n_inputs) * 0.01 # weights
self.b = 0.0 # bias
def forward(self, X: np.ndarray) -> float:
"""Compute weighted sum + bias → activation."""
z = np.dot(self.W, X) + self.b # linear combination: z = W1x1 + W2x2 + ... + b
return self._sigmoid(z) # squash to [0,1]
def _sigmoid(self, z: float) -> float:
"""Maps any real number to (0,1). Used for binary classification."""
return 1 / (1 + np.exp(-z))
def train_step(self, X: np.ndarray, y_true: float, lr: float = 0.01) -> float:
"""One gradient descent step."""
y_pred = self.forward(X)
error = y_true - y_pred
# Gradient of loss w.r.t weights and bias (chain rule simplified for sigmoid)
self.W += lr * error * y_pred * (1 - y_pred) * X
self.b += lr * error * y_pred * (1 - y_pred)
loss = 0.5 * error**2 # MSE loss
return loss
# Demonstrate: learn AND gate
p = Perceptron(n_inputs=2)
AND_data = [([0,0], 0), ([0,1], 0), ([1,0], 0), ([1,1], 1)]
for epoch in range(1000):
total_loss = 0
for X, y in AND_data:
total_loss += p.train_step(np.array(X, dtype=float), y, lr=0.1)
print("AND gate learned:")
for X, y_true in AND_data:
pred = p.forward(np.array(X, dtype=float))
print(f" Input {X} → Pred: {pred:.3f} (True: {y_true})")
# WHY SINGLE PERCEPTRON FAILS:
# A single perceptron can only learn LINEARLY SEPARABLE problems.
# XOR is NOT linearly separable → single perceptron cannot learn it.
# Solution: Multi-Layer Perceptron (MLP) = stack multiple neuronsTip
Tip
Practice The Perceptron Building Block of All Neural Networks in small, isolated examples before integrating into larger projects. Breaking concepts into small experiments builds genuine understanding faster than reading alone.
Input → Hidden layers → Output. Train via backpropagation.
Practice Task
Note
Practice Task - (1) Write a working example of The Perceptron Building Block of All Neural Networks from scratch without looking at notes. (2) Modify it to handle an edge case (empty input, null value, or error state). (3) Share your solution in the Priygop community for feedback.
Quick Quiz
Common Mistake
Warning
A common mistake with The Perceptron Building Block of All Neural Networks is skipping edge case testing - empty inputs, null values, and unexpected data types. Always validate boundary conditions to write robust, production-ready ai code.