Image Segmentation
Master image segmentation techniques — from semantic segmentation to instance segmentation — for pixel-level understanding of images. This is a foundational concept in artificial intelligence and machine learning that professional developers rely on daily. The explanations below are written to be beginner-friendly while covering the depth and nuance that comes from real-world AI/ML experience. Take your time with each section and practice the examples
50 min•By Priygop Team•Last updated: Feb 2026
Types of Image Segmentation
- Semantic Segmentation: Labels every pixel with a class (sky, road, car, person) — all cars are treated as one 'car' class with no individual distinction
- Instance Segmentation: Labels every pixel AND distinguishes individual objects — Car 1 vs Car 2 vs Car 3. Combines detection + segmentation
- Panoptic Segmentation: Unifies semantic + instance segmentation — labels both 'stuff' (sky, road) and 'things' (individual cars, people)
- Applications: Medical imaging (tumor boundaries), autonomous driving (drivable area), satellite imagery (land use), AR/VR (background removal), video editing (rotoscoping)
Segmentation Architectures
- FCN (Fully Convolutional Network): Replaced FC layers with convolutions — first end-to-end segmentation network. Uses upsampling to produce pixel-level predictions
- U-Net (2015): Encoder-decoder with skip connections — designed for medical imaging. The skip connections preserve fine-grained spatial information
- DeepLab (v1-v3+): Uses atrous/dilated convolutions for larger receptive fields without losing resolution, plus CRF post-processing for sharp boundaries
- Mask R-CNN: Extends Faster R-CNN with a mask branch — predicts segmentation mask for each detected instance. Foundation for instance segmentation
- SAM (Segment Anything Model, 2023): Foundation model for segmentation — can segment any object in any image with prompts (points, boxes, text). Trained on 11M images
- SegFormer (2021): Transformer-based segmentation — hierarchical encoder with lightweight MLP decoder. Efficient and accurate