Hierarchical Clustering - Dendrograms
Hierarchical clustering builds a tree of clusters (dendrogram) by successively merging the two most similar clusters (agglomerative) or splitting the largest cluster (divisive). Unlike K-Means, you don't need to specify K upfront - the dendrogram shows all possible clusterings and you cut at the desired level. Excellent for small datasets where visual exploration matters.
Agglomerative Clustering and Dendrogram
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.cluster import AgglomerativeClustering
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_blobs
from scipy.cluster.hierarchy import dendrogram, linkage
from scipy.spatial.distance import squareform
np.random.seed(42)
X, _ = make_blobs(n_samples=150, centers=4, cluster_std=0.8, random_state=42)
X_sc = StandardScaler().fit_transform(X)
# LINKAGE METHODS
fig, axes = plt.subplots(1, 4, figsize=(20, 5))
for ax, linkage_method in zip(axes, ["ward", "complete", "average", "single"]):
Z = linkage(X_sc, method=linkage_method)
dendrogram(Z, ax=ax, no_labels=True, color_threshold=2.0)
ax.set_title(f"Linkage: {linkage_method}")
ax.set_ylabel("Distance")
ax.set_xlabel("Samples")
plt.suptitle("Hierarchical Clustering Dendrograms -- cut at big jump = cluster number", fontsize=13)
plt.tight_layout()
plt.savefig("dendrograms.png", dpi=100, bbox_inches="tight")
plt.show()
# CHOOSING K FROM DENDROGRAM
Z = linkage(X_sc, method="ward")
distances = Z[:, 2]
# Large jumps in merge distances suggest natural clusters
gaps = np.diff(distances[::-1])
print("Merge distances (largest 5 gaps -> natural cluster boundaries):")
top_gaps = np.argsort(gaps)[::-1][:5]
for i, gap_idx in enumerate(top_gaps, 1):
print(f" Gap {i}: between merging to {X_sc.shape[0] - gap_idx - 1} and {X_sc.shape[0] - gap_idx - 2} clusters -- distance jump: {gaps[gap_idx]:.3f}")
# FIT AGGLOMERATIVE CLUSTERING
for n_clusters in [3, 4, 5]:
agg = AgglomerativeClustering(n_clusters=n_clusters, linkage="ward")
labels = agg.fit_predict(X_sc)
from sklearn.metrics import silhouette_score
sil = silhouette_score(X_sc, labels)
print(f" n_clusters={n_clusters}: silhouette={sil:.4f}")
# LINKAGE METHOD GUIDE
linkage_guide = {
"ward": "Default -- minimizes within-cluster variance. Best for compact spherical clusters.",
"complete": "Maximum distance between two cluster members. Biased toward compact clusters.",
"average": "Average distance between all pairs. Compromise between ward and single.",
"single": "Minimum distance. Sensitive to outliers, creates 'chaining' effect.",
}
print("\nLinkage method guide:")
for method, desc in linkage_guide.items():
print(f" {method:10s}: {desc}")Tip
Tip
Practice Hierarchical Clustering Dendrograms in small, isolated examples before integrating into larger projects. Breaking concepts into small experiments builds genuine understanding faster than reading alone.
Neural networks learn by adjusting connection weights via backpropagation
Practice Task
Note
Practice Task - (1) Write a working example of Hierarchical Clustering Dendrograms from scratch without looking at notes. (2) Modify it to handle an edge case (empty input, null value, or error state). (3) Share your solution in the Priygop community for feedback.
Quick Quiz
Common Mistake
Warning
A common mistake with Hierarchical Clustering Dendrograms is skipping edge case testing - empty inputs, null values, and unexpected data types. Always validate boundary conditions to write robust, production-ready ml code.