Embeddings Explained Simply
Embeddings are the technology that makes semantic search possible in RAG systems. They convert text into lists of numbers that capture the meaning of the text, allowing the system to find documents that are conceptually related to a question.
What is an Embedding?
An embedding is a list of numbers that represents the meaning of a piece of text.
A simple analogy:
Imagine you could describe any word by its position in a 3D space:
- 'King' is at position (8, 3, 7)
- 'Queen' is at position (7, 3, 7) (similar to King in most dimensions)
- 'Car' is at position (2, 9, 1) (very different from King)
Real embeddings work the same way but with hundreds or thousands of dimensions instead of just 3. Words and phrases with similar meanings end up close together in this space.
This means that 'automobile' and 'car' will have similar embeddings, even though they are different words. Semantic search uses this to find relevant documents even when the exact words differ.
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
Using Embeddings in Python
# Creating text embeddings with the OpenAI API
# These are used for semantic search in RAG systems
import os
from openai import OpenAI
def create_embedding(text):
"""
Convert text into an embedding (a list of numbers representing meaning).
The OpenAI text-embedding-3-small model produces 1536-dimensional embeddings.
Each text is represented as a list of 1536 numbers.
"""
api_key = os.environ.get("OPENAI_API_KEY")
if not api_key:
return None
client = OpenAI(api_key=api_key)
response = client.embeddings.create(
model="text-embedding-3-small",
input=text
)
# The embedding is a list of 1536 floating point numbers
embedding_vector = response.data[0].embedding
return embedding_vector
# Concept illustration
print("Embedding concept illustration:")
print()
texts_to_embed = [
"How do I return a product?", # User's question
"Our return policy allows returns within 30 days.", # Document 1 (relevant)
"Shipping takes 3-5 business days.", # Document 2 (not relevant)
"What is the refund process?", # Another question (semantically similar)
]
for text in texts_to_embed:
print(f"Text: '{text}'")
print(f" -> Converted to a vector of ~1536 numbers representing its meaning")
print()
print("The RAG system compares the question's embedding")
print("to each document chunk's embedding.")
print("The most similar documents are retrieved.")
print()
print("'How do I return a product?' and 'Our return policy allows returns within 30 days.'")
print("would have HIGH similarity -> Document 1 is retrieved")
print()
print("'How do I return a product?' and 'Shipping takes 3-5 business days.'")
print("would have LOW similarity -> Document 2 is not retrieved")