Context
Context is everything the LLM can 'see' when generating a response. The context window is the maximum amount of text the model can process at once.
What Context Includes
When you chat with an LLM, the full context sent to the model includes:
- The system prompt (instructions about the model's role and behavior)
- The entire conversation history (all previous messages)
- Your current message
The model has no memory between separate conversations. Each new conversation starts fresh. What seems like 'memory' is just the conversation history being included in the context.
This means: if a conversation gets very long, older parts may be removed from the context to make room. The model then effectively forgets early parts of the conversation.
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
Context Window Limits
- Every LLM has a maximum context window (maximum tokens it can process at once)
- If your input is longer than the context window, some content is truncated or chunked
- Larger context windows are useful for: analyzing long documents, maintaining long conversations, processing large codebases
- Context window sizes are growing: early GPT-3 had 4,096 tokens, Gemini 1.5 Pro supports 1,000,000
- Within the context window, the model can pay attention to any token when generating each response token
Context Management Example
# Illustrating context management in a conversation
class LLMConversation:
def __init__(self, max_context_tokens=100, system_prompt=""):
self.system_prompt = system_prompt
self.history = []
self.max_context_tokens = max_context_tokens
def estimate_tokens(self, text):
return len(text.split()) # simplified: 1 word = 1 token
def add_message(self, role, content):
self.history.append({"role": role, "content": content})
self._trim_to_fit()
def _trim_to_fit(self):
"""Remove oldest messages if context is too long."""
while self._total_tokens() > self.max_context_tokens and len(self.history) > 1:
removed = self.history.pop(0)
print(f" [Context trimmed: removed old message from {removed['role']}]")
def _total_tokens(self):
total = self.estimate_tokens(self.system_prompt)
for msg in self.history:
total += self.estimate_tokens(msg["content"])
return total
def show_context(self):
print(f"Context status: {self._total_tokens()}/{self.max_context_tokens} tokens")
print(f"Messages in context: {len(self.history)}")
# Simulate a conversation
conversation = LLMConversation(
max_context_tokens=50,
system_prompt="You are a helpful AI assistant."
)
messages = [
("user", "Hello, what is your name?"),
("assistant", "I am an AI assistant here to help you learn."),
("user", "Can you explain what machine learning is?"),
("assistant", "Machine learning is a type of AI that learns from data."),
("user", "What is deep learning?"),
("assistant", "Deep learning uses neural networks with many layers."),
("user", "Now, what was my first question?"), # Model may have forgotten!
]
print("Conversation with Context Window Limits:")
print()
for role, content in messages:
print(f" {role.upper()}: {content}")
conversation.add_message(role, content)
conversation.show_context()
print()Key Takeaways
- Context is everything the LLM can 'see' when generating a response.
- Every LLM has a maximum context window (maximum tokens it can process at once)
- If your input is longer than the context window, some content is truncated or chunked
- Larger context windows are useful for: analyzing long documents, maintaining long conversations, processing large codebases