Beginner-Friendly Topic
Take your time - it's perfectly normal to re-read this topic 2-3 times. Try the interactive code editor below to run code yourself. Use the Q&A section to check your understanding before moving on. You've got this! 🚀
Context Window
The context window is the maximum amount of text an LLM can process at once. Everything outside the context window is invisible to the model. Understanding this explains the limitations of long conversations.
Context Window in Practice
- Context window sizes as of mid-2024: GPT-4o (128,000 tokens), Gemini 1.5 Pro (1,000,000 tokens), Claude 3.5 (200,000 tokens)
- 128,000 tokens is approximately 100,000 words, or a medium-length novel
- If a conversation exceeds the context window, the model starts losing memory of early messages
- Larger context windows allow longer documents and conversations to be processed at once
- Processing more tokens in a context window costs more and takes longer
Tip
Tip
When working with very long documents, start a new conversation for each major topic rather than having one very long conversation. This ensures all your context is fresh and relevant. Alternatively, paste only the relevant section of a document into your prompt rather than the entire document.
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
Key Takeaways
- The context window is the maximum amount of text an LLM can process at once.
- Context window sizes as of mid-2024: GPT-4o (128,000 tokens), Gemini 1.5 Pro (1,000,000 tokens), Claude 3.5 (200,000 tokens)
- 128,000 tokens is approximately 100,000 words, or a medium-length novel
- If a conversation exceeds the context window, the model starts losing memory of early messages