Beginner-Friendly Topic
Take your time - it's perfectly normal to re-read this topic 2-3 times. Try the interactive code editor below to run code yourself. Use the Q&A section to check your understanding before moving on. You've got this! 🚀
Training vs Inference
Training and inference are the two main phases of an AI model's life. Knowing the difference explains why AI models have a knowledge cutoff date and why they do not learn from your conversations.
Side by Side Comparison
- Training: happens once before the model is released to users. Inference: happens every time a user interacts with the model
- Training: takes weeks using thousands of powerful computers. Inference: takes seconds on specialized hardware
- Training: costs millions of dollars in computing resources. Inference: costs a fraction of a cent per response
- Training: the model learns and its parameters change. Inference: the model's parameters stay fixed
- Training: uses the entire training dataset. Inference: only uses the current context window
- Training: determines everything the model knows. Inference: applies that knowledge to a new input
Why This Matters for Users
Most AI chat tools do not learn from your conversations. When you close the chat window and start a new conversation, the AI remembers nothing from before. This is because the model's parameters are fixed after training. Your conversation only exists temporarily in the context window.
This also explains the knowledge cutoff date. If a model's training data was collected up to a certain date, it will not know about anything that happened after that date, because it never saw that information during training.
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
Key Takeaways
- Training and inference are the two main phases of an AI model's life.
- Training: happens once before the model is released to users. Inference: happens every time a user interacts with the model
- Training: takes weeks using thousands of powerful computers. Inference: takes seconds on specialized hardware
- Training: costs millions of dollars in computing resources. Inference: costs a fraction of a cent per response