Beginner-Friendly Topic
Take your time - it's perfectly normal to re-read this topic 2-3 times. Try the interactive code editor below to run code yourself. Use the Q&A section to check your understanding before moving on. You've got this! 🚀
Model Inference
Inference is what happens when you actually use a Generative AI model. You provide a prompt, and the model uses its trained parameters to generate a response.
What is Inference?
Inference is the process of using a trained AI model to generate an output for a new input.
Training: the model learns from data (happens once, takes weeks).
Inference: the model applies its learning to a new prompt (happens for every user interaction, takes seconds).
When you type a message to ChatGPT and press send, that is inference. The model is not learning new things from your message. It is applying everything it already learned to generate a response to your specific prompt.
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
Inference in Simple Steps
- Step 1: You type a prompt (e.g., 'Explain photosynthesis simply')
- Step 2: Your prompt is converted into tokens (small pieces of text the model understands)
- Step 3: The tokens are fed into the model
- Step 4: The model processes the tokens through its layers using its trained parameters
- Step 5: The model produces output tokens one at a time
- Step 6: The output tokens are converted back into human-readable text
- Step 7: You see the response appear on screen
Key Takeaways
- Inference is what happens when you actually use a Generative AI model.
- Step 1: You type a prompt (e.g., 'Explain photosynthesis simply')
- Step 2: Your prompt is converted into tokens (small pieces of text the model understands)
- Step 3: The tokens are fed into the model