Beginner-Friendly Topic
Take your time - it's perfectly normal to re-read this topic 2-3 times. Try the interactive code editor below to run code yourself. Use the Q&A section to check your understanding before moving on. You've got this! 🚀
Training Data
Training data is the raw material that Generative AI learns from. The quality and quantity of training data directly affects the quality of what the AI can create.
What is Training Data?
Training data is the collection of examples that an AI model studies during training.
For a text model like GPT:
- Books and articles across every topic
- Websites and web pages
- Code repositories
- Scientific papers
- Conversations
For an image model like Stable Diffusion:
- Millions of images with text descriptions
- Photographs, illustrations, artwork
- Each image paired with a caption describing what it shows
For an audio model:
- Voice recordings paired with written transcripts
- Music with genre and style labels
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
Why Training Data Quality Matters
- If the training data contains errors, the model will learn to make the same types of errors
- If the training data is biased (for example, mostly content from one region), the model will reflect that bias
- If the training data is outdated, the model's knowledge will be outdated
- If the training data is too small, the model will not learn broad enough patterns to be useful
- If the training data contains harmful content, the model may reproduce harmful outputs
Tip
Tip
This is why Generative AI sometimes produces wrong or outdated information. The model can only know what was in its training data. If something happened after the training cutoff date, the model simply does not know about it. Always verify important facts from a Generative AI response with an authoritative source.
Key Takeaways
- Training data is the raw material that Generative AI learns from.
- If the training data contains errors, the model will learn to make the same types of errors
- If the training data is biased (for example, mostly content from one region), the model will reflect that bias
- If the training data is outdated, the model's knowledge will be outdated