Beginner-Friendly Topic
Take your time - it's perfectly normal to re-read this topic 2-3 times. Try the interactive code editor below to run code yourself. Use the Q&A section to check your understanding before moving on. You've got this! 🚀
Why Good Data Matters
The single most important factor in building a good AI system is the quality of its training data. An AI is only as good as the data it learns from.
Data Quality Principles
There is a saying in AI: garbage in, garbage out.
If you train an AI on bad data, it learns bad patterns. If you train an AI on biased data, it learns biased patterns.
A facial recognition system trained only on one demographic will perform poorly on others. A medical AI trained on data from one hospital may not work well in a different hospital. A translation AI trained on formal text may struggle with casual conversation.
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
What Makes Data Good or Bad
- Good data is representative: it includes examples from all the situations the AI will face in real use
- Good data is accurate: the labels are correct. Wrong labels teach wrong patterns
- Good data is diverse: it includes edge cases and unusual situations, not just common ones
- Good data is balanced: it has enough examples of each category, not overwhelmingly from one group
- Bad data is biased: over-represents some groups and under-represents others
- Bad data has errors: incorrect labels, missing values, or duplicate examples
Real-World Data Quality Problem
# Demonstrating the effect of imbalanced data
# Scenario: fraud detection
# 99% of transactions are real, 1% are fraud
transaction_data = [
# (amount, late_night, foreign_country) -> is_fraud
(50, False, False, False), # not fraud
(100, False, False, False), # not fraud
(80, False, False, False), # not fraud
# ... 99 more non-fraud examples
(5000, True, True, True), # fraud!
]
# A badly trained model on this data might just predict
# "not fraud" for everything and be 99% accurate!
# But it would miss every fraud case.
def naive_model(transaction):
# This model always says "not fraud"
# It achieves 99% accuracy but is completely useless
return "not_fraud"
# Illustration:
total = 100
fraud_count = 1
non_fraud_count = 99
naive_accuracy = non_fraud_count / total * 100
print(f"Naive model accuracy: {naive_accuracy}%")
print(f"Naive model fraud detection rate: 0%")
print()
print("High accuracy does not always mean useful AI!")
print("This is why data balance matters so much in real projects.")Tip
Tip
Before building any AI model, spend time understanding your data. Ask: Is it representative? Is it balanced? Are the labels accurate? Are there biases? Fixing data problems before training always gives better results than trying to fix a model after it has learned from bad data.
Key Takeaways
- The single most important factor in building a good AI system is the quality of its training data.
- Good data is representative: it includes examples from all the situations the AI will face in real use
- Good data is accurate: the labels are correct. Wrong labels teach wrong patterns
- Good data is diverse: it includes edge cases and unusual situations, not just common ones