Limitations of Generative AI

Core Limitations

• Hallucination: generates convincing but factually incorrect information. A model may cite a paper that does not exist
• Bias: trained data reflects human biases. The model may perpetuate or amplify stereotypes
• Knowledge cutoff: trained on data up to a certain date. Does not know about recent events without search tools
• Context window: can only work with a limited amount of text at once. Long documents must be chunked
• No true understanding: generates statistically likely responses, not responses based on genuine comprehension
• Inconsistency: may give different answers to the same question asked in different ways
• Copyright: some generated content may closely resemble copyrighted training material

How to Work Around Limitations

# Strategies for working with Generative AI limitations

limitations_and_workarounds = {
    "Hallucination": [
        "Always verify specific facts, statistics, and citations",
        "Ask the model to indicate uncertainty ('say you are not sure if unsure')",
        "Use models with search/retrieval tools for factual questions",
    ],
    "Knowledge cutoff": [
        "For recent events, provide the relevant context in your prompt",
        "Use models with internet search capabilities for current information",
        "Specify 'as of [date]' when you need current information",
    ],
    "Inconsistency": [
        "Run the same prompt multiple times and compare outputs",
        "Ask the model to reason step by step (chain-of-thought prompting)",
        "Provide examples of the format and quality you expect",
    ],
    "Context window limits": [
        "Summarize long documents before asking questions about them",
        "Use retrieval-augmented generation (RAG) for large knowledge bases",
        "Break complex tasks into smaller, sequential steps",
    ],
}

print("Working Around Generative AI Limitations:")
print()
for limitation, strategies in limitations_and_workarounds.items():
    print(f"Limitation: {limitation}")
    for strategy in strategies:
        print(f"  - {strategy}")
    print()

Key Takeaways from This Module

• Generative AI creates new content: text, images, audio, and video
• It differs from discriminative AI which classifies or predicts existing data
• Text generation works by predicting likely next tokens based on patterns learned from training data
• Image generation uses diffusion: learning to reverse the process of adding noise to images
• Audio generation creates speech, music, and sound effects from text or audio samples
• Video generation is advancing rapidly but remains computationally expensive
• Modern chatbots generate responses one token at a time through Transformer layers
• Key limitations: hallucination, bias, knowledge cutoff, inconsistency