Privacy and AI
AI systems consume vast amounts of personal data to train and operate. Managing this data responsibly is both an ethical obligation and, increasingly, a legal requirement under regulations like GDPR and CCPA.
Privacy Risks in AI
Training data exposure: AI models trained on personal data can inadvertently memorise specific examples. Large language models have reproduced verbatim passages including personal information from their training data.
Inference attacks: Even without sharing your personal data, your AI usage patterns can reveal sensitive information. Your health condition might be inferred from the questions you ask an AI.
Model inversion: An adversary with access to a trained model can sometimes reconstruct approximations of the training data by probing the model with carefully crafted inputs.
Re-identification: Combining AI-generated insights with other publicly available datasets can de-anonymise individuals who were supposedly anonymous.
Deep Learning ⊂ Machine Learning ⊂ Artificial Intelligence
Privacy Best Practices for AI
- Data minimisation: collect only the personal data you actually need for the specific task
- Purpose limitation: do not use data collected for one purpose for a different unrelated purpose
- Anonymisation: remove or replace identifying information (names, emails, IDs) where possible
- Differential privacy: add carefully calibrated noise to training data or model outputs to prevent individual records from being reconstructed
- Federated learning: train models on data that stays on users' devices—only aggregated model updates, never raw data, are sent to a central server
- Access controls: limit who can access raw training data and who can query the deployed model
- Data retention limits: delete training data when it is no longer needed for model maintenance
Privacy Assessment Checklist
# Privacy checklist for any AI data collection
def privacy_assessment(dataset_name):
"""Run a privacy checklist before using a dataset for AI training."""
checklist = [
("Consent", "Did the people whose data is used give informed consent for this AI use?"),
("Minimisation", "Does the dataset contain ONLY the fields needed for this specific task?"),
("Anonymisation", "Have direct identifiers (name, email, phone, ID numbers) been removed?"),
("Re-ID risk", "Could records be re-identified by combining with other available datasets?"),
("Purpose", "Is the data being used only for the original stated purpose?"),
("Legal basis", "Does this collection and use comply with GDPR, CCPA, or applicable laws?"),
("Retention", "Is there a documented plan to delete data when no longer needed?"),
("Access control", "Is access to raw data restricted to authorised personnel only?"),
]
print(f"Privacy Assessment: {dataset_name}")
print()
print("Items to verify before training:")
print()
for item, question in checklist:
print(f" [ ] {item}")
print(f" {question}")
print()
print("Every unchecked item is a privacy risk. Resolve before proceeding.")
privacy_assessment("Customer support chat logs for chatbot training")Key Takeaways
- AI systems consume vast amounts of personal data to train and operate.
- Data minimisation: collect only the personal data you actually need for the specific task
- Purpose limitation: do not use data collected for one purpose for a different unrelated purpose
- Anonymisation: remove or replace identifying information (names, emails, IDs) where possible