Production Checklist

Production Readiness Checklist

• SECURITY: All API keys in a secrets manager, not in code or environment files
• SECURITY: Least-privilege permissions configured for all agent roles
• SECURITY: All sensitive actions protected by human approval gates
• SECURITY: Audit logging enabled and writing to an immutable store
• SECURITY: Prompt injection defences in place
• SECURITY: Action restrictions enforced (financial limits, bulk action guards)
• RELIABILITY: Retry logic with exponential backoff on all external calls
• RELIABILITY: Fallback strategies defined for every critical tool
• RELIABILITY: Loop detection with a hard step limit
• RELIABILITY: Timeout set on every tool call and the full task
• RELIABILITY: Idempotency keys used on all write operations
• RELIABILITY: Human escalation path works end-to-end
• OBSERVABILITY: Structured logging on every node
• OBSERVABILITY: Execution traces captured and stored
• OBSERVABILITY: Key metrics tracked — success rate, cost, latency, escalation rate
• OBSERVABILITY: Alerting configured for success rate drops, cost spikes, and errors
• TESTING: Unit tests for all tool validation and workflow logic
• TESTING: Integration tests with mocked tools cover happy path and key failure cases
• TESTING: Evaluation dataset of 50+ cases with known expected outcomes
• TESTING: All regression tests passing
• TESTING: Security tests confirm permission boundaries are enforced
• DEPLOYMENT: Separate staging environment tested and confirmed working
• DEPLOYMENT: Load tested at 2x expected peak traffic
• DEPLOYMENT: Rollback procedure documented and tested
• DEPLOYMENT: On-call runbook written and reviewed
• DEPLOYMENT: Cost budget alerts configured