Annual AI Audit Template

1. Purpose and Ecosystem Position

SUS-05 defines the annual, organisation-wide audit for all deployed legal AI systems. It validates three integrated dimensions:

• Compliance with professional rules, regulations, privacy and vendor obligations.
• Bias and fairness using the GOV-04 methodology and protected class analysis.
• Performance and ROAI against accuracy, uptime and value benchmarks.

The audit is the sustaining governance backbone of the Legal AI OS and connects to other instruments:

• SUS-05 (annual audit) → GOV-03 (risk register updates)
• → STR-07 (critical finding escalation)
• → DPS (annual defensibility evidence refresh)
• → VEN-01 (vendor score updates)
• → MAT-01 (governance maturity update)

Triggers: annual governance cycle, plus significant AI incidents, regulatory inquiries, new high-risk deployments, or material vendor changes.

---

2. Risk Taxonomy 2026 Coverage

The audit validates compliance against all nine Risk Taxonomy 2026 classes, with minimum evidence standards:

1. Class 1: Hallucination and accuracy – measured accuracy vs. thresholds by complexity.
2. Class 2: Privilege and confidentiality – data isolation tests, privilege audit trails, access logs.
3. Class 3: Bias and fairness – GOV-04 statistical tests and fairness metrics.
4. Class 4: Privacy and data protection – GDPR/CCPA checks and DAT-03 DPA currency.
5. Class 5: Supply chain and vendor dependency – sub-processor chains, AI BoM reconciliation.
6. Class 6: Shadow AI – discovery scans and inventory reconciliation.
7. Class 7: Regulatory compliance drift – ABA, EU AI Act, state bar and sector rules.
8. Class 8: IP and licensing – output ownership and training data licensing.
9. Class 9: Operational resilience – uptime, incidents, and Agentic Tier governance.

Any severity High failure in any class is a critical finding and must be escalated via STR-07 with a remediation plan within 30 days.

---

3. Three-Pillar Assessment Approach

3.1 Compliance Audit

Scope:

• ABA Model Rules 1.1, 1.4, 1.6, 5.3.
• EU AI Act, GDPR/CCPA, state AI laws, sector regulations.
• DAT-03 DPA currency and scope coverage.
• AI BoM reconciliation and shadow AI detection.
• Vendor compliance currency (SOC 2, ISO 27001) and VEN-04 triggers.
• Policy adherence to GOV-01 (governance framework) and GOV-02 (AI use policy).

3.2 Bias Assessment