Hallucination

Hallucination (Risk Taxonomy 2026 Summary)

Concept

Hallucination is when a generative AI system produces content that looks factual but is wholly or partly fabricated. In legal and regulated settings, this includes:

• Invented or misformatted case citations
• Non-existent statutes or regulations
• Misattributed or fabricated quotations
• Confident factual statements that are not grounded in any provided source

This is not a bug in intent but a structural property of probabilistic text generation: models are optimized for plausible language, not for truth. As a result, hallucination is an inherent risk that must be controlled, not a defect that can be fully eliminated.

Why It Is a Distinct Risk Class

Under Risk Taxonomy 2026, Hallucination is its own class because:

1. Unavoidable in generative systems
2. Asymmetric harm profile
   • One fabricated citation in a court filing can trigger sanctions, reputational damage, and client harm.
   • High volumes of correct outputs do not compensate for a single high-impact hallucination.
3. Requires specific controls

Canonical Control Set

A defensible governance programme treats Hallucination as a controlled exposure with documented mitigations, including:

• Retrieval grounding
  • Model outputs must cite specific sources (documents, cases, statutes).
  • Cited sources must be accessible to reviewers for direct inspection.
• Citation and fact verification
  • Every legal authority, statute, regulation, or key factual assertion is checked against the cited source before being relied on.
  • Verification can be manual, tool-assisted, or automated, but must be traceable.
• Structured outputs
  • Use schemas (e.g., JSON, tables, forms) instead of free-form prose where possible.
  • Force the model to populate discrete, verifiable fields (case name, reporter, jurisdiction, date, statute section, URL, etc.).
• Hallucination-rate telemetry
  • Define a measurable unit (e.g., “percentage of outputs containing ≥1 ungrounded claim”).
  • Continuously sample and score outputs.
  • Report hallucination rates to governance and risk owners, with thresholds for escalation.
• Tier-appropriate human review
  • Tier 1 – Augmentation (e.g., drafting aids): all materially consequential uses require human review.
  • Tier 2 – Co-pilot (e.g., research assistants): qualified humans must validate authorities and facts before filing or client use.
  • Tier 3–4 – Higher autonomy tools: require stringent pre-deployment grounding, automated checks, and sharply limited scopes of action.

Relationship to Other Risk Classes

Hallucination is interacting with but distinct from other risks:

• Professional conduct exposure
  • When a lawyer or judge relies on hallucinated content, the downstream issue is also a professional responsibility and ethics failure.
• Regulatory non-compliance
  • Hallucinated references to regulations or supervisory guidance in official submissions can create direct compliance violations.
• Reduced supervisory capacity
  • Under-resourced or poorly designed review processes increase the chance that hallucinations reach the record or affect decisions.

By treating Hallucination as a standalone risk class—rather than a generic “model quality” issue—governance can:

• Assign explicit risk owners and control owners
• Define control objectives (e.g., maximum tolerated hallucination rate by use case tier)
• Implement monitoring and reporting specific to this failure mode

This framing enables organizations to treat hallucination as a managed, auditable exposure rather than an informal, ad hoc concern.