The operational problem
Not every AI-assisted decision carries the same consequence. A contract-summary recommendation and a contract-execution authorisation operate at different thresholds; the AI capability that recommends should not require Full HITL on every transaction, and the AI capability that executes must require it. Without an explicit calibration, every function defaults to one of two failure modes: every decision routes through Full HITL (capability does not scale; ROAI collapses) or no decision routes through Full HITL (Class 1 and Class 9 exposure compounds).
The institutional standard for Defensible AI requires explicit, documented, refreshed threshold-setting. Not every threshold is captured in a vendor’s product configuration; not every threshold is regulatory; not every threshold is constant across decision types. The Materiality Calibration Methodology defines the institutional discipline that sets the threshold per decision type per Tier 3+ capability and refreshes the calibration on the cadence the performance record warrants.
Without it, every Tier 3+ capability operates under an implicit threshold that the function cannot defend if a decision is wrong.
Legal AI OS Relevance
The Materiality Calibration Methodology sits at the intersection of Pillar P4 (Governance, Risk & Defensible AI) and the Governance Layer (Layer G). It produces evidence for two Defensibility Elements:
- DE-2 Methodology transparency — the calibration register is itself the documented methodology by which thresholds are set and refreshed
- DE-4 Governance posture — the calibration is signed by the GC and ratified at the AI Task Force quarterly cadence
Anchoring to Bands 3, 4, and 5 (Integrated → Optimised → Defensible), the Module is required at Band 3 for Tier 3+ deployment, operated at quarterly cadence at Band 4, and integrated with the Evidence Register (GOV-13) at Band 5 such that every threshold has a contemporaneous proof of operation. Methodology v2026.1.
Pillar Alignment
Pillar 4 (Governance, Risk & Defensible AI) is the institutional capability that holds AI accountable through documented practice. The Materiality Calibration Methodology operationalises threshold-setting — for this decision type, in this capability, at this consequence level, the function requires X tier of human-in-the-loop review.
The Pillar 4 posture this Module advances: from implicit thresholds set by vendor product configuration to signed, dated, four-factor-scored calibration per decision type per capability, refreshed quarterly.
Operating Layer Impact
The Governance Layer (Layer G) is the institutional substrate that constrains how AI may be operated. The Materiality Calibration Methodology specifies the constraint per decision type. The Delegation-Authority Register (GOV-14) writes against the calibration; the Evidence Register (GOV-13) records contemporaneous evidence per calibrated decision class; the Governance Operating Cadence (GOV-15) refreshes the calibration at the quarterly cycle.
Without the calibration, every Layer-G artefact specifies what may be delegated without specifying when the delegation is overridden.
Maturity Band Relevance
The Module strengthens the Defensibility and Sophistication lenses of the Maturity Stack.
From Band
To Band
Defensibility-lens shift
Sophistication-lens shift
3 — Integrated
4 — Optimised
Implicit thresholds → signed calibration per decision type per capability
Single HITL pattern → tiered HITL appropriate to consequence
4 — Optimised
5 — Defensible
Calibration operated at quarterly cadence with performance-record feedback
Tiered HITL refreshed as performance and regulatory context shift
5 — Defensible
(sustaining)
Calibration is sample-attested annually; integrated with Evidence Register
Tiered HITL durable across capability churn
Tier 3 (Workflow operator) deployment requires the calibration before Deploy authorisation. Tier 4 (Autonomous agent) deployment requires the calibration and the Agentic Governance Charter (GOV-08) and the Delegation-Authority Register entry (GOV-14).
Operational Outcomes
Operating the Methodology produces five institutional artefacts:
Artefact
Purpose
DPS evidence stream
Materiality Calibration register — per decision type per Tier 3+ capability
The signed calibration with four-factor scoring rationale
DE-2 + DE-4 (primary)
HITL tier assignments — Full / Exception-triggered / Audit-only
The operating-tier per decision type
DE-4
DPS Decision Traceability section input
The calibration feeds Element 1 directly
DE-1
Delegation-Authority Register input
GOV-14 writes against the calibration row
DE-1 + DE-4
Quarterly refresh delta report
What changed in the calibration since last quarter and why
DE-5
Records retain for the regulator’s limitation period plus the lifecycle of every capability referenced.
Defensibility and Governance Considerations
The Module produces evidence for DE-2 (Methodology transparency) and DE-4 (Governance posture); the calibration also feeds DE-1 (Decision traceability) via the Delegation-Authority Register integration. The Risk Class primarily addressed is Class 9 (Accountability dilution); the calibration is the institutional response to Class 9 at the per-decision level.
For Tier 4 (Autonomous agent) capability the calibration requires:
- A row per decision class the capability may execute
- The four-factor score with rationale
- The HITL tier assignment
- The materiality threshold (financial, regulatory, client-impact) where Full HITL is mandatory regardless of class
- The escalation path when materiality threshold is crossed
- The revocation trigger when performance breaches the calibrated floor
- Quarterly refresh with performance-record review
For Tier 3 (Workflow operator) capability the calibration is lighter: one row per workflow with the same four-factor score and HITL tier, refreshed annually unless performance triggers interim review.
The editorial-independence attestation applies — the Module makes no vendor-specific recommendations.
Institutional Use Cases
Use case 1 — A 14-partner firm calibrating a Tier 3 contract-extraction capability. The capability operates against 6 decision types. The four-factor analysis: consequence severity (low — extraction is preparatory, not dispositive); reversibility (high — review gate catches errors); regulatory exposure (low — no licensed-activity boundary); institutional risk (low — partner review is standing). All 6 decisions calibrate as Exception-triggered HITL: extraction proceeds; flagged extractions route to partner review. Annual refresh shows performance within calibrated floor; no tier shift.
Use case 2 — A 200-lawyer in-house function calibrating a Tier 4 vendor-renewal triage system. The capability operates against 18 decision types. Four-factor analysis varies: routine-renewal-within-band (consequence: low; reversibility: medium; regulatory: low; institutional: low) calibrates as Audit-only. Renewal-with-price-change-beyond-band (consequence: medium; reversibility: low; regulatory: medium; institutional: medium) calibrates as Exception-triggered HITL. Renewal-with-vendor-substitution (consequence: high; reversibility: low; regulatory: high; institutional: high) calibrates as Full HITL regardless of any threshold. Materiality threshold of $50K caps the autonomous scope; any renewal above $50K is Full HITL regardless of decision class. Quarterly refresh at quarter +2 shifts one decision class from Audit-only to Exception-triggered HITL after performance dipped.
Use case 3 — A global energy GC office calibrating 11 Tier 3+ capabilities. The calibration register holds 137 in-scope decision class rows. Annual external attestation samples 14 calibrations across the year and verifies each against the four-factor evidence; one calibration revoked, two amended. The Audit Committee reviews the refresh delta report quarterly; the GC signs each refresh.
Recommended Stakeholders
RACI role
Stakeholders
Owner
General Counsel
Approvers
General Counsel · Audit Committee Chair · Finance Partner
Contributors
Risk and Compliance · AI Task Force
Informed
Board · CIO / CISO · External Counsel
The GC signs each calibration row. The Audit Committee Chair and Finance Partner approve at the threshold layer because materiality is inherently a financial-and-risk calibration. The AI Task Force operates the quarterly refresh.
Implementation Complexity
Dimension
Specification
First run per Tier 3+ capability
3 hours
Quarterly refresh per capability
1 hour
Cross-team dependencies
Finance (financial threshold setting) · Risk (regulatory-exposure assessment) · the Practice Group (decision-type taxonomy)
Self-serve viability
Partial — first run benefits from advisory for Tier 4; Tier 3 is self-serve with template
Advisory recommendation
Programme Design for first Tier 4 calibration; Strategic Retainer for functions operating ≥3 Tier 3+ capabilities
The Module is methodology-versioned (v2026.1); each row’s effective date and next review are the operating telemetry.
Inputs
Input
Source
Capability Portfolio Tier 3+ capability list
SUS-10
Decision type inventory per capability
Practice Group + capability owner
Risk Taxonomy 2026 class exposure per decision type
GOV-04 + per-capability assessment
Performance record from Operate quarterly reviews
GOV-15 + Evidence Register (GOV-13)
Regulatory and professional conduct exposure assessment
Legal Operations + External Counsel where applicable
Framework — Four-Factor Calibration
The four factors
Every decision type per Tier 3+ capability is scored against four factors. Each factor scores 1–4.
Factor
Score 1 (low)
Score 2 (med-low)
Score 3 (med-high)
Score 4 (high)
Consequence severity
Preparatory action; advisory output
Operational decision with reversible effect
Decision with material client / firm impact
Decision with regulatory, professional-conduct, or fiduciary consequence
Reversibility
Fully reversible — review gate catches all
Reversible with effort within the workflow window
Reversible only at significant cost
Irreversible or reversible only by external action
Regulatory exposure
No regulated activity
Activity adjacent to a regulated boundary
Activity within a regulated scope
Activity within a licensed-only scope
Institutional risk
Practice-group-level review only
Workflow-level review with named accountable individual
Quarterly Audit Committee review
Standing Board oversight
The four scores sum to a calibration total (4–16).
Three HITL tiers
Calibration total
HITL tier
Operating mode
4–7
Audit-only
System decides; human audits the record per quarterly cadence; no per-transaction review
8–11
Exception-triggered HITL
System decides per playbook; flagged or threshold-crossing cases route to named human for review per the calibration
12–16
Full HITL
Every transaction routes through human review; system supports the human decision
Additionally, a materiality threshold may force Full HITL regardless of calibration total — e.g., any decision with financial impact above $50K is Full HITL regardless of decision class. The threshold is calibrated per capability and refreshed quarterly.
Calibration row structure
Each row in the register holds:
Section
Content
Capability identifier
Canonical name from Capability Portfolio (SUS-10)
Decision type
One verb-level decision class per row
Consequence severity
Score 1–4 with rationale
Reversibility
Score 1–4 with rationale
Regulatory exposure
Score 1–4 with rationale
Institutional risk
Score 1–4 with rationale
Calibration total
Sum 4–16
HITL tier
Audit-only / Exception-triggered / Full
Materiality threshold
Financial / scope / client-impact threshold where Full HITL is mandatory
Escalation path
What triggers handoff; who handles
Revocation trigger
What performance or risk condition triggers tier elevation
Signatory
GC + Audit Committee Chair + Finance Partner
Effective date
When in force
Next review
Quarterly cadence date
Delegation-Authority Register cross-reference
GOV-14 entry ID
Evidence Register cross-reference
GOV-13 row IDs for affected classes
Quarterly refresh discipline
Activity
Cadence
Output
Performance-record review per calibrated decision type
Quarterly
Per-decision-type variance summary
Tier shift candidates flagged
Quarterly
Candidates for elevation (e.g., Audit-only → Exception-triggered) or relaxation (with strong rationale)
Materiality threshold review
Quarterly
Threshold adjustment recommendations
Regulatory context review
Quarterly
New rules or guidance affecting calibration
Refresh delta report compiled
Quarterly
One-page delta for Cadence agenda
Tier elevation triggers
Trigger
Action
Performance below the calibrated floor (> 2% error in audit sample for Audit-only; > 5% over-ride rate for Exception-triggered)
Tier elevation to next-up level
Audit finding requiring scope reduction
Tier elevation pending recalibration
Regulatory change increasing exposure
Tier elevation pending review
Pattern of near-miss events
Tier elevation pending review
Worked Example
A 200-lawyer in-house function calibrates a new Tier 3 contract-extraction capability with 6 decision types.
Decision type
Severity
Reversibility
Regulatory
Institutional
Total
Tier
Extract canonical clauses for review
1
1
1
1
4
Audit-only
Extract financial terms with summary
2
1
1
1
5
Audit-only
Flag unusual clause patterns for partner review
2
1
1
2
6
Audit-only
Suggest standard-form drafting amendments
2
2
1
2
7
Audit-only
Recommend negotiation positions
3
2
2
3
10
Exception-triggered HITL
Approve routine clause amendments below threshold
3
2
3
3
11
Exception-triggered HITL
Materiality threshold: any contract value above £250K triggers Full HITL across all decision types regardless of calibration total. Signed by GC, Audit Committee Chair, Finance Partner; effective 2026-09-01; next review 2026-12-01.
At quarter +1: audit sample of 50 extractions; 1 error in flagging; no tier shift required. At quarter +2: performance trend stable; no shifts; calibration continues. At quarter +3: regulatory guidance issues new requirement on AI-assisted contract review; institutional-risk score reviewed; one decision class shifts from Audit-only (total 7) to Exception-triggered HITL (total 9 after re-score); refresh delta documented; new calibration signed.
Common Failure Modes
Failure mode
Detection signal
Recovery
Calibration set but never refreshed
Last-review date older than two quarters
Bind refresh to Governance Cadence (GOV-15)
Tier elevation trigger met but not actioned
Performance breach in audit sample; no tier shift recorded
Audit Committee escalation; mandatory shift at next quarterly review
Materiality threshold drifts
Financial threshold not adjusted as currency / financial context changes
Finance Partner standing review item
Four-factor rationale missing
Score recorded; rationale absent
Treat as unsigned; require rationale before signature
Calibration tier not enforced in operation
Audit-only decision routed to Full HITL or vice versa
Operating-system configuration check; bind to Delegation-Authority Register (GOV-14)
New decision type deployed without calibration
Capability extended; calibration register stale
Bind capability extension to gate session per GOV-15
Refresh delta report missing
Quarterly delta not produced
Standing agenda item; produce or document non-production with reason
Edge Cases
The Module does not apply when:
- The function operates only Tier 1 and Tier 2 capabilities — no calibration required; AI Use Policy (GOV-02) is sufficient
- The function operates under a parent organisation’s enterprise-wide calibration framework — the Module produces sub-rows that feed the parent’s framework
- The function is below Band 3 — Tier 3+ deployment is premature; build Bands 1-2 through CHG-01 + GOV-01 first
- The capability is a Tier 4 system whose decision space is too dynamic to enumerate (e.g., open-ended generative tasks) — the calibration operates at the decision-class level rather than the decision-type level, with broader escalation triggers