1. Purpose and Scope
The Legal AI ROAI Dashboard is the central measurement instrument for tracking return on AI investment across legal. It operationalises STR-08 (ROAI Matrix) by converting baselines, pilot results, and live deployment data into a unified, defensible view of AI performance.
The scope covers:
- All AI use cases approved via USE-01
- Both augmentation and agentic deployments
- Four value dimensions: Operational, Financial & Client, Capability & Culture, Strategic & Reputational
- All nine canonical Risk Taxonomy 2026 classes
Cadence: continuous data ingestion, daily operational tracking, weekly reporting, monthly ROAI analysis, and quarterly strategic reviews.
2. ROAI Dimensions and KPIs
Dimension 1 — Operational Gains
Focus: Time saved, efficiency, throughput.
Core KPIs:
- Time Saved Per Task (20–50% reduction; daily tracking)
- Document Processing Speed (60–80% faster; real-time)
- Matter Turnaround Time (25–40% reduction; weekly)
- Automation Rate (70–90% of eligible tasks; weekly)
- Capacity Utilisation (15–30% increase; monthly)
Dimension 2 — Financial and Client Gains
Focus: Cost reduction, revenue, client satisfaction.
Core KPIs:
- Cost Per Matter (20–40% reduction; monthly)
- Outside Counsel Spend Reduction (15–30%; quarterly)
- Client Satisfaction / NPS (85%+ satisfaction; 10+ NPS uplift; quarterly)
- Revenue Per Hour (10–20% improvement; monthly, law firms)
- Client Response Time (50%+ improvement; weekly)
Dimension 3 — Capability and Culture Gains
Focus: AI literacy, adoption, wellbeing, innovation.
Core KPIs:
- AI Adoption Rate (80%+ active; weekly). Shadow AI signal: <60% in any team triggers Class 6 investigation and potential GOV-05 review.
- User Satisfaction (≥8/10; quarterly)
- Training Effectiveness (90%+ completion; 80%+ competency; quarterly)
- Work-Life Balance (10%+ overtime reduction; monthly)
- Innovation Index (2–3 new use cases per quarter; ≥50% implemented)
Dimension 4 — Strategic and Reputational Gains
Focus: Compliance, risk reduction, market differentiation.
Mapped KPIs:
- Risk Detection Rate — Class 1
- Compliance Monitoring — Classes 7 & 4
- Error Reduction — Classes 1 & 2
- Bias Audit Rate — Class 3
- IP Content Review Rate — Class 8
- AI BoM Vendor Compliance — Class 5
- Audit Readiness — Classes 7 & 9
Targets include ≥90% risk detection accuracy, <5% false positives, ≥95% compliance issue prevention, ≥70% error reduction, and ≥50% audit preparation time reduction.
3. Agentic Tier Monitoring
Autonomous AI agents require a dedicated track:
- Real-time Intervention Rate: human overrides; target <5% for stable deployments.
- Kill-switch Activation Log: all circuit-breaker events, root-caused by Risk Taxonomy 2026 class, feeding GOV-05.
- Autonomous Decision Accuracy: AI-only decisions validated against attorney samples (Class 1).
- Scope Creep Detection: tasks outside permission scope; triggers Class 6 Shadow AI flag.
- Agentic ROAI Attribution: separate ROAI for agentic vs augmentation tiers.
Agentic anomalies automatically generate GOV-05 incident workflows and GOV-03 risk entries where applicable.
4. Ecosystem and Governance Hooks
The dashboard is embedded in the broader operating system:
- STR-08 (ROAI Matrix): Monthly ROAI updates Quadrant 2 (financial) and Quadrant 4 (strategic) performance.
- USE-01: Quarterly re-scoring uses dashboard data to expand, optimise, or retire use cases.
- USE-02: Phase 3 pilot evaluation pulls ROAI metrics for scale / no-scale decisions.
- USE-05: Provides pre-AI baselines for all KPIs.
- DPS: Quarterly outputs feed Adoption and Sophistication lenses; annual export is the DPS evidence package.
- AI BoM: Only registered tools receive ROAI credit; unregistered tools trigger Class 6 Shadow AI flags.
- GOV-01: Governance metrics (adoption, policy adherence, audit trail quality) support annual review.
- GOV-03: Risk detection metrics and incidents populate the Risk Register by class.
- GOV-05: Anomalies, kill-switch events, and severe metric deviations trigger incident response.
- STR-07: AI Task Force reviews monthly performance and signs off quarterly ROAI updates.
5. AI BoM Integration Requirements
Each monitored tool must have:
- Confirmed AI BoM entry before any ROAI attribution
- Vendor performance metrics tied to BoM records
- Contract and renewal alerts surfaced in Dimension 4
- New tool onboarding gated by BoM registration
Unregistered tools detected in usage analytics:
- Trigger a Class 6 Shadow AI flag
- Generate a GOV-03 Risk Register entry
- May initiate GOV-05 investigation depending on impact
6. Executive Summary View Design
The executive view provides leadership-ready insight:
- ROAI Scorecard: overall monthly ROAI %, YTD trend, comparison to STR-05 projections, RAG status.
- Key Metrics Scorecard: top 5 KPIs vs targets with trend arrows and alerts.
- Financial Impact Summary: total savings, payback progress, budget variance.
- DPS Evidence Package: auto-formatted quarterly outputs for Adoption and Sophistication lenses.
- Critical Alerts: system issues, metric deviations, GOV-05 triggers, AI BoM compliance flags.
7. Operational Performance Dashboard
This view focuses on day-to-day efficiency:
- Time Savings Visualisation: daily hours saved, task-level drill-down, cumulative savings since AI BoM registration.
- Automation Effectiveness: automation rate gauges, volume charts, quality indicators.
- Capacity Utilisation: before/after comparisons by practice area and matter type.
8. Financial Impact Dashboard
This view supports finance and client value conversations:
- Cost Reduction Analysis: cost per matter trends, outside counsel spend, budget variance.
- ROAI Calculation Methodology: transparent inputs and assumptions aligned to STR-08 Quadrant 2.
- Investment Recovery Tracking: payback progress vs STR-05, with multi-scenario modelling.
9. Risk and Compliance Dashboard
This view aligns with Risk Taxonomy 2026 and governance:
- Risk Detection Performance by Class: accuracy, false positive/negative trends, escalation patterns.
- AI BoM Compliance Track: registration status, unregistered tool flags (Class 6), renewal alerts (Class 5).
- Agentic Tier Monitor: intervention rate, kill-switch log, scope adherence.
- GOV-03 Risk Register Feed: incidents by class, remediation status, repeat-incident alerts.
10. Implementation Phase 1 — Baseline Establishment (Weeks 1–4)
Activities:
- Collect 12 months of historical data for all planned KPIs.
- Establish statistical baselines and confidence intervals.
- Document data collection methodologies and source systems.
- Map data flows from DMS, matter management, billing, and AI tool APIs.
- Confirm AI BoM entries for all tools; block ROAI attribution until confirmed.
Deliverables:
- Baseline dataset and KPI catalogue
- Data flow diagrams and source-of-truth registry
- Initial ROAI target mapping to STR-08
11. Implementation Phase 2 — Dashboard Development (Weeks 5–8)
Activities:
- Configure APIs and connectors to all source systems.
- Build the executive ROAI Scorecard aligned to STR-08 targets.
- Implement AI BoM compliance track and Class 6 Shadow AI alerts.
- Build Agentic Tier monitor (if agentic deployments exist) with GOV-05 triggers.
- Configure GOV-03 integration for Dimension 4 risk feeds.
Deliverables:
- Working dashboard environment
- Configured ROAI calculation engine
- Governance and alerting ruleset
12. Implementation Phase 3 — Testing and Validation (Weeks 9–12)
Activities:
- Validate ROAI calculations against STR-08 Quadrant 2 standards.
- Test GOV-05 alert triggers and AI BoM flag logic.
- Conduct UAT with Legal Operations and STR-07 AI Task Force.
- Train Legal Operations on DPS evidence export and quarterly review workflows.
Deliverables:
- Signed-off ROAI calculation model
- UAT report and remediation log
- Training materials and runbooks
13. Sample ROAI Calculation
| Cost Category | Pre-AI Monthly | Post-AI Monthly | Monthly Savings |
|—|—|—|—|
| Attorney Time (Hours × Blended Rate) | $150,000 | $97,500 | $52,500 |
| Outside Counsel Spend | $45,000 | $31,500 | $13,500 |
| Administrative Overhead | $25,000 | $18,750 | $6,250 |
| Total Monthly Savings | | | $72,250 |
| AI Investment (Monthly) | | $18,000 | — |
| Net Monthly Benefit | | | $54,250 |
| Monthly ROAI | | | 301% |
Formula: ROAI = (Total Savings − AI Investment) ÷ AI Investment × 100, applied only to AI BoM-registered tools.
14. Predictive Analytics and Optimisation
Advanced analytics capabilities:
- Adoption Trend Modelling: forecast Shadow AI risk and training needs.
- Capacity Planning: anticipate future workload vs capacity using USE-01 pipeline.
- ROAI Projection: scenario modelling for STR-05 business case refresh.
- Performance Degradation Alerts: early warnings for declining accuracy or adoption, triggering GOV-03 and GOV-05 where needed.
15. Continuous Improvement Loop
- Quarterly: USE-01 re-scoring based on dashboard performance.
- Monthly: Vendor performance from AI BoM reviewed; underperforming vendors flagged.
- Annual: DPS evidence package exported across all four dimensions and submitted into GOV-01 review.
16. Roles, Ownership, and Cadence
- Owner: Legal Operations, with AI Task Force (STR-07) and GC oversight.
- Cadence:
- Continuous ingestion and monitoring
- Weekly operational and adoption reports
- Monthly ROAI and vendor performance reviews
- Quarterly strategic and DPS reviews
17. Governance and Measurement Metadata
- Methodology Version: v2026.1
- Risk Class: cross-class (all nine canonical classes)
- DPS Lenses: Adoption and Sophistication
- Certification Track: Practitioner
- Advisory Relevance: Programme Design (months 2–6) and Strategic Retainer (ongoing)
- Telemetry: dashboard activations, monthly ROAI scores, DPS exports, GOV-05 trigger counts.
18. Success Metrics and Leading Indicators
Success indicators include:
- ROAI meeting or exceeding STR-08 targets across dimensions
- High AI adoption with low Shadow AI incidents
- Reduced outside counsel spend and improved client satisfaction
- Strong audit readiness and regulator-grade documentation
Leading indicators:
- Rising automation rates with stable or improved accuracy