Baseline Metrics Capture Guide for Legal AI

Purpose

The Baseline Metrics Capture Guide defines how legal departments measure current performance before deploying AI. It ensures that every AI pilot or implementation has a defensible pre-AI benchmark across operational, quality, financial, and user experience dimensions. These baselines are mandatory inputs to ROAI calculations, governance artefacts, and scaling decisions.

Scope and Positioning

This module is used per engagement, before each AI pilot or implementation, and whenever significant process or organisational changes require re-baselining. It assumes a defined AI use case (USE-01), an initial pilot design (USE-02 Phase 1), and a draft business case (STR-05).

It produces the pre-AI comparison series for the Legal AI ROAI Dashboard (USE-04), denominator inputs for the ROAI Matrix (STR-08), and initial population of the GOV-03 Risk Register, including Shadow AI baseline counts.

Measurement Dimensions

1. Operational Efficiency – Time per task, throughput, cycle time, and resource allocation patterns.
2. Quality and Accuracy – Error rates by severity, revision cycles, client satisfaction, and compliance adherence.
3. Financial Performance – Cost per matter, outside counsel spend, utilisation, and value generation metrics.
4. User Experience – Attorney satisfaction, training and onboarding efficiency, system usability, and burnout indicators.

Agentic Tier implementations add a dedicated baseline for human intervention frequency, decision audit trail completeness, scope documentation, and error rates for autonomous-candidate tasks.

AI BoM Metric 0

Before any other data collection, the module requires an AI Bill of Materials baseline:

• Inventory of approved AI tools in use and confirmation of BoM entries.
• Anonymous Shadow AI survey to quantify unapproved AI usage.
• Identification of existing vendor contracts with AI components.
• Detection of AI features that are enabled but not tracked.

The Shadow AI count becomes the Class 6 baseline for policy compliance improvement.

DPS Zero-State

The module documents the organisation’s DPS zero-state:

• Adoption lens – Confirmation that no formal AI deployment has occurred for the scoped use case.
• Sophistication lens – Manual performance benchmarks (time, errors, cost per matter) as pre-AI standards.
• Defensibility lens – Status of GOV-01, GOV-02, and GOV-03 at baseline date.

A credible zero-state is required before USE-02 Phase 2 deployment is authorised.

Implementation Phases

1. Phase 1 – Planning (Weeks 1–2)
   • Confirm use case scope and pilot parameters.
   • Secure AI Task Force approval of measurement scope.
   • Configure data collection systems and complete AI BoM Metric 0.
2. Phase 2 – Initial Data Collection (Weeks 3–6)
   • Connect to operational, financial, and matter systems.
   • Launch surveys, including Shadow AI usage.
   • Begin structured error and compliance tracking.
3. Phase 3 – Baseline Period (Weeks 7–14)
   • Maintain consistent measurement.
   • Run weekly data quality checks and log process changes.
4. Phase 4 – Analysis and Documentation (Weeks 15–16)
   • Compute descriptive statistics and segment by role, practice, and matter type.
   • Map findings to Risk Taxonomy 2026 classes and compare to STR-05 assumptions.
5. Phase 5 – Review and Target Setting (Weeks 17–18)
   • Present results to STR-07 AI Task Force and leadership.
   • Set ROAI targets aligned with STR-08 thresholds.
   • Submit the baseline package as a gate for USE-02 Phase 2.