Validation, Metrics & Sampling
Validation, Metrics & Sampling
Core measurements and quality-control concepts users need to understand before relying on TAR outputs.
TAR Metrics That Matter
Recallrelevant found / total relevant
Estimates how much relevant material the workflow captured.
Precisiontrue relevant reviewed / total marked relevant
Shows how much review effort is spent on truly relevant material.
Elusionrelevant documents in the non-reviewed population / sampled non-reviewed documents
Estimates what relevant material may remain after the stopping point.
Richnessrelevant documents / total collection
Provides the base rate that affects sampling, review effort, and expectations.
Confidence Intervalestimate ± margin of error
Expresses uncertainty in sampling-based metrics.
Overturn Ratechanged coding decisions / QC-reviewed decisions
Flags coding instability, unclear guidance, or reviewer training issues.
| Metric | Formula | Purpose |
|---|---|---|
| Recall | relevant found / total relevant | Estimates how much relevant material the workflow captured. |
| Precision | true relevant reviewed / total marked relevant | Shows how much review effort is spent on truly relevant material. |
| Elusion | relevant documents in the non-reviewed population / sampled non-reviewed documents | Estimates what relevant material may remain after the stopping point. |
| Richness | relevant documents / total collection | Provides the base rate that affects sampling, review effort, and expectations. |
| Confidence Interval | estimate ± margin of error | Expresses uncertainty in sampling-based metrics. |
| Overturn Rate | changed coding decisions / QC-reviewed decisions | Flags coding instability, unclear guidance, or reviewer training issues. |
Thresholds
RecallMin 70 · Target 80 · Strong 90
ElusionMin 15 · Target 10 · Strong 5
Confidence LevelMin 90 · Target 95 · Strong 99
| Metric | Minimum | Target | Excellent |
|---|---|---|---|
| Recall | 70 | 80 | 90 |
| Elusion | 15 | 10 | 5 |
| Confidence Level | 90 | 95 | 99 |
Validation reminder
Targets are matter-dependent. A low-stakes matter, a government investigation, a privilege-heavy production, and a bet-the-company case may justify different thresholds. Document the rationale, not just the number.