accuracy

Legal review is high-stakes and precision-critical: a missed clause or hallucinated citation carries real consequences, and lawyers have been sanctioned over fabricated cites. The signal is grounding in the actual documents, citing exact passages, and lawyer-in-the-loop. Here is the design.

Naive 4-bit quantization can wreck quality, and the reflex to give up and serve fp16 leaves a large speedup unclaimed. The accuracy is usually recoverable. Here is the ladder of fixes.

FP8 serving is fast and usually fine, until it quietly degrades on the one workload your benchmark did not cover. What counts is knowing what FP8 breaks and how to validate it. Here is the recipe.