faithfulness

Chain-of-thought looks like the model showing its work, but it may not mirror the real computation. The signal is knowing CoT can be a post-hoc story, with evidence, and what that breaks for safety and oversight.

Preventing hallucinations is one job; catching the ones that slip through at runtime is a separate one. The signal is the detection toolkit, why each signal is imperfect, and how you merge them into an action.

Summarization is the one task where the model was handed everything it needed and still made something up. The signal is separating intrinsic from extrinsic hallucination, knowing why ROUGE cannot see either, and gating on entailment before you ship.

Most RAG systems ship without genuine evaluation, which is why most of them quietly degrade. The signal is breaking evaluation into retrieval and generation, and scoring faithfulness apart from relevance. Here is the framework that makes RAG improvements measurable.

Citations turn a RAG answer from 'trust me' into something checkable, and they are a genuine engineering problem. The signal is grounding each claim to a specific passage and verifying the attribution, not tacking a bag of links onto the end.

Real corpora contradict themselves (old versus new policy, different teams), and naive RAG quietly picks one or fuses them into a wrong answer. The signal is spotting conflict and resolving it through recency, authority, and transparency.

An LLM can cite a source that does not say what the answer claims. The signal is verifying claim-by-claim entailment against the cited text, not merely that a citation marker exists, and knowing what to do when grounding fails.

An answer can be correct yet still unfaithful, asserting things the retrieved context never said. The signal is scoring faithfulness against the context, not against truth, and knowing why LLM-judge faithfulness scores drift.

Asking a model 'is this answer hallucinated?' barely beats a coin flip. This is the build: claim extraction, a judge forbidden from using world knowledge, a denominator that excludes hedges, and one model call per claim, which decides where the check can run at all.

Two forces shape this design: documents longer than the context window, and summaries that must not lie. The strong answer pairs hierarchical map-reduce with claim-level grounding, then serves it cheaply at scale.