chunking

Truncate, map-reduce, refine, retrieve, or stuff a long-context model. Each buys you something different and breaks differently. The candidates who score are the ones who pick based on the question being asked, not the document being read.

The two knobs that decide whether a RAG system works, and the two candidates tend to gloss over. The signal is tuning chunking against recall and knowing precisely what dense retrieval drops that BM25 recovers. Here is the reasoning, not the rules of thumb.

Chunking quietly caps RAG quality, and 'split every 500 tokens' leaves recall on the table. The signal is knowing the strategy families and the parent-child trick that separates retrieval size from context size.

Garbage parsing means garbage retrieval, and naive PDF text extraction wrecks tables and reading order. The signal is fixing the parser, not the retriever. Here is the answer.

Small chunks match precisely but starve the LLM of context; big chunks carry context but match badly. Parent-child retrieval declines to choose. The signal is separating what you match on from what you feed the model.

A chunk embedded in isolation forgets which document, section, and entity it came from, so it retrieves poorly. Contextual retrieval writes that context back in before indexing. The signal is fixing retrieval at index time rather than query time.

Fixed-size chunking cuts a coherent idea mid-thought; semantic chunking splits on meaning. The signal is naming the methods (structure-aware, embedding-similarity, LLM-based) and the tradeoffs that decide which one to ship.

Late chunking reverses the usual order of operations to solve the lost-context problem with zero additional LLM calls. Everything hinges on which step runs first. Here is the answer.

When the right answer isn't even in the top-50, the generator can't rescue you. Low recall has a short list of usual suspects. Here is the order to check them so you fix the cause, not a symptom.

Every RAG system starts here, and most candidates ship the same bug: they size the chunk in characters and call it tokens. Below is running code for all four strategies, plus the infinite loop in the overlap arithmetic and the one in the recursion.

Most RAG projects do not die at the retriever, they die at ingestion. Naive PDF extraction interleaves columns into nonsense, shreds tables, and stamps the footer into all 4,000 chunks. Here is the parser that survives real documents.