ingestion

Most RAG projects do not die at the retriever, they die at ingestion. Naive PDF extraction interleaves columns into nonsense, shreds tables, and stamps the footer into all 4,000 chunks. Here is the parser that survives real documents.

Selling RAG as a product means wrangling messy ingestion, multi-tenant isolation, and per-customer index freshness, all behind a simple API. See the ingestion and query planes, how to keep tenants isolated, and how to hand customers eval and observability.

Metrics, traces, and IoT data are append-heavy, time-ordered, and seldom updated. A general database handles this poorly. The wins come from columnar layout, compression tuned for timestamps, downsampling, and retention. Here is the time-series design interviewers want.