vector database

At a billion vectors the index choice is a memory budget question before it's a recall question. Flat search is off the table, HNSW may not fit, and PQ swaps recall for RAM. Here is the call a staff engineer makes on the whiteboard.

When you shard a vector index or route requests across model replicas, naive 'hash mod N' reshuffles everything the moment N changes. Consistent hashing moves only a small fraction. Here is the implementation.

The service under every RAG stack and semantic search box. The signal is your ANN index choice and how you deal with the three things that quietly break it: metadata filtering, live updates, and scale.

Selling RAG as a product means wrangling messy ingestion, multi-tenant isolation, and per-customer index freshness, all behind a simple API. See the ingestion and query planes, how to keep tenants isolated, and how to hand customers eval and observability.