embeddings

Embeddings are the foundation of modern NLP and retrieval. This tests whether you grasp the distributional idea behind them, not just 'words become vectors.' What matters is how training on context yields geometry that encodes meaning.

A transformer emits one vector per token, but retrieval needs one vector per text. The signal is knowing the pooling options and the catch nearly everyone overlooks: the model has to be trained for whichever one you pick.

A single trained embedding you can cut to any length and still use. The signal is the nested-prefix training objective and the coarse-to-fine retrieval win it enables at scale. Here is the answer.

Modern embedding models expect a task instruction prepended to the text, and a wrong prefix quietly wrecks retrieval. The signal is why instructions help and the asymmetric query/passage convention.

Three different systems are failing at once (the tokenizer, the model, and your pipeline) and most candidates blur them into one. Separating them tells you which fix buys the most, and the highest-leverage one is not the prompt.

The embedding model is the base of retrieval, and swapping it is deceptively risky. The signal is choosing on domain-relevant retrieval quality (not a leaderboard) and knowing that a new model means re-embedding everything. Here is the answer.

Dense versus sparse is the core retrieval tradeoff, and 'just use embeddings' quietly loses every exact-match query. The signal is knowing what each representation captures and why production blends them. Here is the answer.

Real documents carry images, charts, and tables, not only text, and text-only RAG quietly drops them. The signal is knowing the two retrieval approaches and why the generator has to change too. Here is the answer.

A short question and the passage that answers it sit in different neighborhoods of embedding space. HyDE bridges that gap by embedding a fake answer rather than the question. The signal is understanding why a wrong hypothetical still retrieves the right docs.

A chunk embedded in isolation forgets which document, section, and entity it came from, so it retrieves poorly. Contextual retrieval writes that context back in before indexing. The signal is fixing retrieval at index time rather than query time.

Fixed-size chunking cuts a coherent idea mid-thought; semantic chunking splits on meaning. The signal is naming the methods (structure-aware, embedding-similarity, LLM-based) and the tradeoffs that decide which one to ship.

Late chunking reverses the usual order of operations to solve the lost-context problem with zero additional LLM calls. Everything hinges on which step runs first. Here is the answer.

A model that leads MTEB can still stumble on your jargon-heavy corpus, and most teams reach straight for the costly fix. There is a ladder, and the cheap rungs are the ones people skip. Here is the order to climb it.

A 0.91 cosine score and a useless passage are not a contradiction. The candidates who score here name the encoding mismatch, the topical-but-not-answering chunk, and why an absolute score cutoff was never going to work.

float32 is far more precision than ranking needs. int8 is close to free, binary is 32x smaller, and the difference between a working binary index and a recall disaster is one word: rescoring.

Two frontier coding agents made opposite bets on the same problem, and both published evidence. Candidates who pick a side on vibes miss what the interviewer wants: the axes that actually decide it, and what each choice costs in staleness, latency, tokens, and infrastructure.

The core operation beneath every embedding/RAG retrieval, posed as a coding exercise. The signal is vectorizing the similarity, normalizing correctly, and using a partial selection (argpartition) rather than a full sort. Here is the efficient implementation and the scaling follow-up.

Negative sampling in word2vec and recommendation pulls millions of weighted samples; running each at O(log n) becomes a bottleneck. The alias method turns each draw into O(1) after an O(n) setup. Here it is.

The lookup is three lines of linear algebra. What separates a cache from an incident is where the similarity threshold came from, and the one-token queries ('2023' vs '2024', 'not') that no threshold can catch because the embedding barely moves.

Every RAG system starts here, and most candidates ship the same bug: they size the chunk in characters and call it tokens. Below is running code for all four strategies, plus the infinite loop in the overlap arithmetic and the one in the recursion.

Everyone can write the happy path. The screen is decided by the two branches most candidates skip: what the system does when retrieval finds nothing good, and what it does when the model cites a chunk you never sent.

Recsys underpins half of applied ML, and this question checks whether you fit the approach to data availability and hold a real answer for the cold-start problem that breaks naive systems. Here is the foundations answer.

Contrastive learning is how modern embeddings (CLIP, sentence encoders, SimCLR) are actually trained. What matters is the pull-positives-push-negatives objective, the InfoNCE loss, and why the number and hardness of negatives makes or breaks quality.

GNNs power recommendations, fraud, and molecule modeling by learning over graph structure. What matters is the message-passing mechanism, why k-hop matters, and why you keep them shallow. Here is the answer.

Metric learning succeeds or fails on the pairs you feed it. Random negatives teach almost nothing, and the margin plus the mining strategy determine whether the embeddings are any good. Here is how the choices interact.

A dedicated vector database is not always required. When embeddings sit beside your relational data, nearest-neighbor in SQL makes filters and joins easy. Here is how, and the scale where it breaks down.

Multimodal search checks whether you grasp shared embedding spaces and the retrieve-then-rank pattern under a real catalog's scale and freshness. What interviewers reward: CLIP-style joint embeddings, hybrid retrieval, and honest relevance evaluation. This lays out the design.

How large recommenders and search draw candidates from millions of items in milliseconds. The signal is why the user and item towers stay separate, how that enables precomputed embeddings plus an ANN index, and where ranking takes over.

The service under every RAG stack and semantic search box. The signal is your ANN index choice and how you deal with the three things that quietly break it: metadata filtering, live updates, and scale.

Image search is embedding retrieval at billion-image scale. A strong answer shows when a vision-only encoder is enough and when you need a CLIP-style shared space, plus how the ANN index actually serves the query. Here is the design.

Matrix factorization is the classic collaborative-filtering method and the conceptual seed of modern embedding-based recsys. What matters is the latent-factor idea and how it leads to two-tower/neural models.

Exact-match caching does little when no two prompts match. Semantic caching reuses answers across similar queries, and its entire danger is handing back a near-match that is subtly wrong. Here is how to build it safely.

Video search is image search plus a time axis, and the time axis is the whole problem. A strong answer embeds frames, pools them into scene vectors, indexes with ANN at billion scale, and explains how a multimodal query (clip, text, or both) locates the right moment, not just the right video.

AI features create cross-tenant leak paths that classic SaaS isolation never had to reckon with: shared embeddings, shared caches, and fine-tunes that memorize. One leak ends an enterprise contract. Here is the isolation model.

A track runs three minutes, a session runs an hour, and a fresh release has zero plays on launch day. The interview centers on mixing collaborative filtering with audio content embeddings, handling cold start on both sides, and reading a skip as the loud negative it really is.

Millions of landmarks exist, most with only a handful of photos, and the next photo could show something not in your catalog at all. A flat classifier collapses on the long tail and never says 'I don't know.' The interview is embeddings plus retrieval plus a confident refusal.