130Write an async batch caller for an LLM API: N requests, a concurrency cap, timeouts, and retries with backoff.▼mediumOpenAIAnthropicScale AI◆ premiumThe most job-shaped coding screen in AI, ML, and GenAI engineering: fan out N LLM calls without melting the rate limit or losing the batch to one bad request. What matters is the retry policy, not the async syntax. Below is the version that passes.Open full answer →
131Implement a conversation memory system: buffer, sliding window, summary, and token-budget eviction.▼mediumOpenAIAnthropicLangChain◆ premiumThe most-asked hands-on LLM exercise: a chat history that outgrows the context window. What gets scored is not the sliding window, it is the region eviction is never allowed to touch. Below is the version that passes.Open full answer →
132Write token counting and context-window packing for an LLM call: fit the budget, reserve room for the completion.▼mediumOpenAIAnthropicCohere◆ premiumEvery RAG system packs a prompt, and the packing bug is always the same one: the input fits the window exactly, so the model has nowhere left to answer. The arithmetic here is what separates a candidate who has shipped from one who has read about it.Open full answer →
133Implement a semantic cache for LLM responses. When is a similar-enough query actually a hit?▼hardOpenAIAnthropicPerplexity◆ premiumThe lookup is three lines of linear algebra. What separates a cache from an incident is where the similarity threshold came from, and the one-token queries ('2023' vs '2024', 'not') that no threshold can catch because the embedding barely moves.Open full answer →
140Consume a streaming LLM response: SSE parsing, incremental output, cancellation, and partial JSON.▼mediumOpenAIAnthropicVercel◆ premiumThe naive version splits on newlines and works right up until a TCP chunk lands mid-line. Then there is the error that arrives after a 200 OK, the user who closes the tab while you keep paying for tokens, and JSON you cannot parse until it closes.Open full answer →
