routing
AI, ML & GenAI interview questions tagged routing, across every topic.
3 questions · 2 unlocked for you
Concepts behind "routing"
The curriculum that explains the ideas these questions test.
Foundational
The LLM GatewayAn LLM gateway is one proxy layer sitting between your application and one or more model providers. It consolidates the cross-cutting concerns every LLM app needs: routing and fallback across models/providers, caching, rate limiting, authentication, cost tracking, observability, and guardrails. By hiding providers behind a single interface, it also guards against vendor lock-in. AI, ML, and GenAI engineer interviews probe it because it forms the backbone of a production LLM platform and holds most operational controls.⚙️ System Design for AI in Production
Foundational
LLM Cost OptimizationLLM systems get expensive fast, and the cost model comes down mostly to tokens and number of model calls. The levers, in rough order of impact: route easy queries to cheaper/smaller models, cache repeated and similar requests, trim context (fewer, better chunks), use cheaper retrieval/reranking, and for agents cut unnecessary steps. The discipline is measuring cost per request and going after the dominant contributor. AI, ML, and GenAI engineer interviews probe it because cost is a primary production constraint and most teams overspend by defaulting to the biggest model on everything.⚙️ System Design for AI in Production
Core
Mixture-of-ExpertsA Mixture-of-Experts model swaps the dense feed-forward layer for many expert networks plus a router that sends each token to only a few of them. This separates total parameters (capacity) from per-token compute: the model can be huge while each token activates only a slice. The trade-offs are routing complexity, memory (all experts must be loaded), and load balancing. AI, ML, and GenAI interviews probe it because most frontier models are MoE, and it explains how models grow more capable without proportionally more inference cost.🧠 Foundations of LLMs & GenAISign in
Core
Small vs Large Models and RoutingBigger is not always better in production: small models are far cheaper and faster, and for many tasks they are good enough, especially when fine-tuned or given retrieval. The mature pattern is routing, send easy queries to a small/cheap model and hold back large or reasoning models for genuinely hard ones, often with a cascade that escalates on low confidence. AI, ML, and GenAI interviews probe it because picking and routing models is where most of the cost and latency budget is won or lost.🧠 Foundations of LLMs & GenAISign in