← 🖥️ ML Infrastructure & Serving
Core
Green AI: Compute, Energy, and Carbon
Energy is roughly GPU-hours times average power draw times data-center PUE, and carbon is that energy times the grid's carbon intensity where and when you ran it, which varies by an order of magnitude across regions. The counterintuitive consequence: for a widely-deployed model, lifetime inference energy usually dwarfs the one-off training run, so the biggest lever is the serving stack (quantization, distillation, batching, higher utilization) rather than the training job that gets the headlines. AI, ML, and GenAI interviews probe it because efficiency work is cost work, and because a candidate who can write the calculation down is rare.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
ML Infrastructure & GPUsWhat is MFU (Model FLOPs Utilization), and why can GPU utilization be misleading?→ML Infrastructure & GPUsHow do you serve many fine-tuned model variants efficiently (multi-LoRA serving)?→ML Infrastructure & GPUsYour model training is burning a lot of energy. How do you measure and cut the carbon footprint?→ML Infrastructure & GPUsServe a 70B-parameter model with high throughput. Do the memory math and name the optimizations.→ML Infrastructure & GPUsExplain the KV cache: prefill vs decode, why it grows, and how MQA/GQA and PagedAttention help.→ML Infrastructure & GPUsExplain speculative decoding and the other main levers for cutting LLM generation latency.→
COMPANIES THAT ASSUME THIS