95What is FP8, and how does it differ from INT8 for LLM training and inference?▼hardNVIDIAGoogle DeepMindMicrosoft2 replies◆ premiumFP8 underpins modern H100-class training and serving. What matters is knowing the two FP8 variants, why a floating format beats INT8 on dynamic range, and where scaling still counts.Open full answer →
03Explain quantization for inference: INT8/INT4, GPTQ/AWQ, what breaks, and how you validate it.▼hard★ EssentialNVIDIAOpenAIxAI2 repliesunlockedQuantization is the opening move for shrinking and accelerating models, and the interviewer expects more than 'use fewer bits.' What they grade is whether you know what each precision level gains you, why outliers wreck naive quantization, and how you demonstrate quality survived. This lays that out.Open full answer →
