99How does a vision-language model connect an image encoder to an LLM, and where does it fail?▼hardOpenAIGoogle DeepMindMeta2 replies◆ premiumVLMs such as GPT-4V and LLaVA attach a vision encoder to a language model through a projector. The signal is the image-tokens-as-prefix design, the alignment training, and the resolution and hallucination failure modes.Open full answer →
21How does a Vision Transformer (ViT) work, and when does it beat a CNN?▼hardGoogleMetaNVIDIA1 replies◆ premiumPatches as tokens, global attention from layer one, and a weaker inductive bias than a CNN. What interviewers reward is naming the data regime where each architecture wins and why. Here is the answer interviewers score highest.Open full answer →
