architecture

A tidy fundamentals question that trips up anyone who only knows GPT. The signal is pairing each architecture to its task through its attention pattern, and explaining why the field settled on decoder-only even for tasks that are not generation. Here is that answer.

Two defaults that every Llama-class model shares and almost no candidate can justify. One is a compute win that turned out to cost nothing in quality; the other is what makes a 60-layer stack converge at all.

Most candidates answer with the 2015 ResNet answer and stop. The senior answer treats the residual stream as a shared communication bus, and that reframing is what makes logit attribution, the logit lens, and activation steering possible at all.

Frameworks promise speed and hand you abstraction you'll eventually fight. Rolling your own is more code but full control. Here is the honest tradeoff, and why many production teams end up thinner than they started.

The industry uses 'agent' for anything that calls an LLM. The distinction that matters is how much control you hand to the model, and more autonomy is not better. Here is the spectrum and the decision rule.

Tool calling costs one LLM round trip per action. Code costs one round trip for a loop over 500 records. The signal is knowing exactly what you trade away when you let the model write the program instead of the call.

A hardware-literacy question, particularly at NVIDIA and the labs. What they grade is whether you understand throughput vs latency hardware, the memory hierarchy, and why matrix multiplication maps onto GPUs and TPUs. This is the architecture-aware answer.