The AI Engineer Interview Process: What the Loop Looks Like

Companies are hiring AI engineers because they have shipped something built on a model and it is not reliable enough. So the loop is engineered to find people who can make a probabilistic system trustworthy. Every round is a variation on one question: when this thing is wrong in production, will you be able to tell, and will you know why.

That is why the loop is not a standard SWE loop with a language-model flavor. Algorithmic ability is table stakes and it is not the discriminator. The discriminators are retrieval judgment, evaluation rigor, and the ability to reason about cost and latency without being asked.

Structures vary by company and team, and any specific claim about a named company's rounds goes stale within a quarter. What follows is the shape that recurs. Ask your recruiter for the actual sequence; they will usually tell you.

Motivation and calibration. Expect to be asked what you have shipped that used a model, and expect the follow-up to be about the part that went wrong. A candidate who describes a project with no failures is describing a demo.

The hiring manager screen is often where the level is set. Be ready to say, in two minutes, what the system did, what it was measured on, what the hardest failure was, and what you changed. If you cannot name the metric, you will be leveled down regardless of how good the project was.

Usually a working session rather than a whiteboard puzzle. Common shapes: implement a token-bucket rate limiter, write a chunker that respects markdown or code structure, parse a file that a real human corrupted, build a small retrieval pipeline and then defend the chunking decision, wrap a flaky API with retries and backoff and make the retry idempotent.

What is graded: whether your code runs, whether you handled empty input and ties and the retry-double-fire case, and whether you can talk while you type. Some loops still include a classic algorithm round at roughly medium difficulty. Prepare for it, but do not let it eat your preparation time, because it is not where offers are decided.

A rapid-fire technical conversation. Tokenization and context limits, embeddings and what similarity actually means in your metric space, the attention cost curve, KV cache, fine-tuning versus retrieval versus prompting and the conditions under which you pick each, quantization, and (if the team trains anything) the classical ML basics: regularization, class imbalance, leakage, why your offline AUC lied.

The interviewer is holding a follow-up in reserve for every confident answer. If you say "I would fine-tune," the follow-up is what data, how much, how you would evaluate it, and what you would do when the base model improves next quarter and your fine-tune is now worse than the thing you forked from.

An open product prompt: build a support agent over a company's knowledge base, build semantic search across a document corpus with per-user permissions, build a pipeline that summarizes long documents at scale. The failure mode is jumping to a diagram.

Do this instead. Name the users and the task. Name the metric you will be judged on and the failure the product cannot tolerate. Then draw the pipeline: ingestion and freshness, chunking, hybrid retrieval, reranking, permission filtering, the generation step and its structured-output guarantees, and the eval harness that gates changes to any of it. Then talk about the numbers: expected QPS, p95 latency budget and where it goes, cost per query and how routing brings it down, and what degrades gracefully when the provider has a bad hour.

The strongest signal you can give is scoping down. Say what you would cut for v1 and why. Interviewers are watching to see if you can ship, not if you can enumerate.

The round that increasingly decides the loop, and the one candidates have practiced least. You are handed a broken system: the agent hallucinates on a class of questions, quality dropped after a model upgrade, the eval score rose while user satisfaction fell. Triage it out loud.

They want a layered diagnosis, not a guess. Reproduce it, look at the trace, isolate whether the failure is ingestion, retrieval, ranking, generation, or the data itself, form a hypothesis, and name the experiment that would falsify it. Then the harder half: how would you have caught this before it shipped, and what do you add to the eval suite so it never recurs silently.

Ownership, disagreement, and how you behave when the system is wrong and a stakeholder is unhappy. Prepare two or three stories in STAR form with real numbers, including one where you were wrong and caught it yourself.

For preparation, split your time roughly: a third on building and evaluating one real system end to end (this is the highest-yield thing you can do, because every round draws on it), a third on the depth material (retrieval, evals, inference economics), and the remainder on practical coding. Do the system-design and debugging rounds out loud with a timer. Reading about evaluation and articulating an eval strategy under pressure are different skills, and only one of them gets you the offer.

Timelines vary widely. Multi-week loops are the norm and take-homes are common at smaller companies. Ask up front how many rounds there are and what each one covers, because the loops differ enough that generic preparation is genuinely inefficient.