ml coding

A recurring ML-coding screen. The loop itself is easy; what interviewers watch for is k-means++ init, a genuine convergence criterion, and awareness of the failure modes. Here is the compact implementation plus the follow-ups interviewers reliably ask.

A deceptively easy ML-coding ask. Writing exp/sum is trivial; the signal is the max-subtraction trick and the log-sum-exp form that stop it overflowing. Here is the stable implementation and the reason the naive version breaks.

The from-scratch implementation that frontier labs and NVIDIA really do ask. Writing it shows you grasp shapes, the scale factor, the causal mask, and how heads split and recombine. Here is a correct, readable implementation plus the follow-ups.

Beam search is the decoding algorithm everyone cites and few can code correctly. The signal is keeping k hypotheses by cumulative log-prob, adding logs (not multiplying probs), and handling completed sequences and length. Here is a correct implementation and the tradeoffs.

A from-scratch ML-coding staple that tests whether you really know the math you rely on. The signal is the clean gradient (it reduces to Xᵀ(ŷ - y)/n), numerical stability, and vectorization. Here is the implementation and the details interviewers push on.

The core operation beneath every embedding/RAG retrieval, posed as a coding exercise. The signal is vectorizing the similarity, normalizing correctly, and using a partial selection (argpartition) rather than a full sort. Here is the efficient implementation and the scaling follow-up.

A common ML-coding check that confirms you actually understand the metrics you quote. The signal is getting TP/FP/FN/TN right and knowing macro vs micro averaging for multiclass, not reaching for sklearn.

Data loading is where training pipelines quietly bottleneck. The signal is a correct shuffle-then-batch iterator plus knowing why prefetching and parallel loading keep the GPU fed.

BPE powers GPT-style tokenizers, and coding it shows you grasp how subword vocabularies get constructed rather than just knowing they exist. What interviewers watch for is the repeated merge of the most common adjacent pair.

k-NN is quick to code and a solid check of vectorization and the lazy-learner tradeoff. Interviewers look for a tidy vectorized distance calculation plus knowing why naive prediction is O(n) and how to accelerate it. The implementation follows.

Coding conv2d shows you understand what a CNN layer really computes rather than just being able to name it. Interviewers watch for correct output-shape math, tidy stride and padding handling, and awareness of the im2col trick frameworks actually rely on.

LayerNorm appears in every transformer, and coding it verifies you know what it normalizes over and why. Interviewers look for normalization across the feature dimension per token, the learnable scale and shift, and the RMSNorm simplification modern LLMs took up.

Coding the KV cache shows you understand why decode is efficient: you append the new token's K/V and reuse the rest rather than recomputing. Interviewers look for the append-and-attend-over-cache logic. The implementation follows.

Coding BatchNorm verifies you know the train/inference difference, the part everyone forgets. Interviewers look for normalizing over the batch with running stats kept for inference. The implementation follows.

TF-IDF is the classic text-vectorization scheme and a quick test of whether you truly grasp term-frequency times inverse-document-frequency, or merely recite it. What interviewers watch for: explaining why IDF kills common words. Here is the implementation.

Building a PR/ROC curve from scratch shows you understand thresholds and the precision/recall tradeoff. What interviewers really want is sweeping the threshold in one sorted pass and integrating the area. The implementation follows.

Coding Adam shows you know what .step() actually does: momentum combined with per-parameter adaptive rates and bias correction. What interviewers watch for is the two moments plus the bias-correction step.

Single-head attention shows up often; multi-head layers on the split-into-heads, attend-per-head, concatenate pattern. What gets scored is the reshape into heads plus a sharp explanation of why several heads help. Here is the implementation.

Dropout takes five lines, yet most candidates fumble two of them: turning it off at inference and rescaling the survivors so the expected activation holds constant. Inverted dropout is the convention they want to see.

Plain multi-head attention is rarely the whole exercise: the usual extension is to chain it into KV-cached decoding and GQA in one sitting. The locked answer walks the NumPy implementation, the equivalence check that proves your cache is correct, and the reshape bugs that silently corrupt attention.

A from-scratch classic that checks recursion plus the split criterion math. The signal is computing impurity correctly, picking the best threshold by information gain, and knowing the stopping rules. Here is a clean recursive implementation.

A from-scratch staple that checks whether you grasp the conditional-independence assumption and why you work in log space. The signal is fitting per-class means and variances, summing log-likelihoods, and adding the log prior. Here is the implementation.

A from-scratch favorite that probes linear algebra fluency. The signal is centering first, using SVD instead of forming the covariance matrix, and reading variance off the singular values. Here is the implementation and the details interviewers push on.

A build-it-yourself check on the EM algorithm and soft clustering. What matters is the pair of alternating steps (responsibilities, then weighted re-estimation), log-sum-exp for numerical stability, and seeing how GMM extends k-means. The code follows.

A build-it-yourself check on the least-squares closed form and the numerical traps hiding in it. What matters is deriving the normal equations, understanding why you never invert explicitly, and reaching for lstsq or the SVD instead. The code follows.

A build-it-yourself check on recurrent forward passes and backprop through time. What matters is the shared-weight recurrence, summing gradients over timesteps, and articulating the vanishing-gradient problem. The code follows.

A build-it-yourself check on gated recurrence. What matters is wiring the forget/input/output gates and candidate correctly, keeping cell state distinct from hidden state, and explaining why the additive cell path cures vanishing gradients. The code follows.

A build-it-yourself check on modern optimizers. What matters is the moment estimates, bias correction, and the single change that counts: AdamW decouples weight decay from the gradient rather than folding it into the loss. The code follows, plus why it wins.

A build-it-yourself check on how learning-rate schedules are actually computed. What matters is correct closed-form formulas, knowing when each schedule shines, and why decaying the rate aids convergence. Below is the implementation of the common ones.

A build-it-yourself check on how PyTorch actually works under the hood. What matters is assembling a computation graph during the forward pass, local derivatives per op, and a topological-order backward pass that accumulates gradients. Below is a minimal engine.

A build-it-yourself check on subword tokenization. What matters is greedy longest-match encoding against a fixed vocabulary, the continuation-prefix convention, and how WordPiece diverges from BPE. The code follows.

A build-it-yourself check on distributed input pipelines. What matters is partitioning data across workers with no overlap and no gaps, epoch-consistent shuffling with a shared seed, and handling the uneven-last-batch problem. The code follows.

The classic recursive-descent exercise with an applied-AI twist: the JSON your model streams stays truncated mid-token for the whole generation. What matters is a clean strict parser plus a small repair layer, not a second parser. The code follows.

Every RAG system starts here, and most candidates ship the same bug: they size the chunk in characters and call it tokens. Below is running code for all four strategies, plus the infinite loop in the overlap arithmetic and the one in the recursion.

Writing the regexes takes ten minutes. The half of the question that separates candidates is the evaluation: a detector that blocks 'ignore the noise in the data' has shipped a bug to every analyst using your product. Here is the layered detector and the harness that proves it works.

This code runs on every single response, so it sits on the critical path and a heavyweight check here doubles your p95. The signal is in the ordering, the fail-open decision when the guardrail times out, and the fact that you cannot un-send a token you already streamed.

Anyone can call a rerank endpoint. The word doing the work in this question is 'prove': you have to implement DCG, ideal DCG, and nDCG@k from graded labels and report the before-and-after. Plus the ceiling nobody mentions until it bites them in production.

The naive version splits on newlines and works right up until a TCP chunk lands mid-line. Then there is the error that arrives after a 200 OK, the user who closes the tab while you keep paying for tokens, and JSON you cannot parse until it closes.