dimensionality reduction

A from-scratch favorite that probes linear algebra fluency. The signal is centering first, using SVD instead of forming the covariance matrix, and reading variance off the singular values. Here is the implementation and the details interviewers push on.

Dimensionality questions probe linear-algebra intuition and practical judgment together. What interviewers reward is what PCA really does (project onto max-variance directions), why high dimensions hurt, and the honest catch: it is unsupervised, so it can discard the exact direction your label needs. Here is the answer.

These three get lumped together, yet they answer different questions: one is a preprocessing tool, two are eyeballing tools. The candidates who pass know precisely which structure each keeps and which it silently discards.

LDA is the supervised cousin of PCA, and the contrast is a favorite. What matters is knowing that LDA uses the labels to maximize class separation while PCA only chases variance, plus the C minus 1 dimension cap that trips up most candidates.

High-dimensional data breaks the intuitions and methods that work in two or three dimensions. What matters is naming the concrete effects (distances concentrate, data goes sparse, overfitting climbs) and the mitigations that actually move the needle.