pipelines

Hand-tuned prompts break on the next model upgrade and fail to compose across pipeline stages. Frameworks like DSPy compile prompts against a metric instead. The signal is treating prompting as an optimized program. Here is the answer.

Pipelines fail and retry, so the real question is whether a retry corrupts your data. What interviewers look for is idempotent writes (not 'exactly-once delivery') alongside watermarks for late data. Here is how to keep re-runs safe without double-counting.

Bad data quietly corrupts everything downstream (dashboards, models, decisions), and 'it ran without error' is not the same as 'it's correct.' The signal is the categories of checks and failing loud at the boundary.

A new column is safe; a rename quietly corrupts every dashboard downstream. The signal is knowing which changes stay compatible, enforcing data contracts in CI, and running expand-contract migrations for the breaking ones.

A backfill that double-counts or knocks over the live pipeline is worse than the bug it fixes. The signal is idempotent partition-scoped rewrites, keeping backfill compute isolated from live runs, and validating before the swap.

The rookie reply is 'retrain weekly.' The experienced call is a hybrid: drift-triggered retraining backed by a max-staleness fallback, plus a gate requiring any new model to beat the incumbent. Here is how to think it through.

A feature store is simple to define and simple to botch. The signal is naming the precise bug it prevents (training-serving skew) and the point-in-time correctness that blocks label leakage. Here is the full picture.

ML workflows are multi-step DAGs (ingest, feature, train, eval, deploy), and orchestrators run them reliably. What matters is the DAG/scheduling model plus what's genuinely ML-specific: data deps, versioned artifacts, drift triggers, eval gates.