Idempotent Data Pipelines

Data pipelines fail and get rerun, so a pipeline has to be idempotent: running it again yields the same result rather than duplicated or corrupted data. You get there with insert-overwrite by partition, MERGE/upsert keyed on a business id, and deterministic transforms, instead of blind appends that double-count on retry. AI, ML, and GenAI interviews probe it because flaky pipelines are the norm, and a non-idempotent pipeline turns a routine retry into duplicated revenue numbers or a corrupted table.