schema evolution

A new column is safe; a rename quietly corrupts every dashboard downstream. The signal is knowing which changes stay compatible, enforcing data contracts in CI, and running expand-contract migrations for the breaking ones.

Raw clickstream lands in object storage and somehow turns into clean tables, dashboards, and ML features. The design questions are table format, the bronze-silver-gold layering, batch versus streaming, and how you handle schema drift and late data. Here is the lakehouse blueprint.

A data engineer renames a column and three production models quietly begin scoring on garbage. Features form a shared contract, and altering them demands the same care as a breaking API change. Here is the discipline it takes.