Pipelines

Bad data is one of the most insidious problems in data engineering. Unlike a pipeline failure — which is loud, […]

Schema changes are inevitable. This is not a pessimistic take — it is just the reality of building data pipelines

If you have spent any time building data pipelines, you have encountered this decision more times than you can count.

There is a meaningful difference between a pipeline that works and a pipeline that is reliable. A pipeline that works