r/dataengineering - Reddit

So what abstractions do we have as of today? For example, let’s take the resource abstraction (Dagster, Prefect, referred to as an operator in Airflow). You abstract complex environments and connections away with a simple construct like that. You have the immediate benefits of defining that once and using it in every task or pipeline with context.r... See more

Lots to chew on in Instacart S-1, but was curious what people make of this. Their Snowflake bill was $13M, $28M, $51M in 2020/21/22. 2023 bill will be $15M. Is that just "optimization"? Did they migrate? Is that indicative of the magnitude of optimizations? What's the read here?