Reproducible data science over data lakes: replayable data p...

Reproducible data science over data lakes: replayable data pipelines with Bauplan and Nessie.

Reproducible data science is enabled through Bauplan and Nessie, providing time-travel and branching semantics on data lakes, decoupling compute from data management.

arxiv.org

RelatedHighlights

Expose Delta Tables via REST APIs

Git repo to test 3 architectures to expose delta tables via REST APIs. See also my blogpost here. Architectures can be described as follows:

Architecture A: Direct, Web App with DuckDB. In this architecture, APIs are directly connecting to the delta table and there is no layer in between. This implies that all data

GitHub - rebremer/expose-deltatable-via-restapi

Initial exploration, focused research, and initial prototypes are fair game. Once planning starts for a larger, more formal implementation of a prototype, it is no longer in the domain of the center. At that point, the center’s resources need to move on to the next problem

Bill Franks • Taming The Big Data Tidal Wave: Finding Opportunities in Huge Data Streams with Advanced Analytics (Wiley and SAS Business Series)

Dune: The Data Must Flow | The Generalist

Mario Gabriele readthegeneralist.com

How DoorDash Designed a Successful Write-Heavy Scalable and Reliable Inventory Platform

Chuanpin Zhu doordash.engineering