Reproducible data science over data lakes: replayable data p...

Reproducible data science over data lakes: replayable data pipelines with Bauplan and Nessie.

Reproducible data science is enabled through Bauplan and Nessie, providing time-travel and branching semantics on data lakes, decoupling compute from data management.

arxiv.org

RelatedHighlights

IT is going to have to maintain the enterprise analytic data set structures and processes within the environment where it will be deployed.

Bill Franks • Taming The Big Data Tidal Wave: Finding Opportunities in Huge Data Streams with Advanced Analytics (Wiley and SAS Business Series)

Dune: The Data Must Flow | The Generalist

Mario Gabriele readthegeneralist.com

Drive architectures using data:

Amazon Web Services • AWS Well-Architected Framework (AWS Whitepaper)

Cube is an open source semantic layer which recently integrated with LangChain to solve these issues in a different way. You can use YAML, Javascript, or Python to create definitions of different metrics, measures and dimensions for your data:

Bill Franks • Taming The Big Data Tidal Wave: Finding Opportunities in Huge Data Streams with Advanced Analytics (Wiley and SAS Business Series)

Dune: The Data Must Flow | The Generalist

Amazon Web Services • AWS Well-Architected Framework (AWS Whitepaper)

Artyom Keydunov • Powering your Copilot for Data – with Artem Keydunov of Cube.dev