Riccardo Amadio
Senior Data Engineer at Agile Lab with a background of Data Scientist and Software Engineer.
When I don't work with data pipelines , I juggle between closing some of my 100+ open tabs on the browser and my true passion: collecting stars on GitHub 🔭🌟. In this treasure trove of more than 2,000 repositories, I am pretty sure I can find any tool to solve a problem, and I can’t wait to share them with you.
Sessions
Bored of old pipeline orchestrator? Difficult to understand if data is up-to-date? Trouble with development workflow of data pipeline?
Dagster, an open-source tool, offers a unique paradigm that simplifies the orchestration and management of data pipelines.
By adopting declarative principles, data engineers and data scientists can build scalable, maintainable, and reliable pipelines effortlessly.
We will commence with an introduction to Dagster, covering its fundamental concepts to ensure a comprehensive understanding of the material.
Subsequently, we will explore practical scenarios and use cases, with also DBT for empower the power of SQL language.
Minutes 0-5: Explain the design pattern problem of actual data pipeline framework.
Minutes 5-15: Introduction to Dagster and its core concepts.
Minutes 10-25: Practical examples of building declarative data pipelines with Dagster, with also DBT, the power of gRPC server.
Minutes 25-30: Q&A and conclusion.