Daniel van der Ende
Daniel van der Ende is a Data Engineer at Xebia Data. He enjoys working on high performance distributed computation with Spark, empowering data scientists by helping them to run their models on very large datasets with high performance. He is an Apache Spark and Apache Airflow contributor and speaker at conferences and meetups.
Sessions
09-15
14:20
30min
Return to Data's Inferno: are the 7 layers of data testing hell still relevant?
Daniel van der Ende
Back in 2018, a blogpost titled "Data's Inferno: 7 circles of data testing hell with Airflow" presented a layered approach to data quality checks in data applications and pipelines. Now, 5 years later, this talk looks back at Data's Inferno and surveys what has changed but also what hasn't in the space of ensuring high data quality.
Bar