09-15, 11:30–12:00 (Europe/Amsterdam), Foo (main)
Data scientists and analysts are using quasi-experimental methods to make recommendations based on causality instead of randomized control trials. While these methods are easy to use, their assumptions can be complex to explain. This talk will explain these assumptions for data scientists and analysts without in-depth training of causal inference so they can use and explain these methods more confidently to change people's minds using data.
Instead of relying solely on randomized control trials (also known as A/B tests), which are considered the gold standard for inferring causality, data scientists and analysts are increasingly turning to quasi-experimental methods to make recommendations based on causality. These methods, including open-source libraries such as CausalImpact (originally an R package but with numerous Python ports), are easy to use, but their assumptions can be complex to explain. I will break down these assumptions and explain how they can help practitioners determine when to use these methods (and when not to use them), using examples from the world of digital language learning. The key takeaway is that when it comes to changing people's minds using data, explaining assumptions to decision-makers is just as important as understanding the underlying statistics.
Outline
- Minute 0-5: Introduction and Motivation
- Minute 5-10: Difference-in-Difference / Bayesian Structural Time-Series
- Minute 10-15: Case - Conversion effects of content changes based language-pair specific releases at Babbel
- Minute 15-20: Regression Discontinuity Design
- Minute 20-25: Case - Estimating motivational effects of language assessment
- Minute 25-30: Wrap-up / Take-Aways
Attendees should have basic knowledge of statistics and causality to get the most out of this talk.
Previous knowledge expected
As Head of Product Data at Babbel, I lead data-scientists, analysts and engineers to improve decision-making of people and machines. Before joining Babbel I did quantitative research in Political Science and Political Economy.