09-16, 10:50–11:20 (Europe/Amsterdam), Foo (main)
Optimizing machine learning models using regular metrics is a common practice in the industry. However, aligning model optimization with business metrics is closely tied to the objectives of the business and is highly valued by product managers and other stakeholders. This talk delves into the process of training machine learning models based on business metrics in order to enhance economic outcomes. With a primary focus on data scientists and machine learning practitioners, this talk explores techniques, methodologies, and real-world applications that harness the power of business metrics to propel machine learning models and foster business success. We will present a specific case study that demonstrates how we utilized business metrics at Booking.com that brought significant impact on model performance on business outcomes. Specifically, we will discuss our approaches to leveraging business metrics for hyperparameter tuning and reducing model complexity, which instill greater confidence within our team when deploying improved models to production.
Description
This talk aims to equip data scientists and machine learning practitioners with the knowledge and tools to train machine learning models on business metrics effectively. We will delve into the process of hyperparameter tuning, algorithm selection, and model evaluation specifically tailored for optimizing economic outcomes. A real-world use case at Booking.com will demonstrate the transformative power of this approach.
Outline
- Introduction to training machine learning models on machine learning metrics versus business metrics
- Overview of the significance of leveraging business metrics to improve machine learning models' performance on business metrics
- Introduction to machine learning algorithms suitable for modeling business metrics to drive economic optimizations
- Metrics and evaluation, and training techniques specific to assessing the business impact of machine learning models
- Showcasing practical use case at Booking.com where training models on business metrics has led to significant improvements in economic outcomes
Central Focus
Training machine learning models on business metrics present a powerful methodology for optimizing economic outcomes. By incorporating relevant business data and metrics into the modeling process, data scientists and machine learning practitioners can drive substantial improvements in economic performance. This talk will provide attendees with the necessary insights and techniques to apply this approach successfully.
Key Takeaways
- Understanding the importance of training machine learning models on business metrics for economic optimizations
- Familiarity with machine learning algorithms suitable for modeling business metrics and driving economic outcomes
- Strategies for evaluating and quantifying the economic impact of machine learning models
Real-world inspiration and practical insights for applying this approach to boost economic outcomes
Expected Background Knowledge
Attendees should have a foundation in machine learning concepts and practical experience with data science techniques, in particular, knowledge of Gradient Boosting Machine (GBM) models. Familiarity with business metrics, economic principles, and optimization objectives will be beneficial but not required.
We aim to deliver an informative and practical talk that caters to data scientists and machine learning practitioners. Attendees will gain actionable insights, methodologies, and real-world examples to effectively train machine learning models on business metrics, leading to enhanced economic outcomes.
Previous knowledge expected
I am a machine learning scientist at Booking.com working on personalized discounts under budget constraints.
I have a PhD in Computer Science from the Delft University of Technology. During my PhD, I interned as an applied scientist at Amazon Alexa Shopping, where I worked on finding proxies for what customers find relevant when comparing products during their search shopping journey in order to empower Amazon recommendation systems. Before that I obtained a BSc and MSc in Computer Science from the Federal University of Minas Gerais, visited research labs at NYU and the University of Quebec, and worked as a software engineer intern in a news recommendation system start up.