PyData Amsterdam 2023

Promptly Evaluating Prompts with Bayesian Tournaments
09-14, 14:10–14:40 (Europe/Amsterdam), Foo (main)

Pick your next hot LLM prompt using a Bayesian tournament! Get a quick LLM dopamine hit with a side of decision theory vegetables. It's Bayesian Thunderdome: many prompts enter, one prompt leaves.


How do you chose the best LLM prompt systematically beyond guessing and vibes? Use the winner of a Bayesian tournament! Get a quick dopamine hit from fun LLM prompt magic with a side of Bayesian decision theory vegetables. If you are doing stuff with LLMs — you'll get a serious tool to improve your prompting game. If you're not using LLMs — you'll learn about Bayesian tournaments. They are not well known but have wide applicability: they help you optimally choose a winner using a minimal number of matches.


Prior Knowledge Expected

Previous knowledge expected

I've helped found multiple start-ups, including CorticalLabs an AI+Biotech company working on "Synthetic Biological Intelligence". I've co-authored several papers and patents in deep learning and neuroscience. I've made a mess in more than a dozen programming languages over my career. My stack is full. I've worked on custom neural interface hardware to web apps and everything in between. I've won a few hack-a-thons. I started the Machine Learning and AI meetup in Melbourne Australia, helped found & organize the Compose :: Melbourne conference. I have two cats, I scoop their poop most days.