How to Build a Superhuman Poker AI using CFR | Creating a Poker Bot Part 2

Learn Poker Video Source & Info:

In the past few years, poker AIs have defeated the top poker players in the world. In this video, I discuss the Counterfactual Regret Minimization (CFR) algorithm that make superhuman poker bots possible. Be sure to check out Taskade, a great tool for project management, productivity, and collaboration: https://www.taskade.com

Game theory says that there is a Nash equilibrium in poker (meaning an “optimal” solution). In 2017, CMU’s poker bot, Libratus, defeated 4 world-renowned poker players in heads up, at 99.98% statistical significance. In 2019, Pluribus, another CMU poker bot, defeated pros in 6-player No Limit Hold’em. The algorithm behind it all is from a domain of computer science called reinforcement learning. It is a self-play algorithm that learns the optimal strategy by playing against itself. The Counterfactual Regret Minimization (CFR) algorithm decides which decisions to make based off where it might minimize the most regret. In this video, I explain how this algorithm works!

Some of you might want to code your own poker bot. Some of you might be working on other projects. Either way, you should use Taskade! It’s a great tool for managing projects and being super productive. It’s also awesome for streamlining tasks and keeping track of collaboration. Best of all, it is simple to use and free!!! Check it out here: https://www.taskade.com

Timestamps:
0:00 Intro
0:56 Reinforcement Learning
2:34 Basic Idea of CFR
4:04 Game Tree and Regret
7:27 Creating Abstractions
11:38 Putting It Together
12:33 Superhuman AI Performance

Papers/Resources:
http://modelai.gettysburg.edu/2013/cfr/cfr.pdf
https://papers.nips.cc/paper/2012/file/3df1d4b96d8976ff5986393e8767f5b2-Paper.pdf
https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.295.2143&rep=rep1&type=pdf

Counterfactual Regret Minimization – the core of Poker AI beating professional players


https://icml.cc/media/Slides/icml/2019/102(11-11-00)-11-12-15-4443-deep_counterfac.pdf

Feel free to leave any questions.

Please consider subscribing if you liked this video: https://www.youtube.com/c/ycubed?sub_confirmation=1

Thanks for watching everyone!
~~~~~~~~~~~~~~~~~~~~~~~~
Follow me on Instagram: https://www.instagram.com/kylieyying
Follow me on Twitter: https://www.twitter.com/kylieyying
Check out my website: https://www.kylieying.com

Source: YouTube

Share this video:
How to Build a Superhuman Poker AI using CFR | Creating a Poker Bot Part 2

10 thoughts on “How to Build a Superhuman Poker AI using CFR | Creating a Poker Bot Part 2

  1. What are your thoughts on superhuman AIs?? P.S. Check out Taskade, a super awesome platform for managing collaboration and organizing projects. It's simple to use and free! https://www.taskade.com

    Be sure to subscribe for more fun content like this 🙂

  2. Bro this could have been the biggest brain video if you decided for the AI to take into account other people’s regrets and use that against them. I have not heard a poker AI developer said they developed their AI that way (or at least they didn’t reveal that)

  3. But the real question is did you make money on online poker. Asking for a friend I might make one but I heard they have been taking down such outlier performers.

  4. I really liked the video – but I think we should not consider CFR as reinforcement learning since CFR aims to minimize regret (not to maximize reward).
    It looks like it's a small difference, but in practice, it's the difference between trying to possibly exploit our opponent (maximize reward) and minimizing exploitability (CFR) – which also means CFR is unable to be adaptive if I get it correctly.

    RE RL for poker, I think Noam Brown has some new related work.

    Though I am not an expert for RL or CFRs as I do mostly NLP…
    What do you think?

  5. That's all the steps used in training right? I'm trying to understand the Sub-game solving part……..

Comments are closed.