Learn Poker Video Source & Info:
In the past few years, poker AIs have defeated the top poker players in the world. In this video, I discuss the Counterfactual Regret Minimization (CFR) algorithm that make superhuman poker bots possible. Be sure to check out Taskade, a great tool for project management, productivity, and collaboration: https://www.taskade.com
Game theory says that there is a Nash equilibrium in poker (meaning an “optimal” solution). In 2017, CMU’s poker bot, Libratus, defeated 4 world-renowned poker players in heads up, at 99.98% statistical significance. In 2019, Pluribus, another CMU poker bot, defeated pros in 6-player No Limit Hold’em. The algorithm behind it all is from a domain of computer science called reinforcement learning. It is a self-play algorithm that learns the optimal strategy by playing against itself. The Counterfactual Regret Minimization (CFR) algorithm decides which decisions to make based off where it might minimize the most regret. In this video, I explain how this algorithm works!
Some of you might want to code your own poker bot. Some of you might be working on other projects. Either way, you should use Taskade! It’s a great tool for managing projects and being super productive. It’s also awesome for streamlining tasks and keeping track of collaboration. Best of all, it is simple to use and free!!! Check it out here: https://www.taskade.com
Timestamps:
0:00 Intro
0:56 Reinforcement Learning
2:34 Basic Idea of CFR
4:04 Game Tree and Regret
7:27 Creating Abstractions
11:38 Putting It Together
12:33 Superhuman AI Performance
Papers/Resources:
http://modelai.gettysburg.edu/2013/cfr/cfr.pdf
https://papers.nips.cc/paper/2012/file/3df1d4b96d8976ff5986393e8767f5b2-Paper.pdf
https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.295.2143&rep=rep1&type=pdf
Counterfactual Regret Minimization – the core of Poker AI beating professional players
https://icml.cc/media/Slides/icml/2019/102(11-11-00)-11-12-15-4443-deep_counterfac.pdf
Feel free to leave any questions.
Please consider subscribing if you liked this video: https://www.youtube.com/c/ycubed?sub_confirmation=1
Thanks for watching everyone!
~~~~~~~~~~~~~~~~~~~~~~~~
Follow me on Instagram: https://www.instagram.com/kylieyying
Follow me on Twitter: https://www.twitter.com/kylieyying
Check out my website: https://www.kylieying.com
Source: YouTube
What are your thoughts on superhuman AIs?? P.S. Check out Taskade, a super awesome platform for managing collaboration and organizing projects. It's simple to use and free! https://www.taskade.com
Be sure to subscribe for more fun content like this 🙂
Thanks🙏🙏
Wow
New Follower here..:). Keep it up..
Is she Korean descend?
Bro this could have been the biggest brain video if you decided for the AI to take into account other people’s regrets and use that against them. I have not heard a poker AI developer said they developed their AI that way (or at least they didn’t reveal that)
But the real question is did you make money on online poker. Asking for a friend I might make one but I heard they have been taking down such outlier performers.
I really liked the video – but I think we should not consider CFR as reinforcement learning since CFR aims to minimize regret (not to maximize reward).
It looks like it's a small difference, but in practice, it's the difference between trying to possibly exploit our opponent (maximize reward) and minimizing exploitability (CFR) – which also means CFR is unable to be adaptive if I get it correctly.
RE RL for poker, I think Noam Brown has some new related work.
Though I am not an expert for RL or CFRs as I do mostly NLP…
What do you think?
I love your content, please keep making more if you are able!
That's all the steps used in training right? I'm trying to understand the Sub-game solving part……..