3,411
likes
173
comments
daily view 0
monthly view 0

Share

Live Analytics

Comments 0

What rebuilding AlphaGo teaches us about self-play, RL, and future of LLMs - Eric Jang Analytics Table

Income Estimates for What rebuilding AlphaGo teaches us about self-play, RL, and future of LLMs - Eric Jang

Based on this YouTube video's total view count of 180K views and industry-standard rates, the estimated total earning is $126 - $361 through ad revenue. Historical data is not yet available to calculate daily, weekly, or monthly averages.

About What rebuilding AlphaGo teaches us about self-play, RL, and future of LLMs - Eric Jang

Explore What rebuilding AlphaGo teaches us about self-play, RL, and future of LLMs - Eric Jang with 180,370 views, 3,411 likes, and 173 comments. Experience the impact of this video content that has captured audience attention.

Eric Jang walks through how to build AlphaGo from scratch, but with modern AI tools. Sometimes you understand the future better by stepping backward. AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning from experience, and self-play. You have to go back to 2017 to get insight into how the more general AIs of the future might learn. Once he explained how AlphaGo works, it gave us the context to have a discussion about how RL works in LLMs and how it could work better – naive policy gradient RL has to figure out which of the 100k+ tokens in your trajectory actually got you the right answer, while AlphaGo’s MCTS suggests a strictly better action every single move, giving you a training target that sidesteps the credit assignment problem. The way humans learn is surely closer to the second. Eric also kickstarted an Autoresearch loop on his project. And it was very interesting to discuss which parts of AI research LLMs can already automate pretty well (implementing and running experiments, optimizing hyperparameters) and which they still struggle with (choosing the right question to investigate next, escaping research dead ends). Informative to all the recent discussion about when we should expect an intelligence explosion, and what it would look like from the inside. π„ππˆπ’πŽπƒπ„ π‹πˆππŠπ’ * Check out the flashcards I wrote to retain the insights: https://flashcards.dwarkesh.com/eric-jang/ * Transcript: https://www.dwarkesh.com/p/eric-jang * Errata around 1:33:44: https://evjang.com/alphago-tutorial/llm_rl_variance.pdf π’ππŽππ’πŽπ‘π’ - Cursor's agent SDK let me build a pipeline to generate flashcards for this episode. For each card, I had an agent read the transcript, ingest blackboard screenshots, generate an SVG visual, and run everything through a critic. A durable agent is much better at this kind of work than a chain of LLM calls, and Cursor's SDK made it easy. Check out the cards at https://flashcards.dwarkesh.com and get started with the SDK at https://cursor.com/dwarkesh - Jane Street gave me a real deep-dive tour of one of their datacenters. I got to ask a bunch of questions to Ron Minsky, who co-leads Jane Street's tech group, and Dan Pontecorvo, who runs Jane Street's physical engineering team. They were willing to literally pull up the floorboards and take out racks to explain how everything works. Check out the full tour at https://janestreet.com/dwarkesh To sponsor a future episode, visit https://dwarkesh.com/advertise. π“πˆπŒπ„π’π“π€πŒππ’ 00:00:00 – Basics of Go 00:08:06 – Monte Carlo Tree Search 00:31:53 – What the neural network does 01:00:22 – Self-play 01:25:27 – Alternative RL approaches 01:45:36 – Why doesn’t MCTS work for LLMs 02:00:58 – Off-policy training 02:11:51 – RL is even more information inefficient than you thought 02:22:05 – Automated AI researchers

About YouTube Real-Time View Count

With SocialCounts.org’s view counter, track your YouTube video’s live view count and YouTube likes count in real time with fast, reliable updates.

Watch every YouTube video live view count rise with our real-time YouTube views trackerβ€”built for accuracy and minimal delay.

Follow YouTube real time views as they happen, using our dedicated view counter for YouTube videos.

Get up-to-date live view count on YouTube and see real-time growth with SocialCounts.org’s smart tracking tools.

Embed Widget

Parameters:

  • fullscreen=true - Fullscreen counter
  • graph=true - Live graph chart
  • counter=0/1/2 - Select counter (0=likes, 1=views, 2=comments)
URL

Click to copy the embed URL to your clipboard