<description>&lt;p&gt;Ilya &amp; I discuss SSI’s strategy, the problems with pre-training, how to improve the generalization of AI models, and how to ensure AGI goes well.&lt;/p&gt;&lt;p&gt;Watch on &lt;a target="_blank" href="https://youtu.be/aR20FWCCjAs"&gt;YouTube&lt;/a&gt;; read the &lt;a target="_blank" href="https://www.dwarkesh.com/p/ilya-sutskever-2"&gt;transcript&lt;/a&gt;.&lt;/p&gt;&lt;p&gt;Sponsors&lt;/p&gt;&lt;p&gt;* &lt;a target="_blank" href="https://gemini.google"&gt;Gemini 3&lt;/a&gt; is the first model I’ve used that can find connections I haven’t anticipated. I recently wrote a blog post on RL’s information efficiency, and Gemini 3 helped me think it all through. It also generated the relevant charts and ran toy ML experiments for me with zero bugs. Try Gemini 3 today at &lt;a target="_blank" href="https://gemini.google"&gt;gemini.google&lt;/a&gt;&lt;/p&gt;&lt;p&gt;* &lt;a target="_blank" href="https://labelbox.com/dwarkesh"&gt;Labelbox&lt;/a&gt; helped me create a tool to transcribe our episodes! I’ve struggled with transcription in the past because I don’t just want verbatim transcripts, I want transcripts reworded to read like essays. Labelbox helped me generate the &lt;em&gt;exact&lt;/em&gt; data I needed for this. If you want to learn how Labelbox can help you (or if you want to try out the transcriber tool yourself), go to &lt;a target="_blank" href="https://labelbox.com/dwarkesh"&gt;labelbox.com/dwarkesh&lt;/a&gt;&lt;/p&gt;&lt;p&gt;* &lt;a target="_blank" href="https://sardine.ai/dwarkesh"&gt;Sardine&lt;/a&gt; is an AI risk management platform that brings together thousands of device, behavior, and identity signals to help you assess a user’s risk of fraud &amp; abuse. Sardine also offers a suite of agents to automate investigations so that as fraudsters use AI to scale their attacks, you can use AI to scale your defenses. Learn more at &lt;a target="_blank" href="https://sardine.ai/dwarkesh"&gt;sardine.ai/dwarkesh&lt;/a&gt;&lt;/p&gt;&lt;p&gt;To sponsor a future episode, visit &lt;a target="_blank" href="https://www.dwarkesh.com/advertise"&gt;dwarkesh.com/advertise&lt;/a&gt;.&lt;/p&gt;&lt;p&gt;Timestamps&lt;/p&gt;&lt;p&gt;(00:00:00) – Explaining model jaggedness&lt;/p&gt;&lt;p&gt;(00:09:39) - Emotions and value functions&lt;/p&gt;&lt;p&gt;(00:18:49) – What are we scaling?&lt;/p&gt;&lt;p&gt;(00:25:13) – Why humans generalize better than models&lt;/p&gt;&lt;p&gt;(00:35:45) – SSI’s plan to straight-shot superintelligence&lt;/p&gt;&lt;p&gt;(00:46:47) – SSI’s model will learn from deployment&lt;/p&gt;&lt;p&gt;(00:55:07) – How to think about powerful AGIs&lt;/p&gt;&lt;p&gt;(01:18:13) – “We are squarely an age of research company”&lt;/p&gt;&lt;p&gt;(01:20:23) – Self-play and multi-agent&lt;/p&gt;&lt;p&gt;(01:32:42) – Research taste&lt;/p&gt; &lt;br/&gt;&lt;br/&gt;Get full access to Dwarkesh Podcast at &lt;a href="https://www.dwarkesh.com/subscribe?utm_medium=podcast&amp;#38;utm_campaign=CTA_4"&gt;www.dwarkesh.com/subscribe&lt;/a&gt;</description>

Dwarkesh Podcast

Dwarkesh Patel

Ilya Sutskever – We're moving from the age of scaling to the age of research

NOV 25, 202596 MIN
Dwarkesh Podcast

Ilya Sutskever – We're moving from the age of scaling to the age of research

NOV 25, 202596 MIN

Description

<p>Ilya & I discuss SSI’s strategy, the problems with pre-training, how to improve the generalization of AI models, and how to ensure AGI goes well.</p><p>Watch on <a target="_blank" href="https://youtu.be/aR20FWCCjAs">YouTube</a>; read the <a target="_blank" href="https://www.dwarkesh.com/p/ilya-sutskever-2">transcript</a>.</p><p>Sponsors</p><p>* <a target="_blank" href="https://gemini.google">Gemini 3</a> is the first model I’ve used that can find connections I haven’t anticipated. I recently wrote a blog post on RL’s information efficiency, and Gemini 3 helped me think it all through. It also generated the relevant charts and ran toy ML experiments for me with zero bugs. Try Gemini 3 today at <a target="_blank" href="https://gemini.google">gemini.google</a></p><p>* <a target="_blank" href="https://labelbox.com/dwarkesh">Labelbox</a> helped me create a tool to transcribe our episodes! I’ve struggled with transcription in the past because I don’t just want verbatim transcripts, I want transcripts reworded to read like essays. Labelbox helped me generate the <em>exact</em> data I needed for this. If you want to learn how Labelbox can help you (or if you want to try out the transcriber tool yourself), go to <a target="_blank" href="https://labelbox.com/dwarkesh">labelbox.com/dwarkesh</a></p><p>* <a target="_blank" href="https://sardine.ai/dwarkesh">Sardine</a> is an AI risk management platform that brings together thousands of device, behavior, and identity signals to help you assess a user’s risk of fraud & abuse. Sardine also offers a suite of agents to automate investigations so that as fraudsters use AI to scale their attacks, you can use AI to scale your defenses. Learn more at <a target="_blank" href="https://sardine.ai/dwarkesh">sardine.ai/dwarkesh</a></p><p>To sponsor a future episode, visit <a target="_blank" href="https://www.dwarkesh.com/advertise">dwarkesh.com/advertise</a>.</p><p>Timestamps</p><p>(00:00:00) – Explaining model jaggedness</p><p>(00:09:39) - Emotions and value functions</p><p>(00:18:49) – What are we scaling?</p><p>(00:25:13) – Why humans generalize better than models</p><p>(00:35:45) – SSI’s plan to straight-shot superintelligence</p><p>(00:46:47) – SSI’s model will learn from deployment</p><p>(00:55:07) – How to think about powerful AGIs</p><p>(01:18:13) – “We are squarely an age of research company”</p><p>(01:20:23) – Self-play and multi-agent</p><p>(01:32:42) – Research taste</p> <br/><br/>Get full access to Dwarkesh Podcast at <a href="https://www.dwarkesh.com/subscribe?utm_medium=podcast&#38;utm_campaign=CTA_4">www.dwarkesh.com/subscribe</a>