In this episode, Stanislav Khromov joins the Svelte Radio team to discuss his work on Svelte Bench, abenchmarking tool that scientifically measures how well different LLMs understand and write Svelte 5 code.The conversation explores the challenges of AI-assisted coding with Svelte 5, the development of the officialSvelte MCP (Model Context Protocol) that provides LLMs with documentation and auto-fixing capabilities, andthe surprising performance differences between major AI providers. Stanislav also shares insights on workingwith AI tools, the future of local models, and the economics of AI coding assistants.Notes<ul><li><a href="https://stanislav.garden">Stanislav Khromov</a></li><li><a href="https://khromov.github.io/svelte-bench/benchmark-results-merged.html">Svelte Bench</a><ul><li><a href="https://github.com/khromov/svelte-bench">GitHub</a></li></ul></li><li><a href="https://svelte.dev/docs/mcp/overview">Svelte MCP</a></li><li><a href="https://www.wheresyoured.at/costs/">Anthropic Spending VC Money</a></li></ul>Unpopular Opinions<ul><li>Kevin: Serverless is overrated</li><li>Stanislav: OpenAI has the worst models of the big three</li><li>Antony: Heat Pumps are bad<ul><li><a href="https://www.youtube.com/c/TechnologyConnections/videos">Technology Connections</a></li><li><a href="https://www.youtube.com/@TechIngredients/videos">Tech Ingredients</a></li></ul></li></ul>Picks<ul><li>Kevin: <a href="https://eu.kasaigrills.com/">Kasai Hibachi Grill</a> (now tested, and can confirm, it is PENG!) Stanislav: <a href="https://store.steampowered.com/app/2062430/BALL_x_PIT/">Ball x Pit</a> Antony: Saunas By the Sea</li></ul>

<description>
 &lt;p&gt;In this episode, Stanislav Khromov joins the Svelte Radio team to discuss his work on Svelte Bench, a&lt;/p&gt;&lt;p&gt;benchmarking tool that scientifically measures how well different LLMs understand and write Svelte 5 code.&lt;/p&gt;&lt;p&gt;The conversation explores the challenges of AI-assisted coding with Svelte 5, the development of the official&lt;/p&gt;&lt;p&gt;Svelte MCP (Model Context Protocol) that provides LLMs with documentation and auto-fixing capabilities, and&lt;/p&gt;&lt;p&gt;the surprising performance differences between major AI providers. Stanislav also shares insights on working&lt;/p&gt;&lt;p&gt;with AI tools, the future of local models, and the economics of AI coding assistants.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;a href="https://stanislav.garden"&gt;Stanislav Khromov&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://khromov.github.io/svelte-bench/benchmark-results-merged.html"&gt;Svelte Bench&lt;/a&gt;&lt;ul&gt;&lt;li&gt;&lt;a href="https://github.com/khromov/svelte-bench"&gt;GitHub&lt;/a&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://svelte.dev/docs/mcp/overview"&gt;Svelte MCP&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://www.wheresyoured.at/costs/"&gt;Anthropic Spending VC Money&lt;/a&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;strong&gt;Unpopular Opinions&lt;/strong&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Kevin: Serverless is overrated&lt;/li&gt;&lt;li&gt;Stanislav: OpenAI has the worst models of the big three&lt;/li&gt;&lt;li&gt;Antony: Heat Pumps are bad&lt;ul&gt;&lt;li&gt;&lt;a href="https://www.youtube.com/c/TechnologyConnections/videos"&gt;Technology Connections&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://www.youtube.com/@TechIngredients/videos"&gt;Tech Ingredients&lt;/a&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;&lt;strong&gt;Picks&lt;/strong&gt;&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Kevin: &lt;a href="https://eu.kasaigrills.com/"&gt;Kasai Hibachi Grill&lt;/a&gt; (now tested, and can confirm, it is PENG!)&lt;br&gt;Stanislav: &lt;a href="https://store.steampowered.com/app/2062430/BALL_x_PIT/"&gt;Ball x Pit&lt;/a&gt;&lt;br&gt;Antony: Saunas By the Sea&lt;/li&gt;&lt;/ul&gt;
 </description>

Svelte Radio

Benchmarking AI with Stanislav Khromov

Benchmarking AI with Stanislav Khromov

Description