<p><strong>Aishwarya Naresh Reganti</strong> and <strong>Kiriti Badam</strong> have helped build and launch more than 50 enterprise AI products across companies like OpenAI, Google, Amazon, and Databricks. Based on these experiences, they’ve developed a small set of best practices for building and scaling successful AI products. The goal of this conversation is to save you and your team a lot of pain and suffering.</p><p></p><p><strong>We discuss:</strong></p><p>1. Two key ways AI products differ from traditional software, and why that fundamentally changes how they should be built</p><p>2. Common patterns and anti-patterns in companies that build strong AI products versus those that struggle</p><p>3. A framework they developed from real-world experience to iteratively build AI products that create a flywheel of improvement</p><p>4. Why obsessing about customer trust and reliability is an underrated driver of successful AI products</p><p>5. Why evals aren’t a cure-all, and the most common misconceptions people have about them</p><p>6. The skills that matter most for builders in the AI era</p><p>—</p><p><strong>Brought to you by:</strong></p><p><a target="_blank" href="https://merge.dev/lenny"><strong>Merge</strong></a>—The fastest way to ship 220+ integrations: <a target="_blank" href="https://merge.dev/lenny">https://merge.dev/lenny</a></p><p><a target="_blank" href="https://strella.io/lenny"><strong>Strella</strong></a>—The AI-powered customer research platform: <a target="_blank" href="https://strella.io/lenny">https://strella.io/lenny</a></p><p><a target="_blank" href="https://www.brex.com/product/business-account?ref_code=bmk_dp_brand1H25_ln_new_fs"><strong>Brex</strong></a>—The banking solution for startups: <a target="_blank" href="https://www.brex.com/product/business-account?ref_code=bmk_dp_brand1H25_ln_new_fs">https://www.brex.com/product/business-account?ref_code=bmk_dp_brand1H25_ln_new_fs</a></p><p>—</p><p><strong>Transcript: </strong><a target="_blank" href="https://www.lennysnewsletter.com/p/what-openai-and-google-engineers-learned">https://www.lennysnewsletter.com/p/what-openai-and-google-engineers-learned</a></p><p>—</p><p><strong>My biggest takeaways (for paid newsletter subscribers): </strong><a target="_blank" href="https://www.lennysnewsletter.com/i/183007822/referenced">https://www.lennysnewsletter.com/i/183007822/referenced</a></p><p>—</p><p><strong>Get 15% off Aishwarya and Kiriti’s Maven course, </strong><strong><em>Building Agentic AI Applications with a Problem-First Approach</em></strong><strong>, using this link: </strong><a target="_blank" href="https://bit.ly/3V5XJFp">https://bit.ly/3V5XJFp</a></p><p>—</p><p><strong>Where to find Aishwarya Naresh Reganti:</strong></p><p>• LinkedIn: <a target="_blank" href="https://www.linkedin.com/in/areganti">https://www.linkedin.com/in/areganti</a></p><p>• GitHub: <a target="_blank" href="https://github.com/aishwaryanr/awesome-generative-ai-guide">https://github.com/aishwaryanr/awesome-generative-ai-guide</a></p><p>• X: <a target="_blank" href="https://x.com/aish_reganti">https://x.com/aish_reganti</a></p><p>—</p><p><strong>Where to find Kiriti Badam:</strong></p><p>• LinkedIn: <a target="_blank" href="https://www.linkedin.com/in/sai-kiriti-badam">https://www.linkedin.com/in/sai-kiriti-badam</a></p><p>• X: <a target="_blank" href="https://x.com/kiritibadam">https://x.com/kiritibadam</a></p><p>—</p><p><strong>Where to find Lenny:</strong></p><p>• Newsletter: <a target="_blank" href="https://www.lennysnewsletter.com">https://www.lennysnewsletter.com</a></p><p>• X: <a target="_blank" href="https://twitter.com/lennysan">https://twitter.com/lennysan</a></p><p>• LinkedIn: <a target="_blank" href="https://www.linkedin.com/in/lennyrachitsky/">https://www.linkedin.com/in/lennyrachitsky/</a></p><p>—</p><p><strong>In this episode, we cover:</strong></p><p>(00:00) Introduction to Aishwarya and Kiriti</p><p>(05:03) Challenges in AI product development</p><p>(07:36) Key differences between AI and traditional software</p><p>(13:19) Building AI products: start small and scale</p><p>(15:23) The importance of human control in AI systems</p><p>(22:38) Avoiding prompt injection and jailbreaking</p><p>(25:18) Patterns for successful AI product development</p><p>(33:20) The debate on evals and production monitoring</p><p>(41:27) Codex team’s approach to evals and customer feedback</p><p>(45:41) Continuous calibration, continuous development (CC/CD) framework</p><p>(58:07) Emerging patterns and calibration</p><p>(01:01:24) Overhyped and under-hyped AI concepts</p><p>(01:05:17) The future of AI</p><p>(01:08:41) Skills and best practices for building AI products</p><p>(01:14:04) Lightning round and final thoughts</p><p>—</p><p><strong>Referenced:</strong></p><p>• LevelUp Labs: <a target="_blank" href="https://levelup-labs.ai/">https://levelup-labs.ai/</a></p><p>• Why your AI product needs a different development lifecycle: <a target="_blank" href="https://www.lennysnewsletter.com/p/why-your-ai-product-needs-a-different">https://www.lennysnewsletter.com/p/why-your-ai-product-needs-a-different</a></p><p>• <a target="_blank" href="http://Booking.com">Booking.com</a>: <a target="_blank" href="https://www.booking.com">https://www.booking.com</a></p><p>• Research paper on agents in production (by Matei Zaharia’s lab): <a target="_blank" href="https://arxiv.org/pdf/2512.04123">https://arxiv.org/pdf/2512.04123</a></p><p>• Matei Zaharia’s research on Google Scholar: <a target="_blank" href="https://scholar.google.com/citations?user=I1EvjZsAAAAJ&hl=en">https://scholar.google.com/citations?user=I1EvjZsAAAAJ&hl=en</a></p><p>• The coming AI security crisis (and what to do about it) | Sander Schulhoff: <a target="_blank" href="https://www.lennysnewsletter.com/p/the-coming-ai-security-crisis">https://www.lennysnewsletter.com/p/the-coming-ai-security-crisis</a></p><p>• Gajen Kandiah on LinkedIn: <a target="_blank" href="https://www.linkedin.com/in/gajenkandiah">https://www.linkedin.com/in/gajenkandiah</a></p><p>• Rackspace: <a target="_blank" href="https://www.rackspace.com">https://www.rackspace.com</a></p><p>• The AI-native startup: 5 products, 7-figure revenue, 100% AI-written code | Dan Shipper (co-founder/CEO of Every): <a target="_blank" href="https://www.lennysnewsletter.com/p/inside-every-dan-shipper">https://www.lennysnewsletter.com/p/inside-every-dan-shipper</a></p><p>• Semantic Diffusion: <a target="_blank" href="https://martinfowler.com/bliki/SemanticDiffusion.html">https://martinfowler.com/bliki/SemanticDiffusion.html</a></p><p>• LMArena: <a target="_blank" href="https://lmarena.ai">https://lmarena.ai</a></p><p>• Artificial Analysis: <a target="_blank" href="https://artificialanalysis.ai/leaderboards/providers">https://artificialanalysis.ai/leaderboards/providers</a></p><p>• Why humans are AI’s biggest bottleneck (and what’s coming in 2026) | Alexander Embiricos (OpenAI Codex Product Lead): <a target="_blank" href="https://www.lennysnewsletter.com/p/why-humans-are-ais-biggest-bottleneck">https://www.lennysnewsletter.com/p/why-humans-are-ais-biggest-bottleneck</a></p><p>• Airline held liable for its chatbot giving passenger bad advice—what this means for travellers: <a target="_blank" href="https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know">https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know</a></p><p>• Demis Hassabis on LinkedIn: <a target="_blank" href="https://www.linkedin.com/in/demishassabis">https://www.linkedin.com/in/demishassabis</a></p><p>• We replaced our sales team with 20 AI agents—here’s what happened | Jason Lemkin (SaaStr): <a target="_blank" href="https://www.lennysnewsletter.com/p/we-replaced-our-sales-team-with-20-ai-agents">https://www.lennysnewsletter.com/p/we-replaced-our-sales-team-with-20-ai-agents</a></p><p>• Socrates’s quote: <a target="_blank" href="https://en.wikipedia.org/wiki/The_unexamined_life_is_not_worth_living">https://en.wikipedia.org/wiki/The_unexamined_life_is_not_worth_living</a></p><p>• Noah Smith’s newsletter: <a target="_blank" href="https://www.noahpinion.blog">https://www.noahpinion.blog</a></p><p>• <em>Silicon Valley </em>on HBO Max: <a target="_blank" href="https://www.hbomax.com/shows/silicon-valley/b4583939-e39f-4b5c-822d-5b6cc186172d">https://www.hbomax.com/shows/silicon-valley/b4583939-e39f-4b5c-822d-5b6cc186172d</a></p><p>• Clair Obscur: Expedition 33: <a target="_blank" href="https://store.steampowered.com/app/1903340/Clair_Obscur_Expedition_33/">https://store.steampowered.com/app/1903340/Clair_Obscur_Expedition_33/</a></p><p>• Wisprflow: <a target="_blank" href="https://wisprflow.ai">https://wisprflow.ai</a></p><p>• Raycast: <a target="_blank" href="https://www.raycast.com">https://www.raycast.com</a></p><p>• Steve Jobs’s quote: <a target="_blank" href="https://www.goodreads.com/quotes/463176-you-can-t-connect-the-dots-looking-forward-you-can-only">https://www.goodreads.com/quotes/463176-you-can-t-connect-the-dots-looking-forward-you-can-only</a></p><p>—</p><p><strong>Recommended books:</strong></p><p>• <em> When Breath Becomes Air</em>: <a target="_blank" href="https://www.amazon.com/When-Breath-Becomes-Paul-Kalanithi/dp/081298840X">https://www.amazon.com/When-Breath-Becomes-Paul-Kalanithi/dp/081298840X</a></p><p>• <em>The Three-Body Problem</em>: <a target="_blank" href="https://www.amazon.com/Three-Body-Problem-Cixin-Liu/dp/0765382032">https://www.amazon.com/Three-Body-Problem-Cixin-Liu/dp/0765382032</a></p><p>• <em>A Fire Upon the Deep</em>: <a target="_blank" href="https://www.amazon.com/Fire-Upon-Deep-Zones-Thought/dp/0812515285">https://www.amazon.com/Fire-Upon-Deep-Zones-Thought/dp/0812515285</a></p><p>—</p><p>Production and marketing by <a target="_blank" href="https://penname.co/">https://penname.co/</a>. For inquiries about sponsoring the podcast, email <a target="_blank" href="mailto:
[email protected]">
[email protected]</a>.</p><p>—</p><p><em>Lenny may be an investor in the companies discussed.</em></p> <br/><br/>To hear more, visit <a href="https://www.lennysnewsletter.com?utm_medium=podcast&utm_campaign=show-notes-no-free-preview-language">www.lennysnewsletter.com</a>