Shipping Agents, Not Vulnerabilities with Ian Webster, PromptFoo CEO
FEB 16, 202645 MIN
Shipping Agents, Not Vulnerabilities with Ian Webster, PromptFoo CEO
FEB 16, 202645 MIN
Description
<p>As LLM apps evolve from simple chatbots to tool-using agents, the attack surface explodes, and the old security playbooks don’t hold. In this episode of <strong>Alexa’s Input (AI)</strong>, Alexa Griffith sits down with <strong>Ian Webster</strong>, co-founder and CEO of <strong>PromptFoo</strong>, to break down what AI security actually looks like in practice: automated red teaming, prompt injection and jailbreak testing, evaluation workflows that scale, and why “guardrails alone” is not a security strategy.</p><p>Ian shares how PromptFoo grew from a side project into a widely adopted open-source standard, what it means to raise multi-millions in a fast-moving market, and how enterprises are approaching the full vulnerability lifecycle, from finding issues to triage, remediation, and validation. Ian also discusses the “lethal trifecta” that makes agents fundamentally risky (untrusted input + sensitive data + exfil path), and why MCP security isn’t just about users and tools, it’s about dangerous tool combinations and rogue servers.</p><p><strong>Podcast Links</strong></p><p>Watch: <a href="https://www.youtube.com/@alexa_griffith">https://www.youtube.com/@alexa_griffith</a></p><p>Read: <a href="https://alexasinput.substack.com/">https://alexasinput.substack.com/</a></p><p>Listen:<a href="https://creators.spotify.com/pod/profile/alexagriffith/"> https://creators.spotify.com/pod/profile/alexagriffith/</a></p><p><br></p><p>More: <a href="https://linktr.ee/alexagriffith">https://linktr.ee/alexagriffith</a></p><p><br></p><p>Website: <a href="https://alexagriffith.com/">https://alexagriffith.com/</a></p><p>LinkedIn: <a href="https://www.linkedin.com/in/alexa-griffith/">https://www.linkedin.com/in/alexa-griffith/</a></p><p><br></p><p><strong>Find out more about the guest at:</strong></p><p>PromptFoo Website: <a href="https://www.promptfoo.dev/">https://www.promptfoo.dev/</a></p><p>Github: <a href="https://github.com/promptfoo/promptfoo" target="_blank" rel="noopener noreferer">https://github.com/promptfoo/promptfoo</a></p><p>Ian’s LinkedIn: <a href="https://www.linkedin.com/in/ianww/">https://www.linkedin.com/in/ianww/</a></p><p><br></p><p><strong>Chapters</strong></p><p>00:00 Introduction to AI Security Challenges</p><p>02:06 Funding and Growth of PromptFu</p><p>06:16 The Genesis of PromptFu</p><p>11:05 Career Journey and Lessons Learned</p><p>12:53 Understanding AI Red Teaming</p><p>17:36 Recent AI Security Vulnerabilities</p><p>19:46 The Dual Nature of AI in Security</p><p>21:47 Understanding the Lethal Trifecta in AI Security</p><p>24:22 Exploring Model Context Protocol (MCP) and Its Security Implications</p><p>26:22 Common Security Issues in MCP Systems</p><p>28:17 The Role of Identity and Permissions in AI Security</p><p>30:00 Practical Implications of Using PromptFoo for Developers</p><p>31:33 Evaluating Language Models: Challenges and Techniques</p><p>36:34 The Limitations of Guardrails in AI Security</p><p>38:25 Best Practices for Engineers in AI Development</p><p>39:58 Future Trends in AI and Security</p><p>42:28 Everyday Applications of AI and Language Models</p>