For AI Agent Fleets, the Model Is Rarely the Bottleneck | Theory Ventures' Andy Triedman

Andy Triedman is a partner at Theory Ventures, and he is building his own software factory, while assessing how startups are leveraging theirs. His read after running agent fleets: the models are frankly good enough for most of the work. The vast majority of things you'd ask an agent to do are not intelligence limited. They are context limited. We have infrastructure work to do - but we can be more ambitious! So the hard part moves upstream. Deciding what the agents should build, what they need to know, and when they should stop and ask a person is now most of the job. We cover: Moving from solving one problem at a time to building systems that solve whole classes of problems, like generating and testing integrations Which tasks still need human approval, and which can be fully handed off Why a short feature spec leaves too much unsaid, and how better questions up front cut review interruptions Keeping shared context and company decisions current, because an agent takes everything it reads as true Inference ROI, and when well-scoped tasks can go to cheaper models Sandboxes and simulated customer environments for agents First steps for teams: shared skills, cloud agents and approval gates Protecting customer data, and organizing agent memory from session traces to company priorities New episodes, the ideas behind them and Conor's essays land in the Chain of Thought newsletter first. Subscribe: https://newsletter.chainofthought.show/ Chapters: (0:00) Agents are context limited, not intelligence limited (0:34) From coding agents to software factories (5:53) What still needs human approval (10:16) Agent loops that improve the product on their own (14:52) Where agent automation breaks down (17:59) The bitter lesson and engineering as experimentation (20:08) From writing code to directing agent fleets (24:18) Shared context and stale decisions (27:25) Inference ROI and cheaper models (30:23) Moving human time from code to customer intent (34:42) Sandboxes and simulated customer environments (37:26) First steps: shared skills and cloud agents (40:14) Protecting customer data and isolating agents (42:32) Context limits and hierarchical agent memory (47:05) What Theory Ventures looks for in founders Connect with Andy Triedman: LinkedIn: https://www.linkedin.com/in/andy-triedman/ Andy at Theory Ventures: https://theoryvc.com/team-members/andy-triedman Theory Ventures: https://theoryvc.com/ Connect with Chain of Thought host Conor Bronsdon: Newsletter: https://newsletter.chainofthought.show/ Twitter/X: https://x.com/ConorBronsdon LinkedIn: https://www.linkedin.com/in/conorbronsdon/ YouTube: https://www.youtube.com/@ConorBronsdon More episodes: https://chainofthought.show Thanks to Svix, presenting sponsor of season four of Chain of Thought. Svix delivers billions of reliable webhooks for startups and the Fortune 500. Get started at https://link.svix.com/cot. Qualified startups get $12,000 in credits, and YC companies get $50,000. Thanks to Walrus, presenting sponsor of season four of Chain of Thought. Walrus Memory gives AI agents portable, verifiable memory that carries context across apps, sessions, and other agents. Get started at https://walrus.xyz/cot Thanks to Inngest, presenting sponsor of season four of Chain of Thought. Agents in production run long - they call models and wait on APIs and people. But the longer agents run, the more they break. Inngest handles that with durable execution. You build your agent as steps in TypeScript, Python, or Go. When a step fails, Inngest retries it with exponential backoff, and completed steps are saved and skipped. Try it out: https://inngest.link/cot-pod Thanks to G2i for sponsoring this episode - for over a decade, they vetted and placed engineers at other companies, from startups to FAANG. Two years ago, they turned that same judgment inward, building their own bench to review RL environments, evals, and training data that models are trained on. Get access: https://fandf.co/3SFxVm6