Ship It Weekly - DevOps, SRE, Platform and Cloud Engineering News

This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Ryan Packer about why a CTO is writing code again, what happens to CI/CD when every engineer runs a fleet of agents, and who trains junior engineers when AI can fix the bugs. Ryan is the CTO at BambooHR and a former VP of Engineering at Adobe. He's watched engineering organizations change from Omniture to Adobe to BambooHR, and he says he's spent more time coding this year than in the rest of his career combined. We start with why. Ryan will never write production code for his team again, but he wants to use the same tools and learn the same techniques they're learning, and to show an idea in working code instead of a PRD or a Slack message. He's also honest about the limits: his sandbox serves maybe a hundred people, and his team's software serves millions. Then we get into the software factory. In Ryan's own version, a customer files a bug, an agent opens the PR, tests it for security and performance, scores it, and ships it if it meets the bar. At BambooHR, pull requests are up almost 80% since the start of the year, and quality has improved by the metrics they track. The part he's most excited about is more at-bats with customers: an idea to a working prototype in a day. For platform teams, Ryan says AI has been more of a challenge than a help for CI/CD. BambooHR was already releasing hundreds of times a day, but agents do their best work when they can run and test what they build. Ten, twenty or a hundred agents per engineer means environments that spin up and down fast, with real data. We also cover protecting HR and payroll data: terms with the frontier labs, data retention, and an AI council. On careers, Ryan thinks mid-career engineers face the hardest change, and that the ones who keep learning have little to worry about. On junior engineers, he doesn't claim a perfect answer. He no longer needs them to fix bugs, so he suggests making the LLM explain its choices and using an adversarial agent that lists five things it would have done differently. We finish with every engineer managing a team of agents, small pods doing what 10 to 15 people used to, and why he expects the same or more engineers a year from now. The big takeaway: if every project in your portfolio has better than a 50% chance of success, you're not being ambitious enough. Put something in there with a 10% chance. Highlights • Why a CTO is coding again, and why his sandbox isn't production • The software factory: a customer bug to an agent PR to a tested release • Pull requests up almost 80%, with quality improving • More at-bats with customers: idea to working prototype in a day • Why AI is straining CI/CD and test environments • Protecting HR and payroll data while adopting AI • Junior engineers, adversarial agents and making the LLM explain itself • Every engineer with a team of agents, and the 10% project Links Ryan on LinkedIn · BambooHR · BambooHR careers · More episodes · On Call Brief · LMGT Awards

This week on Ship It Weekly, GitHub will block pull_request_target in public repos by default from December 2, with brownouts on Mondays, November 9 and 16, 11 AM to 1 PM ET. A WordPress core update race (CVE-2026-119977) is on CISA's exploited list: during a full-package update, a fresh installer can sit for a moment at a public URL under wp-content/upgrade. A hijacked DebugBear script drained about $2.7 million from Polymarket users after DebugBear's DNS was compromised through its registrar. And Alpine Linux explains why upgrading GitLab didn't fix CVE-2026-97229 on its own image: the fix was a patch to Ruby. The thread: what actually ran, as opposed to what you think ran. A trigger that runs a fork's code with your secrets, an update that opens its own window, a script served by whoever controls the DNS, and a version number that said "fixed". In the lightning round: OpenSSH 10.6p2 fixes a password login bypass on musl builds with PAM off, which is how Alpine builds openssh-server; Debian and Ubuntu patch an APT package swap (CVE-2026-109518); and .NET 8 and 9 reach end of support on Tuesday, November 10. The human closer comes back to Alpine's "we cannot rule it out." If an attack hit you this week, could your logs prove it didn't work? Highlights • pull_request_target: what it runs, and what to check before Monday • WordPress: when keeping current is what opens the window • DNS, HTTPS and subresource integrity for third-party scripts • Registrar lock, MFA and alerts on DNS changes • Alpine: GitLab from your own image needs a fixed Ruby too • "No evidence" is only as good as the logs you kept Links pull_request_target blocked by default (GitHub): https://tsn.io/KwebM WordPress advisory GHSA-4w3m-c66w-69mg: https://tsn.io/1HGiK WordPress upgrade-window RCE (Wordfence): https://tsn.io/Pc6X2 Polymarket postmortem: https://tsn.io/rOc2M DebugBear on the registrar DNS compromise: https://tsn.io/4BjIg Polymarket losses and refunds (Hackread): https://tsn.io/qNhoe Alpine's GitLab security incident: https://tsn.io/GpNoR OpenSSH 10.6p2 release notes: https://tsn.io/dQHbG Alpine openssh-server advisory: https://tsn.io/lExh2 APT CVE-2026-109518 (Debian DSA-6607-1): https://tsn.io/LSjTb .NET 8 and 9 end of support: https://tsn.io/7NC5R This week's On Call Brief: https://tsn.io/XJLrm More episodes and show notes: https://tsn.io/NqkdP On Call Brief: https://tsn.io/Gpz2d LMGT Awards: https://tsn.io/lAERA

This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Davlet Dzhakishev about what it takes to let AI agents work on your cloud infrastructure, why guardrails belong in the architecture and not the prompt, and what running your own agents really costs. Davlet is the CEO and co-founder of Cloudgeni, which builds AI agents for cloud infrastructure. He got into this after working through SOC 2 and ISO 27001 compliance prep mostly by hand, and asking what agents would need to fix those problems reliably without making things worse. We start with the incident that changed how Cloudgeni builds. An agent hit a snag on a simple Terraform change and decided the fix was to run apply. On staging. Resources got recreated, and Davlet calls it his wake-up call. Since then, guardrails come first: sandboxed runtimes, scoped credentials, read-only cloud access, and changes that go through Git, CI/CD and tests. At the architecture level, the agent shouldn't be able to do what you don't want it to do. Then we get into agentic PR reviews, which check human and agent work the same way, and why a good reviewer catches more than the obvious errors. Davlet's example is a private network around a key vault: great for security, until your own apps lose access. We also cover drift, and why you need to reconcile it before you can trust agent PRs. On cost, Davlet's view is that AI subscriptions are heavily subsidized, and running your own agents on API tokens can cost ten to twenty times more. Most agent usage also has no audit trail, so a stuck loop can poll a dead API server every day and no one will know. He thinks FinOps for AI is going to be big, orphaned agents and all. That need for audits is part of why Cloudgeni open-sourced its foundation as OpenGeni. We finish with what he still wouldn't hand to an agent, and where a platform team should start without building a monster: PR reviews first, then agents writing IaC from your templates, then automation. His starter kit is a real sandbox, ideally in the cloud, because Docker doesn't count. Give it read-only access to an AWS account or Azure subscription, one IaC repo, and permission to open PRs. Ask it to delete a resource, and the worst it can do is open a PR. The big takeaway: a safe agent is mostly a permissions problem, not a prompting problem. Give it read-only access and PR-only changes, and it can't make that mistake however you word the request. Highlights • The Terraform apply on staging that became his wake-up call • Guardrails at the architecture level, not in the prompt • Read-only access, with every change through Git, CI/CD and tests • Agentic PR reviews that catch operational impact • Why drift has to be reconciled before you can trust agent PRs • Why API tokens can cost 10 to 20 times more • FinOps for AI, stuck loops and orphaned agents • A starter kit: a real sandbox, read-only cloud access, one IaC repo, PRs only Links Davlet on LinkedIn · Davlet on X · Cloudgeni · OpenGeni · Agents guide · More episodes · On Call Brief · LMGT Awards

This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Devansh about why the trickiest legal AI hallucination is a real case used the wrong way, how his team splits their system into three tiers to find failures fast, and why he'd rather optimize for maintenance than for generation. Devansh is the Founder and Head of AI at Irys. He builds reasoning and verification systems for legal AI, a tough place for hallucinations and confident wrong answers. He also publishes open source AI research on Substack. We start with why AI for professionals breaks down. Devansh thinks models are getting good at doing a lot of things somewhat well, but they fall short on the last mile to reliable. So his team engineers structure back in instead of hoping a better model fixes it. In legal, fake citations are the failure everyone has heard about. The more interesting one is a real case that doesn't apply. So much of law is interpretation, and a model built to predict the most probable token pulls toward the average. Then he walks through the three tiers behind Irys One. Ingestion maps messy input, like PDFs with handwritten notes and email chains that refer to them, into a knowledge graph so the AI knows where to look. Reasoning navigates that graph. Delivery applies how a firm or a lawyer likes to work. The payoff is error isolation: if a lawyer says something is missing, two or three targeted queries tell them whether it was ingestion or reasoning. We also get into why chunk-based retrieval misses connections, why going from 10 docs to 10 gigabytes is mostly about finding the few documents that matter, and how "answer contracts" handle "it depends." Then I take a DevOps angle. Devansh says the hardest part of running AI in production is tracking everything. A fully agentic system might perform 20% better, but you never know how it breaks until it breaks. The first thing his team tracks is whether the system behaves as expected, because labs change models behind the scenes with post-training, quantization and tokenizer changes. We also cover why he calls reinforcement learning the most overhyped technique in AI: it works when the output is truly verifiable, and falls apart when "better" has no real signal. The big takeaway: you can't fix what you can't find. Engineer for error isolation first, and optimize for maintenance, not generation. Coding agents will churn out heaps of code, but maintenance is where a system lives or dies. Highlights • Why a real case applied wrong is worse than a fake citation • Why AI's pull toward the average struggles with legal judgment • Ingestion, reasoning and delivery as three separate tiers • Knowledge graphs vs chunking for finding what matters • Isolating failures with two or three targeted queries • Models that change under you, and tracking expected behavior • Why reinforcement learning is overhyped • Optimize for maintenance, not generation Links Devansh on LinkedIn: https://tsn.io/7Hjfm Irys: https://tsn.io/ytG1A Artificial Intelligence Made Simple (Devansh's Substack): https://tsn.io/ZM6eS More episodes and show notes: https://tsn.io/NqkdP On Call Brief: https://tsn.io/Gpz2d LMGT Awards: https://tsn.io/lAERA

This week on Ship It Weekly, GitHub Actions can now run eligible Linux jobs on AWS as well as Azure, so allowlists and Entra Conditional Access policies that only trust Microsoft ranges can break. The Brotli decoder bug (CVE-2026-111497) landed on CISA's exploited list, and it reaches past Node.js to Python Brotli, Bun, Envoy, Istio, a Go port and .NET. A campaign called Gheist is using stolen GitHub tokens to push ci-sync branches that print Actions secrets past log masking, then delete the run. And attackers are dropping AGENTS.md files into Downloads folders to hijack AI coding agents. The thread: each one is an inventory question. Which services decompress Brotli, which allowlists assume CI runs on Azure, which OIDC trust policies accept any branch, and which laptops have started an agent in Downloads? In the lightning round: GitHub Actions workflow lockfiles are generally available, Railway disclosed a breach of its internal systems, Tailscale 1.104.4 fixes four security bulletins, and Datadog moved its US1 status page to its own Status Pages product. The human closer, from Bill Duncan: "Twenty minutes in, you haven't started diagnosing anything. You've been doing archaeology." Write down who owns it before the pager asks. Highlights • GitHub Actions on AWS: why the same job can pass, then fail • Sync runner IP ranges from GitHub's meta API, not one cloud's • Brotli: a decoder bug your HTTP stack may run for you • Gheist: what private repos and OIDC trust let through • AGENTS.md in Downloads: an instruction file is code now • Service catalogs and the "what depends on me?" question Links GitHub Actions multi-cloud compute: https://tsn.io/fJkLJ Node.js October security releases (Brotli): https://tsn.io/BvZBU Brotli CVE-2026-111497, affected libraries (Wiz): https://tsn.io/HJuKS .NET out-of-band security updates: https://tsn.io/j0Xwf Gheist steals GitHub Actions secrets (Wiz): https://tsn.io/nunEs AI agents hijacked via the Downloads folder (Forever Security): https://tsn.io/qnwuc Actions workflow lockfiles GA: https://tsn.io/M84uA Railway security disclosure: https://tsn.io/21t0j Tailscale security bulletins: https://tsn.io/GvZxR Who Owns This? (Bill Duncan): https://tsn.io/LFrzc SRE Weekly #540: https://tsn.io/O8tpO This week's On Call Brief: https://tsn.io/HqfCP More episodes and show notes: https://tsn.io/NqkdP On Call Brief: https://tsn.io/Gpz2d LMGT Awards: https://tsn.io/lAERA