When guardrails break prod: GitHub “Too Many Requests” from legacy defenses, Kubernetes nodes/proxy GET RCE, HCP Vault resilience in an AWS regional outage, and PCI DSS scope creep

FEB 13, 202615 MIN
Ship It Weekly - DevOps, SRE, and Platform Engineering News

When guardrails break prod: GitHub “Too Many Requests” from legacy defenses, Kubernetes nodes/proxy GET RCE, HCP Vault resilience in an AWS regional outage, and PCI DSS scope creep

FEB 13, 202615 MIN

Description

<p>This week on <strong>Ship It Weekly</strong>, Brian hits four stories where the guardrails become the incident.</p><p>GitHub had “Too Many Requests” caused by legacy abuse protections that outlived their moment. Takeaway: controls need owners, visibility, and a retirement plan.</p><p>Kubernetes has a nasty edge case where nodes/proxy GET can turn into command execution via WebSocket behavior. If you’ve ever handed out “telemetry” RBAC broadly, go audit it.</p><p>HashiCorp shared how HCP Vault handled a real AWS regional disruption: control plane wobbled, Dedicated data planes kept serving. Control plane vs data plane separation paying off.</p><p>AWS expanded its PCI DSS compliance package with more services and the Asia Pacific (Taipei) region. Scope changes don’t break prod today, but they turn into evidence churn later if you don’t standardize proof.</p><p>Human story: “reasonable assurance” turning into busywork.</p><p><strong>Links</strong></p><p>GitHub: When protections outlive their purpose (legacy defenses + lifecycle)</p><p><a target="_blank" rel="noopener noreferrer nofollow" href="https://github.blog/engineering/infrastructure/when-protections-outlive-their-purpose-a-lesson-on-managing-defense-systems-at-scale/">https://github.blog/engineering/infrastructure/when-protections-outlive-their-purpose-a-lesson-on-managing-defense-systems-at-scale/</a></p><p>Kubernetes nodes/proxy GET → RCE (analysis)</p><p><a target="_blank" rel="noopener noreferrer nofollow" href="https://grahamhelton.com/blog/nodes-proxy-rce">https://grahamhelton.com/blog/nodes-proxy-rce</a></p><p>OpenFaaS guidance / mitigation notes</p><p><a target="_blank" rel="noopener noreferrer nofollow" href="https://www.openfaas.com/blog/kubernetes-node-proxy-rce/">https://www.openfaas.com/blog/kubernetes-node-proxy-rce/</a></p><p>HCP Vault resilience during real AWS regional outages</p><p><a target="_blank" rel="noopener noreferrer nofollow" href="https://www.hashicorp.com/blog/how-resilient-is-hcp-vault-during-real-aws-regional-outages">https://www.hashicorp.com/blog/how-resilient-is-hcp-vault-during-real-aws-regional-outages</a></p><p>AWS: Fall 2025 PCI DSS compliance package update</p><p><a target="_blank" rel="noopener noreferrer nofollow" href="https://aws.amazon.com/blogs/security/fall-2025-pci-dss-compliance-package-available-now/">https://aws.amazon.com/blogs/security/fall-2025-pci-dss-compliance-package-available-now/</a></p><p>GitHub Actions: self-hosted runner minimum version enforcement extended</p><p><a target="_blank" rel="noopener noreferrer nofollow" href="https://github.blog/changelog/2026-02-05-github-actions-self-hosted-runner-minimum-version-enforcement-extended/">https://github.blog/changelog/2026-02-05-github-actions-self-hosted-runner-minimum-version-enforcement-extended/</a></p><p>Headlamp in 2025: Project Highlights (SIG UI)</p><p><a target="_blank" rel="noopener noreferrer nofollow" href="https://kubernetes.io/blog/2026/01/22/headlamp-in-2025-project-highlights/">https://kubernetes.io/blog/2026/01/22/headlamp-in-2025-project-highlights/</a></p><p>AWS Network Firewall Active Threat Defense (MadPot)</p><p><a target="_blank" rel="noopener noreferrer nofollow" href="https://aws.amazon.com/blogs/security/real-time-malware-defense-leveraging-aws-network-firewall-active-threat-defense/">https://aws.amazon.com/blogs/security/real-time-malware-defense-leveraging-aws-network-firewall-active-threat-defense/</a></p><p>Reasonable assurance turning into busywork (r/sre)</p><p><a target="_blank" rel="noopener noreferrer nofollow" href="https://www.reddit.com/r/sre/comments/1qvwbgf/at_what_point_does_reasonable_assurance_turn_into/">https://www.reddit.com/r/sre/comments/1qvwbgf/at_what_point_does_reasonable_assurance_turn_into/</a></p><p>More episodes + details: <a target="_blank" rel="noopener noreferrer nofollow" href="https://shipitweekly.fm">https://shipitweekly.fm</a></p>