Search Off the Record

In this episode of Search Off the Record, Gary and Martin dig into what "page size" and "page weight" actually mean for developers, users, and search engines. They discuss exploding web page sizes: median mobile homepages hit 2.3 MB in 2025 Web Almanac (up 3x from 2015), key insights for developers on page weight definitions, Googlebot's crawl limits, HTML bloat from structured data/images, and why size still hurts UX on slow connections despite faster networks. If you build or maintain websites, this conversation will help you rethink how much data your pages ship, where bloat really comes from, and why page weight still matters even as connections get faster. Resources: ​Web Almanac → https://almanac.httparchive.org/en/2025/ HTML living standard → https://html.spec.whatwg.org/multipage/ How page speed helps with conversions → https://www.thinkwithgoogle.com/marketing-strategies/app-and-mobile/mobile-page-speed-data/ Episode transcript → https://goo.gle/sotr106-transcript Listen to more Search Off the Record → https://goo.gle/sotr-yt Subscribe to Google Search Channel → https://goo.gle/SearchCentral Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team. #SOTRpodcast #SEO #GoogleSearch Speakers: Martin Splitt, Gary Illyes

Developers often talk about Googlebot as if it were a single program you could just run as "googlebot.exe", but that is not how Google's crawling actually works. In this episode of Search Off the Record, Martin and Gary from the Search Relations team unpack how Google's crawling infrastructure is really built and operated.​ They cover why "Googlebot" is a misnomer and how it relates to a central crawling software-as-a-service used by many Google products​, how crawl behavior is controlled centrally to avoid overwhelming sites (throttling, handling 503s, and "don't break the internet" safeguards)​ and more! If you build for the web, work on SEO, or just want a more accurate mental model of how Google crawls pages, this behind‑the‑scenes discussion is for you. Resources: ​Crawlers → https://developer.google.com/crawling Episode transcript → https://goo.gle/sotr107-transcript Listen to more Search Off the Record → https://goo.gle/sotr-yt Subscribe to Google Search Channel → https://goo.gle/SearchCentral Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team. #SOTRpodcast #SEO #GoogleSearch Speakers: Martin Splitt, Gary Illyes