As we know, Google has often declared that its crawlers, Googlebot, etc, come from Mountain View, California. But the truth is, that is where they self-declare the bots are coming from. Google has data centers all over the world, and even though Google may be declaring the crawler is coming from a specific location, it might be coming from a different location.
I asked John Mueller of Google, after he wrote, “I’m pretty sure the physical location [of Googlebot] varies. What kind of issues are you seeing?” I asked as a follow up, “wasn’t it mostly from California but some bots may test or come from other locations but majority CA?”
Gary Illyes from Google responded on Bluesky, “IP location is declared (this is just a text bit you can set for the IPs you manage) to be US, Mountain View, in most cases, but the actual egress points vary.” The example he gave was, “if the IP is assigned to a cluster in ATL, the egress point will be ATL.”
Gary added:
IP addresses don’t have a “geographic location”. They typically default to whatever the registrant’s location is, or they’re defaulted to some location based on BGP anycast routes. Think about addresses like 1.1.1.1 or 8.8.8.8. Where do you put these on a map? They’re technically everywhere…
So while Google crawlers might say they are coming from Mountain View, California, they may actually be coming from a different location.
We know blocking US traffic would cause big problems from Google crawling. We also know, Google can crawl locally from different regions if necessary with locally aware crawlers. In 2011, we saw Google crawling from China but that might have been a regional legal requirement.
Here are the Bluesky threads:
Forum discussion at Bluesky.
