Google spoke about its year-end report on the crawling challenges Google faced in 2025 when crawling and indexing the web for Google Search. The biggest challenges include faceted navigation and action parameters, which make up about 75% of those challenges, Gary Illyes from Google said. This was on its latest Search Off the Record podcast, published this morning.

**What is the issue. **Crawling issues can cause your site to lag and slow, it can overload your server, and make your website unusable and inaccessible. If a robot goes off on your site and gets into some infinite loop of crawling, it can take some time for the site to recover.

“Once it discovers a set of URLs, it cannot make a decision about whether that URL space is good or not unless it crawled a large chunk of that URL space,” Gary said. By then it is too late and your site has slowed to a halt.

**The biggest crawling challenges. **Here are the biggest crawling challenges based on the report:

• 50% are related to faceted navigation. This is a common issue for e-commerce sites where you can have infinite ways to filter products by size, colors, price, and so on.
• 25% are related to action parameters. This is when URL parameters that trigger a specific action rather than changing the page content significantly.
• 10% are related to irrelevant parameters. These are things like session IDs, UTMs or other parameters in the URL.
• 5% are related to plugins or widgets. Some plugins or widgets can cause URL issues and confuse crawlers.
• 2% are related to other “**weird stuff.” **This is like a catch all of URL issues including issues like double-encoding URLs.

Why we care. Ensuring you have a good URL structure, without many of these URL traps for bots is essential for (1) making sure your server is healthy, (2) your pages load quickly and (3) search engines don’t get confused on what are your canonical URLs.

**The recording. **Here is an embed of the podcast recording:

Search Engine Land is owned by Semrush. We remain committed to providing high-quality coverage of marketing topics. Unless otherwise noted, this page’s content was written by either an employee or a paid contractor of Semrush Inc.

About the Author

Barry Schwartz is a technologist and a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics.

In 2019, Barry was awarded the Outstanding Community Services Award from Search Engine Land, in 2018 he was awarded the US Search Awards the "US Search Personality Of The Year," you can learn more over here and in 2023 he was listed as a top 50 most influential PPCer by Marketing O’Clock.

Barry can be followed on X here and you can learn more about Barry Schwartz over here or on his personal site.