When Google Stops Indexing

In a Google SEO roundtable on February 18, 2022, John Mueller, Google’s Senior Webmaster Trends Analyst, was asked how long a site can be listed as “Discovered, not 

Indexed” This designation means that Google has seen the URL, yet has not crawled the site. His response to the question was, “That can last forever”. 

English Google SEO office-hours from February 18, 2022

Mueller continues,

“It’s something where we just don’t crawl and index all pages. And it’s completely normal for any website that we don’t have everything indexed.

And, especially with a newer website if you have a lot of content, then I would assume it’s expected that a lot of the new content for a while will be discovered and not indexed.

And then over time usually it kind of shifts over, like well it’s actually crawled, or it’s actually indexed when we see that there’s actually value in focusing more on the website itself. But it’s not guaranteed.”

This is not a great deal of information to glean a remedy from. However, through our own experience with Google Search Console, we have a solution for this issue.

When a sitemap has been partially or completely excluded from GSC, submit the sitemap with a new custom domain. This may be unsatisfying. However, rather than crossing your fingers, hoping that Google Search Console will deign to index our Landing Pages, for the time being, it is an option that provides results.

However, this is a last resort. Only do so when the following steps have already been taken:

• Test a single page to see if there are any errors that would be preventing a crawl
• Make sure the landing page keywords are not all too similar
• Resubmit sitemap
• Ping google with google.com/ping?sitemap=<path to sitemap>
• Try requesting 1 single page for indexing

We only suggest a new domain after the following actions have been taken and proven to be unsuccessful.