In our second Virtual Keynote with Google Webmaster Trends Analyst Gary Illyes, Eric Enge asked him about how Google treats various SEO tags. In this post I will summarize what Gary had to say about the noindex tag.
You can watch the segment where this discussion occurs in this video:
What is a noindex tag?
According to Eric Enge, “A NoIndex tag is an instruction to the search engines that you don’t want a page to be kept within their search results. You should use this when you believe you have a page that search engines might consider to be of poor quality.”
What does a noindex tag do?
- It is a directive, not a suggestion. I.e., Google will obey it, and not index the page.
- The page can still be crawled by Google.
- The page can still accumulate PageRank.
- The page can still pass PageRank via any links on the page.
[Tweet “Noindex pages still collect and pass PageRank (Gary Illyes). See”]
(Gary noted that although Eric mentioned PageRank, in reality there are a lot of other signals that are potentially passed through any link. Better to say “signals passed” than “PageRank passed.”)
Does crawl frequency of a noindex page decline over time?
Crawl frequency refers to how often Google returns to a page to check whether the page still exists, has any changes, and has accumulated or lost signals.
Typically crawl frequency will decline for any page that Google cannot index, for whatever reason. Google will try to recrawl a few times to check if the noindex, error, or whatever was blocking the crawl, is gone or fixed.
If the noindex instruction remains, Google will slowly start to lengthen the time to the next attempt to crawl the page, eventually reducing to a check about every two-to-three months to see if the no index tag is still there.
Eric noted that this means that the noindex tag is a way of controlling how Google crawls your site, and Gary agreed.
[Tweet “Noindex tag gradually decreases Google crawl frequency for the page (Gary Illyes). See”]
Learn how to implement a noindex tag on your site.
Find out what Perficient Digital can do for your company’s SEO.
Good refresher! I still see a lot of SEOs use the robots.txt file instead of a meta robots noindex tag and are confused why the page is still in Google’s index.
However, I do wish Google would support the tag, where we could disallow part of a page from being indexed. I work in an industry where we have to have boilerplate content on each page and while Matt Cutts has previously said this isn’t a bad thing, it’s hard for me to believe it wouldn’t help if Google didn’t index that part.
That’s indeed an intriguing and probably useful idea, Tyler, for many applications.