Google says that status codes are the first port of call for content crawling as the number provides a quick and clear indication of what is present on a web page.
In a Google Webmaster Central Hangout last week, Google’s John Mueller said that status codes are the very first thing to be checked during the crawling and indexing process.
More specifically, the search giant looks for a ‘200’ status code, which acts as confirmation that a valid page is being crawled and that the content within could be of worth to users.
In contrast, if a ‘400’ or ‘500’ code is thrown up, this will act as a red flag and Google will not attempt to render content for indexing.
Responding to a question about Google’s need to check codes first, Mueller noted that if it is a status code 200, then that is a sign that there is something they might be able to index.
He added that if it is a 400 or 500 error, or a redirect, then those are things that they wouldn’t render.
Mueller said that webmasters should be aware that 404 pages go under Google’s radar, which can be a problem for those attempting to rank content in SERPs.
Even well-designed and optimised 404 pages will be out of reach for Google, so only human visitors will be able to see the content and not the search engine.
Mueller confirmed that Google is always on the lookout for a 200 status and that it won’t render any content unless this specific code is returned.