What are crawl errors?
Google uses so-called Googlebots to crawl and index your page. Crawling, in layman’s language, is the process of Googlebot going over your pages, one link at a time. When crawling, its goal is to get to every important page on your site by following links on pages, in sitemaps, etc. Indexing, on the other hand, is what Googlebot does to take in all the content on your pages, to include it in its search result pages.
There are two types of crawl errors:
- Site errors that affect your entire site. Think along the lines of connectivity issues with your web server, and problems fetching your robots.txt file.
- URL errors that affect a specific page on your website. Googlebot tried to crawl the URL but did not succeed somehow. It was able to resolve your DNS, connect to your server, fetch/read your robots.txt file, and then request the URL. But after that, something went wrong.
Viewing crawl errors in Yoast SEO
In our Yoast SEO plugin (free and paid), you can view the crawl errors that Google came across on your website. All you have to do is connect Google Search Console to Yoast SEO. In our plugin, we guide you through that process. Let me explain the steps here as well.
Connect Google Search Console to Yoast SEO
To connect Google Search Console to the Yoast SEO plugin, all you have to do is navigate to this page in WordPress: SEO › Search Console.
The next step is to connect them. In our plugin, just click the ‘Get Google Authorization Code’ button:
It’ll take you to Google Search Console. There, you’ll be asked to confirm that you want to connect Google Search Console to Yoast SEO and let our plugin view and manage the data for your sites. Click ‘allow’:
Lastly, you’ll get a key to include in our plugin:
Now simply copy-paste that code and insert it into the box in our plugin, hit ‘Authenticate.’
Choose the profile you’d like to connect and save it. Done! Now, you can continue in the first tab of that same section in our plugin (Desktop). Be sure to check the other tabs as well to find specific crawl errors.
Here, you will find the information we collected from your Google Search Console. In this table, you see the URL that gave an error, the date Google crawled it last, the date when Google detected the error first and the response code Google sent. In the screenshot, all response codes are 404 Not Found.
So, if you connect Google Search Console to Yoast SEO, you will have a great overview of how many crawl errors Google finds on your website. Now, you can go and create redirects for these 404s, or simply change them to 410s if that page is of absolutely no use to you anymore. More on status codes in this article. When you have ‘fixed’ the error, hover over the URL in Yoast SEO and click ‘mark as fixed’.
Is there an easy way to create that redirect?
Yes! There is an easier way to complete this process, and it is called Yoast SEO Premium. Besides a lot of extras that plugin has to offer, it allows you to immediately create your redirect in our plugin:
Simply click ‘Create redirect,’ and, unlike in our free plugin (which will prompt that it’s only featured in our premium plugin), you’ll get this screen:
Our plugin will give you the option to create a redirect, or add another status code (301, 302, 307, 410, 451 are all possible). In case of a 301 redirect, like in the example, simply insert the URL you’d like that ‘old’ URL to redirect to. If you want to tell Google Search Console about this fix, simply leave the check ‘Mark as fixed’ as is and hit ‘Create Redirect.’ It’s as simple as that. In tomorrow’s article, we’ll shine a light on the redirects manager.
Now go and connect Google Search Console to Yoast SEO!
I hope this sheds some light on why you want to connect Google Search Console to Yoast SEO. You’ll be able to monitor crawl errors in our free plugin, and for a few bucks a year, our premium plugin will even help you fix them!
If you by any chance have already used this feature in our premium plugin, I’d love for you to share your thoughts and experiences in the comments!