I really enjoyed hearing +PierreFar discuss Google Recommendations for Mobile Websites at the SearchLondon conference this week. The document he authored has become an invaluable resource on the topic, so if by some chance you haven’t studied this in depth, go do it now.
Right, you’re back. Well one of the interesting points raised in the document was Google’s approach to crawling responsive design websites, i.e. websites that serve all devices on the same set of URLs, with each URL serving the same HTML to all devices and using just CSS to change how the page is rendered on the device. For some time now responsive web design (RWD) has been their recommended approach to mobile, but it also seems to bring an advantage in crawling resource.
Why responsive design
It saves resources for both your site and Google’s crawlers. For responsive web design pages, any Googlebot user agents needs to crawl your pages once, as opposed to crawling multiple times with different user agents, to retrieve your content. This improvement in crawling efficiency can indirectly help Google index more of the site’s contents and keep it appropriately fresh.
Both Google’s normal and mobile crawlers can detect RWD from the presence of the media query. There are many ways of implimenting this, but you can often find it in the CSS or page header.
@media only screen and (max-width: 640px) {...}
As Pierre pointed out, this is a very good reason not to block your JavaScript or CSS files within the robots.txt. From my understanding, once Google has recrawled your entire website to ensure that all pages contain this query, they will then register your website as RWD. From then on, they will not need to crawl your site, “multiple times with different user agents” which will give you an indirect benefit of additional resource from their main crawler.
Of course, this reaction by Googlebot would be a good candidate for testing; looking at crawl depth/time on newly indexed RWD websites using the server logs to see if Googlebot spends additional time within the website, and other bots stop calling.
Send to Kindle
Nick Wilsdon
Latest posts by Nick Wilsdon (see all)
- Places Available For Free SEO Seminar 6th June in London - May 24, 2013
- Consumers Turn On Amazon Over UK Tax Avoidance - May 18, 2013
- Using Google Sets To Generate Blogger Lists - March 19, 2013
Warning: strlen() expects parameter 1 to be string, object given in /home/nickwils/public_html/wp-includes/formatting.php on line 1793
RWD makes sense for simple sites like blogs and portfolio web sites with relatively simple content. But what about eCommerce web sites, or those that perform other more complex functions, where one user interface just doesn’t work for all screen sizes. Surely a well written sitemap will help Google enormously? Do Google provide clearer advice on these scenarios?
@James
Absolutely, and this is why these mobile guidelines have been so useful. Google have provided step-by-step, clear recommendations for different approaches to mobile. While they recommend RWD, they have also walked you through other setups.
I assume from your comment that you would want to set up a seperate mobile-friendly website. In that case the recommendations would be to still let Google index this version but use canonical tags to indicate the relationship between the mobile and original versions of each page on your site. This can be done in the page headers or within your sitemap. They also suggest you should use the http vary tag. If you have any questions on the document, let me know and I’d be happy to help.
In terms of this advantage on crawling, you could still increase the crawling of your websites (and thereby the depth and frequency) by reducing page load size. This would have a similar effect.