robots.txt: the ultimate guide • Yoast

The robots.txt file is a file you can use to tell search engines where they can and cannot go on your site. Learn how to use it to your advantage!

Source: robots.txt: the ultimate guide • Yoast

Advertisements