Использование robots.txt
Was this information helpful?