2012-04-21, 08:34 PM
hi how to exclude robots to indexing of archives it is something like this in the robots.txt
example:
User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /~Milos/
example:
User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /~Milos/