Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Not Solved Printable version getting indexed in spite of robots.txt
#1
Not Solved
User-agent: *
Disallow: /private.php
Disallow: /printthread.php
Disallow: /search.php
Disallow: /modcp.php
Disallow: /reputation.php
Disallow: /calendar.php
Disallow: /online.php
Disallow: /contact.php
Disallow: /captcha.php
Disallow: /memberlist.php
Disallow: /task.php
Disallow: /usercp.php
Disallow: /ratethread.php
That is my robots.txt.
When I search the forums on google, first one is normal website mysite/forums and second search result is printable version of the website and it has a tree like structure with other threads that are also printthread.php. However when I search using the thread title then normal thread comes up in the search.
I created the site map using this site freesitemapgenerator.com I do see printthread.php links in the sitemap.
Is robots.txt getting ignored?
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)