MyBB Community Forums
robots.txt for MyBB - Printable Version

+- MyBB Community Forums (https://community.mybb.com)
+-- Forum: 1.8 Support (https://community.mybb.com/forum-175.html)
+--- Forum: General Support (https://community.mybb.com/forum-176.html)
+--- Thread: robots.txt for MyBB (/thread-111191.html)

Pages: 1 2 3 4


robots.txt for MyBB - WbDev - 2012-01-07

Hello!

What is the best robots.txt for MyBB?


RE: robots.txt for MyBB - Jason L. - 2012-01-07

Posting in other sections to void your support denial isn't allowed.


RE: robots.txt for MyBB - WbDev - 2012-01-07

I didn't ask for support.. I just ask what is the best robots.txt for MyBB.


RE: robots.txt for MyBB - weBex - 2012-01-07

That sounds like a support question though.


RE: robots.txt for MyBB - WbDev - 2012-01-07

(2012-01-07, 04:37 AM)weBex Wrote: That sounds like a support question though.

Question removed. Thanks for the support.


RE: robots.txt for MyBB - NewEraCracker - 2012-01-16

Hello,

MyBB doesn't come with any robots.txt file in the package but after seeing some reports over my google webmaster tools account I've created a robots.txt file with:
User-agent: *
Disallow: /private.php
Disallow: /member.php?action=emailuser
Disallow: /search.php



RE: robots.txt for MyBB - t0p3a - 2012-01-16

Supreme Robots.txt file that i use after intensively tests done on my forum this will stop the most unwanted content and duplicated content from appearing in google


User-agent: *
Disallow: /captcha.php
Disallow: /editpost.php
Disallow: /modcp.php
Disallow: /moderation.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /ratethread.php
Disallow: /report.php
Disallow: /sendthread.php
Disallow: /task.php
Disallow: /usercp.php
Disallow: /usercp2.php
Disallow: /archive
Disallow: /online.php
Disallow: /calendar.php
Disallow: /reputation.php
Disallow: /search.php
Disallow: /*nextoldest*
Disallow: /*nextnewest*
Disallow: /*datecut*
Disallow: /*lastpost*
Disallow: /*markread*

if you wanna see what useless things google indexes from your site go to google.com type -> site:yoursite.com


RE: robots.txt for MyBB - WbDev - 2012-01-17

This is my robots.txt:
User-Agent: *
Disallow: /moderation.php
Disallow: /ratethread.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /sendthread.php
Disallow: /usercp.php
Disallow: /usercp2.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /editpost.php
Disallow: /private.php
Disallow: /search.php
Disallow: /memberlist.php
Disallow: /showteam.php


User-agent: dotbot 
Disallow: /

User-agent: 008
Disallow: /
Is it good?


RE: robots.txt for MyBB - Amplifier - 2012-04-09

this is my site map it comes with google seo pack http://www.mydesiadda.com/robots.txt


RE: robots.txt for MyBB - armin3000 - 2012-05-02

I noticed some errors in Webmaster Panel in this kind of URLs:
j<strong></strong>tellFriend(3009,'')
j<strong></strong>printNews(2855)

How can I block them from crawlers?