MyBB Community Forums

Hello , With respect to my earlier thread Here http://community.mybboard.net/thread-63646-page-2.html , I have made a Copy of Robots.txt for my mybb forums at Desktophelp.in/forums (currently site is down for maintanance)

here is my setup
www.desktophelp.in [portal]
www.desktophelp.in/forums [mybb forums]
www.desktophelp.in/gallery [coppermine gallery integrated]

I want to put robots.txt to stop googlebot from indexing calendar and unnessary pages

you can see for example there are 2000 pages of calendar indexed for my site here the link


here i made a copy of robots for mybb , will we uploaded to root folder(ofcourse) , please anaylse this tell me what changes to make

Quote:User-Agent: *
Disallow: /forums/captcha.php
Disallow: /forums/editpost.php
Disallow: /forums/misc.php
Disallow: /forums/modcp.php
Disallow: /forums/moderation.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/ratethread.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/sendthread.php
Disallow: /forums/task.php
Disallow: /forums/usercp.php
Disallow: /forums/usercp2.php
Disallow: /forums/calendar.php
Disallow: /forums/*action=emailuser*
Disallow: /forums/*action=nextnewest*
Disallow: /forums/*action=nextoldest*
Disallow: /forums/*year=*
Disallow: /forums/*action=weekview*
Disallow: /forums/*action=nextnewest*
Disallow: /forums/*action=nextoldest*
Disallow: /forums/*sort=*
Disallow: /forums/*order=*
Disallow: /forums/*mode=*
Disallow: /forums/*datecut=*
Allow: /

Looks good to me. Although I am not sure if you need

Allow: /

(2010-01-23, 02:43 PM)Tierney Wrote: [ -> ]Looks good to me. Although I am not sure if you need

Allow: /

that is optional , i can remove that , please check the other lines , are they proper, currently site is mybb sef url enabled

You should add the link to your sitemap at the top.

^ yah you right

For example

Quote:sitemap: http://www.yourforum.com/forum/sitemap.xml

User-Agent: *
Disallow: /forums/captcha.php
Disallow: /forums/editpost.php
Disallow: /forums/misc.php
Disallow: /forums/modcp.php
Disallow: /forums/moderation.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/ratethread.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/sendthread.php
Disallow: /forums/task.php
Disallow: /forums/usercp.php
Disallow: /forums/usercp2.php
Disallow: /forums/calendar.php
Disallow: /forums/*action=emailuser*
Disallow: /forums/*action=nextnewest*
Disallow: /forums/*action=nextoldest*
Disallow: /forums/*year=*
Disallow: /forums/*action=weekview*
Disallow: /forums/*action=nextnewest*
Disallow: /forums/*action=nextoldest*
Disallow: /forums/*sort=*
Disallow: /forums/*order=*
Disallow: /forums/*mode=*
Disallow: /forums/*datecut=*

Allow: /

(2010-01-24, 04:54 AM)RPicard Wrote: [ -> ]You should add the link to your sitemap at the top.

(2010-01-24, 05:37 AM)afzulnizam Wrote: [ -> ]^ yah you right

For example

Quote:sitemap: http://www.yourforum.com/forum/sitemap.xml

User-Agent: *
Disallow: /forums/captcha.php
Disallow: /forums/editpost.php
Disallow: /forums/misc.php
Disallow: /forums/modcp.php
Disallow: /forums/moderation.php
Disallow: /forums/newreply.php
Disallow: /forums/newthread.php
Disallow: /forums/online.php
Disallow: /forums/printthread.php
Disallow: /forums/private.php
Disallow: /forums/ratethread.php
Disallow: /forums/report.php
Disallow: /forums/reputation.php
Disallow: /forums/sendthread.php
Disallow: /forums/task.php
Disallow: /forums/usercp.php
Disallow: /forums/usercp2.php
Disallow: /forums/calendar.php
Disallow: /forums/*action=emailuser*
Disallow: /forums/*action=nextnewest*
Disallow: /forums/*action=nextoldest*
Disallow: /forums/*year=*
Disallow: /forums/*action=weekview*
Disallow: /forums/*action=nextnewest*
Disallow: /forums/*action=nextoldest*
Disallow: /forums/*sort=*
Disallow: /forums/*order=*
Disallow: /forums/*mode=*
Disallow: /forums/*datecut=*

Allow: /

but i dont have a sitemap, it sitemap feature inbuilt in mybb ? , as af as i know we have use the mod the generate sitemap , correct me if i am wrong

Is there any point to having a robots.txt

(2010-01-24, 08:43 AM)netjockey Wrote: [ -> ]but i dont have a sitemap, it sitemap feature inbuilt in mybb ? , as af as i know we have use the mod the generate sitemap , correct me if i am wrong

Its okay if you don't have sitemap. Sitemap just a file that tells search engines about pages in a web site that can be reviewed by the search engine.

So what About my robots.txt it is ok and correct , should i kinda implement it , or it need any changes waddaya say all

^ there are no wrong or error on your robot txt.