MyBB Community Forums

Full Version: Crawler Settings
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
how to properly set google crawler bec. my page is not visible to guest?

i tried the setting below but it wont log in.

Restricted Directory or URL = http://pinoybodybuilding.com/forum/

Authentication URL= http://pinoybodybuilding.com/forum/membe...tion=login

Authentication method: POst

Parameters=
username=createdbotusername
password=botpassword


thanks please provide a sample...

Created a new user group for spiders / bots. Give that group permission to read threads in forums you want indexed. Now change your settings so that Google and other spiders are assigned to that group.
ok ive created a new group called bots and added the permission to view all forums ..

now how do i add google and other bots to this new group ? thanks
In the ACP click on the "Configuration" tab. On the left, there will be a "Spiders / Bots" tab.

The Spiders / Bots tab will list all the bots your installation of MyBB knows about. Each bot has an "Edit" control. The last field of the bot edit form is "User Group". When MyBB detects this bot (by the user agent string) it will grant that user group's permissions to the bot. This way you can even assign bots to a banned group.