I am trying to make google-bot not index the following urls with my robots.txt file
Example:
/forum-25.html?datecut=0&sortby=lastpost
/showthread.php?tid=195&action=nextnewest
/thread-123-nextoldest.html
/showthread.php?tid=148&action=lastpost
Disallow: /*nextoldest*
Disallow: /*nextnewest*
Disallow: /*datecut*
Disallow: /*lastpost*
Is this the correct way to do it?
perhaps you can do it like this,
Disallow: /forum-25.html?datecut=0&sortby=lastpost
do this for others too, I think those are for when you have the google SEO plugin installed and SEO urls enabled..
(2011-12-28, 10:21 PM)musanaqvi110 Wrote: [ -> ]perhaps you can do it like this,
Disallow: /forum-25.html?datecut=0&sortby=lastpost
do this for others too, I think those are for when you have the google SEO plugin installed and SEO urls enabled..
Your option won't help because every time i will post a new thread i'll have to add another line to robots.txt file the way you told me
I want every url which contains the following keywords/text (datecut/nextoldest/nextoldest/lastpost) to not be indexed
I am using google seo plugin only for Sitemap and Meta description not for Google SEO URL the rest its intact however i have enabled MyBB SEF urls
Do you want to disallow only those four pages? or all nextnewest, nextoldest etc requests?
If so, then I do think that is the correct syntax..
It would work, at least for the Google bot - not all search engines understand * for parameters etc.
Consider using canonical. It will tell Google that those duplicate names are actually the same page.
Google SEO does this (and you don't have to enable any of its other features if you don't want em).
My bad, yes you do not have to have URLs enabled.....
(2011-12-28, 10:34 PM)frostschutz Wrote: [ -> ]It would work, at least for the Google bot - not all search engines understand * for parameters etc.
Consider using canonical. It will tell Google that those duplicate names are actually the same page.
Google SEO does this (and you don't have to enable any of its other features if you don't want em).
Did a search on google for some links seems like google-bot is indexing all the useless threads and actions and skipping all the important ones i added those rules to my robots.txt file i will keep and eye on the search in the couple of days
And yes i do have that option enabled since day one