So I was looking if anything from my new site is indexed or not, and I've seen a couple of topics in there but the problem is that google has indexed user info too.
After the title its like this
Quote:Date - Title. Username(of the thread poster) Offline Junior Member *. Posts: 1. Threads: 1. Thanks Received: 3 in 1 posts
How do I undo that? I don't want it of course. How to sort it out.
Please help.
Thank you
Crawl-delay: 60
User-agent: *
Disallow: /captcha.php
Disallow: /modcp.php
Disallow: /moderation.php
Disallow: /printthread.php
Disallow: /task.php
Disallow: /archive/
Disallow: /online.php
Disallow: /calendar.php
Disallow: /reputation.php
Disallow: /*nextoldest*
Disallow: /*nextnewest*
Disallow: /*datecut*
Disallow: /*lastpost*
Disallow: /*markread*
Disallow: /ratethread.php
Disallow: /report.php
Disallow: /sendthread.php*
Disallow: /usercp.php*
Disallow: /usercp2.php*
Disallow: /newreply.php*
Disallow: /newthread.php*
Disallow: /editpost.php
Disallow: /private.php
Disallow: /search.php
Disallow: /memberlist.php
Disallow: /showteam.php
Disallow: /stats.php
Disallow: /showratings.php
Disallow: /syndication.php*
Disallow: /misc.php
Disallow: /*sortby*
Disallow: /*action=emailuser*
Disallow: /*action=nextnewest*
Disallow: /*action=nextoldest*
Disallow: /*year=*
Disallow: /*action=weekview*
Disallow: /*action=nextnewest*
Disallow: /*action=nextoldest*
Disallow: /*sort=*
Disallow: /*order=*
Disallow: /*mode=*
Disallow: /*datecut=*
Disallow: /usuario-*?action=emailuser
Disallow: /ratethread.php
Disallow: /attachment.php
User-agent: MediaPartners-Google
Allow: /
User-agent: dotbot
Disallow: /
User-agent: 008
Disallow: /
This is my robots.txt, but I only uploaded it a couple of days ago, when I installed the Google SEO plugin, will this work?
Will it also be able to un-index the already indexed and index them properly without the user info that is to be hidden?
Thank you so much for replying.
It's better to use meta noindex here. Blocking with robots.txt won't remove the URLs that are already indexed. They will stay there forever. To remove user profiles from the index simply place <meta name="robots" content="noindex"> in the head section of the member profile template.
Never forget to set up Google spider as member, so you will never have "Hello Guest" near your website description.
I asked once in this forums how to solve that, and got only irrelevant answers.
(2016-05-25, 09:25 PM)Madtrippin Wrote: [ -> ]Never forget to set up Google spider as member, so you will never have "Hello Guest" near your website description.
I asked once in this forums how to solve that, and got only irrelevant answers.
Expect to get a nice warning message soon from Google Webmaster tools starting with "Increase in “soft-404” pages...
having that text "Hello Guest" appear in the index is really not a problem for SEO purposes.
(2016-05-25, 02:48 PM).m. Wrote: [ -> ]you can add Disallow: /member.php to the User-agent: * listing
And if you have signed up with Google WebMasters then you can request removal of specific URLs from indexing.
Thank you m, I've added it to my robots.txt
(2016-05-25, 07:18 PM)Ashley1 Wrote: [ -> ]It's better to use meta noindex here. Blocking with robots.txt won't remove the URLs that are already indexed. They will stay there forever. To remove user profiles from the index simply place <meta name="robots" content="noindex"> in the head section of the member profile template.
Can I place it in the headerinclude template? And will this unindex already indexed member data? Or I should ask for removal of those url?
its better to request removal of
specific pattern URLs at the
Google WebMasters site.
also
<meta name="robots" content="noindex" /> can be added to
member_profile template above
{$headerinclude}
Thank you m. You may lock this now.