Google SEO Plugin XML Sitemap

[Alex Stanford]

I have installed the Google SEO plugin on my forum. But, I am having trouble with the sitemap. Here is the current situation:

In the ACP I have the "Sitemap URL scheme" field set to "sitemap-{url}.xml". In the .htaccess file for the forum I have:

# Google SEO Sitemap:
RewriteRule ^sitemap\-([^./]+)\.xml$ misc.php?google_seo_sitemap=$1 [L,QSA,NC]

# Google SEO URL Forums:
RewriteRule ^Forum\-([^./]+)$ forumdisplay.php?google_seo_forum=$1 [L,QSA,NC]

# Google SEO URL Threads:
RewriteRule ^Thread\-([^./]+)$ showthread.php?google_seo_thread=$1 [L,QSA,NC]

# Google SEO URL Announcements:
RewriteRule ^Announcement\-([^./]+)$ announcements.php?google_seo_announcement=$1 [L,QSA,NC]

# Google SEO URL Users:
RewriteRule ^User\-([^./]+)$ member.php?action=profile&google_seo_user=$1 [L,QSA,NC]

# Google SEO URL Calendars:
RewriteRule ^Calendar\-([^./]+)$ calendar.php?google_seo_calendar=$1 [L,QSA,NC]

# Google SEO URL Events:
RewriteRule ^Event\-([^./]+)$ calendar.php?action=event&google_seo_event=$1 [L,QSA,NC]

When I go to http://techtalkin.com/sitemap-index.xml all seems to look like it is working. But, when you head to the <loc> URLs on that page, you will get a "Sitemap page invalid." error. (e.g. http://techtalkin.com/sitemap-threads.xml?page=1)

Any advice would be appreciated.

[frostschutz]

This user has been denied support.

Do you have two sitemap rules in your .htaccess, with the other coming first and QSA missing?

Are you using an Apache clone?

It could be a plugin interfering, but the rewrite-free variant ( http://techtalkin.com/misc.php?google_se...ads&page=1 ) works fine as opposed to the rewrite variant ( http://techtalkin.com/sitemap-threads.xml?page=1 ) which does not work. So it seems to be missing the page parameter after the rewrite, but with a rewrite rule as shown above and with Apache (any version I'm familiar with) this is not possible.

[Alex Stanford]

Do you have two sitemap rules in your .htaccess, with the other coming first and QSA missing?

Are you using an Apache clone?

It could be a plugin interfering, but the rewrite-free variant ( http://techtalkin.com/misc.php?google_se...ads&page=1 ) works fine as opposed to the rewrite variant ( http://techtalkin.com/sitemap-threads.xml?page=1 ) which does not work. So it seems to be missing the page parameter after the rewrite, but with a rewrite rule as shown above and with Apache (any version I'm familiar with) this is not possible.

Hi frostschutz,

First of all, thank you for your insights.

We are running Apache/fcgi with Varnish Cache and ModSecurity. I don't suspect it's the server environment, as it is heavily tested with other .htaccess dependent applications.

Based on your input it seems likely that another rule in the .htaccess is interfering. Here is the entire .htaccess:

EDIT: I quickly removed all rules except the rules labeled "# Google SEO" (and even the first two of those) for testing and the issue persists. It must not be that...

# Google SEO workaround for search.php highlights:
# Make this rule the first rewrite rule in your .htaccess!
RewriteRule ^([^&]*)&(.*)$ http://techtalkin.com/$1?$2 [L,QSA,R=301]

# Google SEO 404:
ErrorDocument 404 /misc.php?google_seo_error=404

# Google SEO Sitemap:
RewriteRule ^sitemap\-([^./]+)\.xml$ misc.php?google_seo_sitemap=$1 [L,QSA,NC]

# Google SEO URL Forums:
RewriteRule ^Forum\-([^./]+)$ forumdisplay.php?google_seo_forum=$1 [L,QSA,NC]

# Google SEO URL Threads:
RewriteRule ^Thread\-([^./]+)$ showthread.php?google_seo_thread=$1 [L,QSA,NC]

# Google SEO URL Announcements:
RewriteRule ^Announcement\-([^./]+)$ announcements.php?google_seo_announcement=$1 [L,QSA,NC]

# Google SEO URL Users:
RewriteRule ^User\-([^./]+)$ member.php?action=profile&google_seo_user=$1 [L,QSA,NC]

# Google SEO URL Calendars:
RewriteRule ^Calendar\-([^./]+)$ calendar.php?google_seo_calendar=$1 [L,QSA,NC]

# Google SEO URL Events:
RewriteRule ^Event\-([^./]+)$ calendar.php?action=event&google_seo_event=$1 [L,QSA,NC]

AddOutputFilterByType DEFLATE text/css text/javascript application/x-javascript application/javascript text/x-component text/html text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon application/json

DirectoryIndex portal.php

Any ideas?

Thanks,
Alex

[frostschutz]

This user has been denied support.

None other than those I already provided. There is nothing wrong with the .htaccess

You can work around by removing the sitemap url scheme, so it will use misc.php? directly instead. But chances are that this issue (whatever is causing it) will give you trouble later on elsewhere.

[Alex Stanford]

Do you have two sitemap rules in your .htaccess, with the other coming first and QSA missing?

Are you using an Apache clone?

It could be a plugin interfering, but the rewrite-free variant ( http://techtalkin.com/misc.php?google_se...ads&page=1 ) works fine as opposed to the rewrite variant ( http://techtalkin.com/sitemap-threads.xml?page=1 ) which does not work. So it seems to be missing the page parameter after the rewrite, but with a rewrite rule as shown above and with Apache (any version I'm familiar with) this is not possible.

I think I figured it out the cause...

This redirect:

RewriteRule ^sitemap\-([^./]+)\.xml$ misc.php?google_seo_sitemap=$1 [L,QSA,NC]

Would redirect this request:

http://techtalkin.com/sitemap-forums.xml?page=1

To:

http://techtalkin.com/misc.php?google_se...map=forums

Which is a broken link. But, if you go to:

http://techtalkin.com/misc.php?google_se...ums?page=1

Everything works. If I understand correctly, the problem is that the rewrite rule doesn't append the "?page=X" string to the end of the URL. So, I modified the rule:

RewriteRule ^sitemap\-([^./]+)\.xml?page=([0-9]+)$ misc.php?google_seo_sitemap=$1&page=$2 [L,QSA,NC]

But, still no luck...

---

None other than those I already provided. There is nothing wrong with the .htaccess

You can work around by removing the sitemap url scheme, so it will use misc.php? directly instead. But chances are that this issue (whatever is causing it) will give you trouble later on elsewhere.

I removed the "sitemap-{url).xml" in order to try it without rewrites and now I have a new issue:

http://techtalkin.com/misc.php?google_seo_sitemap=index

The links are showing up without a value for the google_seo_sitemap paramater, e.g.:

http://techtalkin.com/misc.php?google_seo_sitemap=&page=1

[frostschutz]

This user has been denied support.

This redirect:

RewriteRule ^sitemap\-([^./]+)\.xml$ misc.php?google_seo_sitemap=$1 [L,QSA,NC]

Would redirect this request:

http://techtalkin.com/sitemap-forums.xml?page=1

To:

http://techtalkin.com/misc.php?google_se...map=forums

No, it would go to http://techtalkin.com/misc.php?google_se...ums&page=1 because the rewrite rule is using the QSA option (query string append) which appends any and all parameters given in the original URL.

If query string append is broken on your host in general, other basic features such as thread pages, post links, et cetera would all break too. You are not supposed to cover parameters like page or pid or highlight or any combinations thereof in rewrite rules. I couldn't test if those are broken on your forum because I didn't see a thread more than one page long.

I removed the "sitemap-{url).xml" in order to try it without rewrites and now I have a new issue:

http://techtalkin.com/misc.php?google_seo_sitemap=index

The links are showing up without a value for the google_seo_sitemap paramater, e.g.:

http://techtalkin.com/misc.php?google_seo_sitemap=&page=1

Then maybe I remembered wrong and you have to put misc.php?google_seo_sitemap={url} as scheme

[Alex Stanford]

Then maybe I remembered wrong and you have to put misc.php?google_seo_sitemap={url} as scheme

I didn't think so because it says "If your host does not support mod_rewrite, leave this empty. Your sitemap will then be called misc.php?google_seo_sitemap=index."

But, I tried changing the scheme to "misc.php?google_seo_sitemap={url}" and it sort of works: http://techtalkin.com/misc.php?google_seo_sitemap=index

The only problem is that it is appending the page query string to the URL via "?" instead of "&"

[frostschutz]

This user has been denied support.

Now, finally that's a bug I have to fix. Thanks for pointing it out to me.

You should still check whatever is going wrong with your rewrites, though!

[Alex Stanford]

Now, finally that's a bug I have to fix. Thanks for pointing it out to me.

You should still check whatever is going wrong with your rewrites, though!

Can you post a fix for that issue here so that I can patch my copy until you release a version with the fix?

[frostschutz]

This user has been denied support.

I updated the plugin, you can grab it from GitHub while it's not validated on the mods site.