I'll see what I can sort out... ...but its not any one particular thread/post - they are all equally slow, the conversion rates for posts seem to be about 1000 posts per hour on my dedicated server (not having to test on my shared hosting account, as all this testing on our dedicated server was hitting performance of the live (still YaBB) site on the same box)
Thats very very strange, normally at 1000 posts per page (the default) it runs a page every 3-5 seconds on every convert I've done except one. That one had very lengthy posts and averaged somewhere between 10 and 15 seconds per page at 1000/pg. You're not dropping that number down to something ridiculously low like 10 posts per page or something are you? That'll slow it down a LOT since its running so many extra SQL queries.
(2011-05-17, 02:56 AM)Dylan M. Wrote: [ -> ]Thats very very strange, normally at 1000 posts per page (the default) it runs a page every 3-5 seconds on every convert I've done except one. That one had very lengthy posts and averaged somewhere between 10 and 15 seconds per page at 1000/pg. You're not dropping that number down to something ridiculously low like 10 posts per page or something are you? That'll slow it down a LOT since its running so many extra SQL queries.
PM sent with details for trying if you want to see if you can see whats going wrong...
Is it possible for you to share the smf database dump? I'm sure Dylan will be running tests to get to the bottom of this and I would like to run a few tries of the import myself and make sure its nothing on the database side. If you're ok with sharing the dump then you can upload it on dropbox or something due to the size and PM me the details.
I have his details in PM, if he's OK with it I can get you a copy of the db from a private source only Staff can access.
Yeah he PM'd them to me too, I got the dump from phpmyadmin. Will test it this weekend.
Ok, I wont be able to do anything with this until Monday or Tuesday because I'm working every day except Sunday, and we have family plans that day.
Thanks guys, I look forward to seeing what you find when you get chance to have a play next week
OK, I ran some tests and found one place where the merge was getting bottlenecked. With the change sugested below, I was able to get the posts imported in a little over 3 hours on a beat up old P4, so I'm sure it'll be better for you on your system.
Before starting the posts module from the merge, run this query in phpmyadmin:
ALTER TABLE `mybb_threads` ADD INDEX (`import_firstpost`)
Change the mybb_ prefix with the one you are using if its different.
Also, with the above index added, I bumped the number of posts to import per page from 1000 to 5000 without any significant slow down.
Interesting, before Ryan left he added indexes to all the import_ fields... or so I thought. He must have missed that one. I'll go back and add it. Thanks G33K.