MyBB Community Forums

Full Version: Question on large database merge (from SMF 2.0)
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi,

I have a large SMF board that's been in use for many years. There are 95,000+ threads and almost 1.6M posts.

I'm currently in the process of doing the merge, haven't completed it yet. since it takes a probably half a day to complete.

Anyway, my question is whether it is possible to do an "incremental merge" on top of a previous merge.

Example, I'm spending a good few hours doing this merge, but it's only my first test. To stay current with the original SMF board, I would like to do another merge on top of this one, as an incremental, instead of having to do the 1.6M posts all over again.

Is this possible?

thanks
Ricardo
Unfortunately not. I agree that it'd be helpful with such large boards but at this point it'd be pretty hard to implement. The best is probably to lock your smf board for a day to avoid losing posts/threads/users.
Ok thanks.

So I completed the merge. Going to the summary, I see that it appears the "threads" might have been converted properly. The original was around 95,000 and myBB also shows that.

But the posts are completely out of whack. On the big forum there were around 1.5M posts and myBB only shows 199k. The merge process as it went along, did show me it was converting all 1.5M+ posts. But when I access after the merge it only shows 199k posts.

However, mysql shows me that the mybb_posts table has 1,578,543 entries. So I'm guessing it must be some sort of mapping
(2015-09-18, 05:28 PM)rkleemann Wrote: [ -> ]Ok thanks.

So I completed the merge. Going to the summary, I see that it appears the "threads" might have been converted properly. The original was around 95,000 and myBB also shows that.

But the posts are completely out of whack. On the big forum there were around 1.5M posts and myBB only shows 199k. The merge process as it went along, did show me it was converting all 1.5M+ posts. But when I access after the merge it only shows 199k posts.

However, mysql shows me that the mybb_posts table has 1,578,543 entries. So I'm guessing it must be some sort of mapping

Try running the recount and rebuild tools in the MyBB Admin Control Panel. We don't manually count posts every load, we instead cache them in the database and sometimes it can get out of sync after an import/merge.
It should do that automatically since 1.8.6. I'd guess it has something to do with unapproved/deleted posts or special forum settings (not sure how exactly that number is calculated, need to look at that tomorrow)
(2015-09-18, 05:41 PM)Euan T Wrote: [ -> ]
(2015-09-18, 05:28 PM)rkleemann Wrote: [ -> ]Ok thanks.

So I completed the merge. Going to the summary, I see that it appears the "threads" might have been converted properly. The original was around 95,000 and myBB also shows that.

But the posts are completely out of whack. On the big forum there were around 1.5M posts and myBB only shows 199k. The merge process as it went along, did show me it was converting all 1.5M+ posts. But when I access after the merge it only shows 199k posts.

However, mysql shows me that the mybb_posts table has 1,578,543 entries. So I'm guessing it must be some sort of mapping

Try running the recount and rebuild tools in the MyBB Admin Control Panel. We don't manually count posts every load, we instead cache them in the database and sometimes it can get out of sync after an import/merge.

The rebuild and recount worked in terms of ensuring all the threads and posts in the forum. However, the count itself is not updated.

For example, the forum has 91k threads and 1.5M posts. What's showing is 91k threads but only 199k posts (even though it appears all posts are indeed represented and in mysql it does show 1.5M posts)