MyBB Community Forums

Full Version: Large forum import (>3.100.000 messages)
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi,

we're planning to move to MyBB 1.6 so we're trying the merge system.
Our forum has more than 110.000 users and more than 3.100.000 messages, so the web interface will take a lot of days (and we have to periodically check the status to proceed with different modules) to complete the process.
So the question is: is there a way to execute the merge system from the command line? Or have you some suggestions related to the import of a forum of this size?

Thanks,
Lorenzo
my experience is not the same of other forum owners with large size in gbytes, but I can share it too

I use to backup my db files with mysqldump in SSH, downloading it and running forum/wordpress upgrade in my computer with Xampplite with the same core (php) files. Then Uploading via FTP and importing it with mysql command to another newly created database.

http://kb.mediatemple.net/questions/129/...atabase%3F

Set up a local webserver that does not time out requests (i.e. that allows PHP request to take an infinite amount of execution times, and run an infinite amount of queries). Then you can use the Merge system like normal, just with steps that are a lot bigger than default (since they can't time out anymore). It shouldn't take days to finish this way...
Hi,
first of all, thanks for the replies.
The problem isn't to make larger pieces of data at one time (and I believe that with the size we're talking about is not a feseable way, because it would use too many resources) but to make the whole process automatic (with a script, or like a script Smile).
Another missing info: the current platform is SMF 1.1.11.

Lorenzo
"use too many resources" is a term that applies to shared hosting, and hosts with limited resources. on a local PC it does not matter if the webserver "hangs" with 100% CPU usage for a while, a local PC does not have to serve webpages, so you can just let it sit there and do its work. unless it uses too much RAM (as in several gigabytes of) which I doubt.

If you want a script, I guess you'll have to write one (and doing this will likely take longer than just using the merge system), the merge system is designed as interactive web install procedure and that's just how it works. In terms of conversion speed, it should not matter either way...
Yes, Like I did before on a million posts, merge it on local webserver Wink
If possible, tweak your my.cnf setting
Yes if you're on Windows just get WampServer, edit the config of PHP, Apache and MySQL to remove the limitations (memory limit, size limit, time limit so it can use unlimited resources), and you're good to go...
The Merge System has converted over a 3+ million post forum before and it went fine in about 12 hours. Mind you this was in the days of it's initial design. The Merge System has gotten a lot better over time.