Well Ryan you're right. I guess I will just deal with merging how I have been with the start, find problem, fix, restart method. Cumbersome but if there is no alternative then so be it. I was hoping to help. Sorry to disturb.
MHryano, I found and fixed an issue with usergroups however, I can't be sure if it was actually the source of the problem or not.
However, let me explain to you a little bit on how the process works. If you open /convert/boards/vbulletin3/users.php and look at the process() function you'll see
$insert_data['import_usergroup'] = $this->board->get_group_id($data['usergroupid'], array("original" => true));
$insert_data['import_additionalgroups'] = $this->board->get_group_id($data['usergroupid'], array("original" => true));
$insert_data['import_displaygroup'] = $data['displaygroupid'];
So as you can see, the import_* fields are simply those copied over from the vB database and stored in the MyBB one until the usergroups converter is run.
If you open up /convert/boards/vbulletin3/usergroups.php and find:
// Restore connections
$db->update_query("users", array('usergroup' => $gid), "import_usergroup = '".intval($group['usergroupid'])."' OR import_displaygroup = '".intval($group['usergroupid'])."'");
As you can see it is there that the actual usergroup connections are restored for custom usergroups. If it wasn't a custom usergroup then it was handled by the get_group_id() function in /convert/boards/vbulletin3.php
However take a look at process() function for the usergroups.php file - As you can see it doesn't take into account the possible "is banned group?" switch. Now I don't remember if this was simply missed or removed for a reason but either way, it should be a simple case of switching it back on in the MyBB ACP after the Merge is completed.
I hope this makes things clearer. It's not as easy as it looks
Oh and I might also mention that the debug data that's being generated as the conversion goes on might be also causing the slow downs. It might go faster if you delete the debug logs in convert/logs/ but then you'll loose that debug data for further analysis. You could pause ("< Back") on the posts converter, archive the logs, and then delete what's in /convert/logs/ so far. Then you can simply continue.
If you want to simply turn off the debug data open up convert/index.php and find:
define("WRITE_LOGS", 1);
and set 1 to 0
Ryan
Ryan,
I haven't got a convert/boards/vbulletin3/ directory, just a convert/boards/vbulletin3.php although in that file, there are lines 188-189
// Restore connections
$db->update_query("users", array('usergroup' => $gid), "import_usergroup = '{$group['usergroupid']}' OR import_displaygroup = '{$group['usergroupid']}'");
and lines 292-294
$insert_user['import_usergroup'] = $this->get_group_id($user['usergroupid'], array("original" => true));
$insert_user['import_additionalgroups'] = $this->get_group_id($user['usergroupid'], array("original" => true));
$insert_user['import_displaygroup'] = $user['displaygroupid'];
The restore connections line is slightly different to what you have said though.
I've turned off the debug data now. It would have been completed by now, but my daily backups caused the MySQL server to stop responding and it stopped the script running
Perhaps changing
$insert_group['isbannedgroup'] = 0;
to
$insert_group['isbannedgroup'] = $isbanned;
and building $isbanned using
SELECT usergroupid FROM vb_usergroup WHERE NOT(genericoptions & 32);
If that query is true then the usergroup is a banned usergroup (in vB3.7). Or, you could run that after importing the usergroups and do an alter query on the banned usergroups to set them as such after importing them all.
Although, none of this explains why my users haven't ended up in the right usergroups
From my vBACP
Quote:(COPPA) Users Awaiting Moderation -
Administrators 2
Real Moderators -
Registered Users 23,815
Super Moderators -
Unregistered / Not Logged In -
Users Awaiting Email Confirmation 3,420
---- Custom Usergroups Below this point ----
100+ Posts 1,498
5+ Posts 6,137
50+ Posts 814
Banned Users 3,098
Banned Users (Auto) 4,162
Banned Users (Infractions) 827
Banned Users (Manual) 1
Bots - WNZ 254
Head Moderators 26
No Adverts 0
Restricted Permissions 3
Site Moderators 36
vBSupport 1
Deep Dungeon Access (additional usergroup)
- 35
vBulletin Users (additional usergroup) - 133
From my MyBB ACP
Quote:Guests
0
Registered
40,664
Super Moderators
0
Administrators
3
Awaiting Activation
3,416
Moderators
0
Banned
0
Banned Users (Manual)
0
Bots - WNZ
1
5+ Posts
2
50+ Posts
0
100+ Posts
0
Deep Dungeon Access
0
vBulletin Users
0
Selected Bots and Troublesome Users.
0
Banned Users (Infractions)
0
vBSupport
0
No Adverts
0
Banned Users (Auto)
0
Head Moderators
0
Site Moderators
0
I hope this helps
I marked the point that my custom usergroups start in my vB ACP there for you too.
I think we are making progress though.
Hm, sorry, that was from my development copy of the new Beta 5. I'll look into the isbannedgroup option. Do you know if that only applies to vB 3.7 or other versions? Also, do you know if vB keeps a list of bitwise operators for usergroups?
In addition, are you aiming for this to be your only conversion or is this just a "test" run? If so, I believe I can have Beta 5 up soon.
Thanks,
Ryan
Don't worry
This is the only conversion for this site. I've got another two sites to go though, so this one can identify issues and the others can be for knocking out issues and testing the new merge code.
It's a contradiction in terms though, this is the smallest site but has the biggest database. The other two are around the 5.5m-6m posts mark each and the databases are around the 10GB-15GB mark for each one.
The bitfields are in the xml files for vBulletin.
(2009-05-21, 02:20 AM)MHryano Wrote: [ -> ]Out of curiosity, what is the largest site currently running MyBB?
Wow, my forum will be shift two or three from MyBB biggest list forum
Goodluck MHryano.
I remember last time, I convert my forum 2 days with Xamp from my PC.
I'm back again
I keep getting MySQL Gone Away errors. I've tweaked the MySQL settings on my server which should take care of it, but the query that is causing the error is
SELECT tid, import_tid FROM mybb_threads WHERE import_tid>0
This is in the posts importer and started after about 1.7m posts. I'm at 1.9m posts now, with 1.3m still to go. Progress has slowed almost to a standstill as I leave it while I do other things/go out/sleep and when I come back to it it hasn't done anything and has stopped.
I can't find that query in the boards/vbulletin3.php file, so I can't see the significance of it but I'm wondering if it can be tweaked to not need to load the full 350k threads tid and import_tid for processing 1k threads. I think it would probably be quicker to get the import_tid of the post from the source database then search the thread table for the tid from the import_tid on a per post basis.
Ah, found where it is doing that. I have an idea, so I'm gonna try it and see if it makes any difference.
hmmm... that wasn't the problem, and the changes didn't make any difference so I put them back. It was one of my other databases that had a few locked queries that caused my server to fall over. The import is running again now
Yeh, it needs that cache and we can't change it
Ok, we have a major problem. It will not get past 2mil posts at all.
Have tried adding indexes to the import_* rows in the database, have made code changes to adjust the number of posts to import at once, have adjusted settings on the server, and it will not proceed at all. It slowed down to 1000 posts imported per 4 hours after 1.8 mil and has finally stopped completely.
On the plus side, I found the reason why we can't set the number of posts per page....
Line 927 of convert/boards/vbulletin3.php is
$import_session['posts_per_screen'] = 1000;
which is then of course set before the form generation for if it isn't set.
I think we need to put our heads together and see if there are any other ways of doing this. I'm not against doing a straight copy of the database, dumping everything other than forums, threads and posts, clearing unneeded fields and renaming fields as needed, then popping an extra prefix on them, reinstalling mybb, running the converter for everything other than forums, threads and posts, then dropping the default tables for them, renaming the manually altered tables, and running a script to associate the users with their posts.
At the end of the day though, if I can't convert then I am stuck with a broken vBulletin which I don't want to use and will never be able to get away from.