MyBB Community Forums

Hello, all.

I know there is another thread on this topic, but it…uhm…devolved. I'm having the same issue, and looking through the code in the 1.4 Merge, I have hopefully a few insights.

What happens? The Uploads folder is empty; none of the attachments actually copied. Errors were only generated after the merge happened (i.e., the module can find my SMF attachments folder just fine, apparently). The DB entries for the attachments all look good, but the files themselves do not exist.

Errors generated: At the end of the merge, the generated report gives an error for each and every attachement in the SMF DB, that looks like this:

The following errors were logged during the process of the Merge System:

    * Attachments:
          o Error could not find the attachment thumbnail (ID: 1)
          o Error could not find the attachment thumbnail (ID: 2)
          o Error could not find the attachment (ID: 3)
          o Error could not find the attachment thumbnail (ID: 4)
          o Error could not find the attachment thumbnail (ID: 5)
          o Error could not find the attachment (ID: 6)
          o Error could not find the attachment thumbnail (ID: 7)

etc. I have quite a number of attachments, so I've truncated this list for convenience.

Interesting observation: There is no attachment with ID 1

Interesting observation 2: The attachment module never attempts to copy attachments with thumbnails (attachments with ID_THUMB != 0 in SMF DB). Perhaps the author misunderstood the role of ID_THUMB? It indicates that the attachment has a thumbnail, and that thumbnail is the attachment with ID_ATTACH equal to the value of ID_THUMB of the regular thumbnail. I notice that the module understands this differently and incorrectly: It presumes that if ID_THUMB !=0 then that attachment is a thumbnail (correct reading: It has a thumbnail). This is an error.
An example from my own SMF DB, I have the following two entries:

ID_ATTACH 	ID_THUMB 	ID_MSG 	ID_MEMBER 	attachmentType 	filename
2 	3 	0 	0 	0  	KA-24-236_01_lg.jpg
3 	0 	0 	0 	3  	KA-24-236_01_lg.jpg_thumb

The first entry, ID_ATTACH=2, is a full attachment, whose thumbnail is the attachment wwith ID_ATTACH=3. The thumbnail, notice, has ID_THUMB=0 and attachmentType=3 (I do not know what attachmentTypes 1 or 2 are).

Fix:
in boards/smf/attachment.php change

                // Transfer attachment thumbnail
                $thumb_not_exists = "";
                if($data['ID_THUMB'] != 0)

to

                // Transfer attachment thumbnail
                $thumb_not_exists = "";
                if($data['ID_THUMB'] == 0)

Not so interesting observation 3: This error doesn't seem to be the cause of the missing attachment files. generate_raw_filename doesn't care whether it is looking at a thumbnail or not; nor does it need to care. But it must be failing somewhere to cause the "could not find the attachment" errors for each file. The only place I can figure that the generate_raw_filename function might be going wrong is in the MD5 bit. But a little testing on the filenames above shows that the function appears to be returning the correct results. So I'm a little mystified…UNLESS

Interesting observation 4:
Recall that I got an error for ATTACH_ID = 1. But there is no attachment in my SMF DB with ATTACH_ID = 1. Is this a off-by-one error, perhaps? So that generate_raw_filename is generating, for the attachment with filename KA-24-236_01_lg.jpg

1_KA-24-236_01_lg_jpgee3bcaecf39f06549b68a93163f99740

instead of the correct

2_KA-24-236_01_lg_jpgee3bcaecf39f06549b68a93163f99740

I wonder? The module is just complex enough that I can't tell readily just browsing the code.

So: I wonder: Are the SMF attachment importation errors a matter of mis-generating the filename based off an off-by-one error in the ID_ATTACH of the attachment?

(holy crap that was a long post! Sorry! But I do hope this is helpful?)

(this merge is being done on a local machine to test the import process for bugs, so I can't send a link; However, I'll gladly provide whatever else I can to help figure this one out.)

Well. Lets see. Hmm. To address your points.

1) That ID refers (I believe) to the post Merge ID of the attachment.

2) Actually that is correct. If you actually look at whats inside that IF you're screwing things up by changing it. What it does is check if there is a thumbnail, and then loads the thumbnail using the get_import_attach_filename() function. So your "fix" is actually messing things up. I document this for you below.

3) There isn't anything to say here, I know for a fact that attachments are able to be converted correctly as I've done many many boards.

4) As I stated in 1, I don't believe this is the ID_ATTACH from SMF, I believe this is the new ID once inserted into the MyBB db.

Most likely you are NOT providing the correct path to the location of SMF attachment files. Can you post me the path you are using, and your directory tree for MyBB and SMF in your localhost folders.

Code diagnosis just for completeness in explanation:

		if($data['ID_THUMB'] != 0) // If the SMF db says we have an attachment here...
		{
			// add the thumbnail ID to our array of thumbnails, which is so that we don't try
			// and transfer the thumbnail as a regular attachment
			$this->thumbs[] = $data['ID_THUMB'];
			// Now we load our thumbnail
			$thumbnail = $this->get_import_attach_filename($data['ID_THUMB']);
			// and here we determine the proper file extension for the thumbnail
			$ext = get_extension($thumbnail['filename']);
			
			// Lastly we set the thumbnail in the database for this attachment
			$insert_data['thumbnail'] = str_replace(".attach", "_thumb.$ext", $insert_data['attachname']);
		}

(2010-04-29, 04:08 PM)ralgith Wrote: [ -> ]Well. Lets see. Hmm. To address your points.

We're looking in different parts of the source.

I'm looking in convert/boards/smf/attachment.php starting at line 189. You're looking at the code at line 173. The problem isn't the insertion into the new DB (which worked just fine for me), but the locating and copying of the attachment file itself.

Its still valid, as that function you're looking at is transferring the thumbnail for the attachment that is generated using the code I quoted.

You'll notice its transferring the THUMBNAIL not the actual attachment. That is done in the next code block below the if, the "else" block.

$attachrs = @fopen($mybb->settings['uploadspath'].'/'.$insert_data['thumbnail'], 'w');

(2010-04-29, 04:27 PM)ralgith Wrote: [ -> ]Its still valid, as that function you're looking at is transferring the thumbnail for the attachment that is generated using the code I quoted.

You'll notice its transferring the THUMBNAIL not the actual attachment. That is done in the next code block below the if, the "else" block.

$attachrs = @fopen($mybb->settings['uploadspath'].'/'.$insert_data['thumbnail'], 'w');

Fair enough. I am pretty sure at this point that I must be misunderstanding the parameters to after_insert(). Could you say a little about how after_insert() gets called and what the parameters being passed in are? I think this is the line that got me:

        function after_insert($data, $import_data, $aid)
        {
                global $import_session, $mybb, $db;

                // Transfer attachment thumbnail
                $thumb_not_exists = "";
                if($data['ID_THUMB'] != 0)
                {
                        // Transfer attachment thumbnail
                        $attachment_thumbnail_file = merge_fetch_remote_file($import_session['uploadspath'].$this->generate_raw_filename($data));
...
                }
                else
                {
                        // Transfer attachment
                        $attachment_file = merge_fetch_remote_file($import_session['uploadspath'].$this->generate_raw_filename($data));
...
                }

$data contains a reference to the line from the SMF attachment table, and $import_data from the MyBB table, right? What is $insert_data? It's not declared in that function. I suspect that I don't understand what this var contains and that might be my source of consternation.

So, here is how I'm reading the code snippet I just posted (tell me where I've gotten confused): The function is trying to determine, first, whether $data contains a reference to an attachment or an attachment thumbnail. It does so by looking at $data['ID_THUMB']; if ID_THUMB !=0, then the code takes $data to be a reference to an attachment thumbnail; else, then the code takes $data to be a reference to an attachment. But of course, ID_THUMB is positive for full attachments and 0 for thumbnails; so my understanding of how this bit of code works suggests that the test is backwards from how it should be: Thumbnails have ID_THUMB=0, and attachments have ID_THUMB!=0. But I must be seeing something incorrectly here.

That said: I'm gonna give the merge another go, and double check my paths. I tried the merge once with an invalid path, and it kindly told me the path was invalid, so I presumed when it didn't complain after I corrected the path that it must have been correct. It threw no error at all the second time around. But I'll give it a second go and see.

FWIW, the paths I'm using are:

MyBB installation (under MAMP on a Mac, hence the unorthodox location):
/Users/dgoodman/Sites/
SMF attachments (as entered into the Merge software) (actually a local backup of the SMF files)
/Volumes/Backup\ Journal/Websites/jnsforum.com\ Backup.0/public_html/attachments

I wonder if its possible that MAMP can't see that folder due to a permissions issue. I know Macs use UNIX style permissions.

Try this...
Copy the attachments folder to your /Sites/ folder. Rename it smf_attachments if need be. Go into a terminal and browse to /Users/dgoodman/Sites and issue this command: "chmod -R 777 attachments" without the quotes. This will eliminate permissions and file path issues for sure.
Then try using that location for the attachments. If it works, just delete the folder when you're done. If not, come back here for help.


To answer your questions above though:
$data is the data read from your SMF database
$insert_data is the converted data for the MyBB database

So what its doing is checking if the current $data position has a thumbnail, and if so bringing that thumbnail over using the proper $insert_data locations and filenames.

(2010-04-29, 05:04 PM)ralgith Wrote: [ -> ]I wonder if its possible that MAMP can't see that folder due to a permissions issue. I know Macs use UNIX style permissions.

Try this...

Doing that now! Will report back.

Quote:To answer your questions above though:
$data is the data read from your SMF database
$insert_data is the converted data for the MyBB database

So what its doing is checking if the current $data position has a thumbnail, and if so bringing that thumbnail over using the proper $insert_data locations and filenames.

I modified my post above while you were replying (oops!) I read some more of your code and got a better sense of things. The first two parameters to after_insert are the SMF row and the MyBB row, respetively, right? But the parameter names are
$data
and
$import_data

What is $insert_data? It's not declared in that function anywhere, nor as a global. It is declared as a local in a different function in the same file…

---

As an aside, obviously my understanding of the code is immaterial to the code's working. I'm OK with that. But something isn't working, and I'm trying to understand why it isn't working so as to be at least somewhat helpful in this process.

And, indeed, it isn't working. I copied the attachments folder right over to the MyBB install, as such:

Bacon:Sites dgoodman$ ls -al
total 2280
drwxr-xr-x    55 dgoodman  dgoodman    1870 Apr 29 12:09 .
drwxr-xr-x    58 dgoodman  dgoodman    1972 Apr 29 12:08 ..
-rw-r--r--@    1 dgoodman  dgoodman    6148 Apr  6 13:58 .DS_Store
-rw-r--r--     1 dgoodman  dgoodman     236 Apr 28 14:59 .htaccess
-rw-r--r--     1 dgoodman  dgoodman       0 Feb 22  2007 .localized
drwxr-xr-x     9 dgoodman  dgoodman     306 Apr 19 02:07 Documentation
drwxr-xr-x     8 dgoodman  dgoodman     272 Apr 19 02:07 admin
-rw-r--r--@    1 dgoodman  dgoodman    2538 Apr 19 02:07 announcements.php
drwxr-xr-x     6 dgoodman  dgoodman     204 Apr 19 02:07 archive
-rw-r--r--@    1 dgoodman  dgoodman    3599 Apr 19 02:07 attachment.php
drwxr-xr-x  5505 dgoodman  dgoodman  187170 Apr 29 12:10 attachments
...etc.

The permissions are good (i.e. everyone can read from the folder)

But the uploads folder remains empty after importing the attachments:

Bacon:Sites dgoodman$ ls -al uploads/
total 8
drwxrwxrwx   4 dgoodman  dgoodman   136 Apr 29 12:10 .
drwxr-xr-x  55 dgoodman  dgoodman  1870 Apr 29 12:09 ..
drwxrwxrwx   3 dgoodman  dgoodman   102 Apr 28 21:03 avatars
-rw-r--r--@  1 dgoodman  dgoodman    67 Apr 19 02:07 index.html

No errors were reported during the import (see attached screenshots).

[attachment=18092]
[attachment=18093]

The same errors as before were reported in the post-merge report.

Hmm, I thought this bug had been fixed in the latest MyBB Merger system. You are using the latest version, yes? You are correct in noting that discrepancy. It is a bug I thought I had reported and was fixed already. Obviously I didn't reporting it now.

In the function call, change import_data to insert_data and try again. If you look in one of the other modules, for example phpBB3's attachments, you'll see they use $insert_data instead of $import_data

(2010-04-29, 05:27 PM)ralgith Wrote: [ -> ]Hmm, I thought this bug had been fixed in the latest MyBB Merger system. You are using the latest version, yes? You are correct in noting that discrepancy. It is a bug I thought I had reported and was fixed already. Obviously I didn't reporting it now.

In the function call, change import_data to insert_data and try again. If you look in one of the other modules, for example phpBB3's attachments, you'll see they use $insert_data instead of $import_data

Makes sense. I am using 1.4 Final (screenshot attached, meant to include it last time

I'll make the fix and try again.


[attachment=18094]

Ok, bug report submitted:
http://dev.mybboard.net/issues/906