Big topic, threaded view, page to big ('allowed memory limit exhausted')

[koziolek]

Topic with 1932 replies (97 pages) - when viewing in "threaded view" PHP exits with something similar to:
Allowed memory size of 16 MB exhausted (tried to allocate ~500kB) in inc/functions.php line 118

Line 118 is on of str_replace in function parsepage($contents). I have checked strlen($contents) when PHP fails - it showed a page about 520 kB. I think this size of output is to big.

I have put memory_get_usage() in that function at the beginning and after every str_replace. Effects (in bytes):

15 782 568 (beginning of function)
16 304 704 (after '$contents = str_replace("<navigation>", buildnav(1), $contents);')
16 304 832
16 304 840
16 304 840

Effects in normal page ($contents=100kb) - about 1 MB memory.

Solution? Make a pagination to threaded view? Or look closer why memory use grows so much before parsepage() (maybe use function unset() to free memory?)?


Apache: 1.3
MyBB: 1.1.7
PHP: 5, eaccelerator v0.9.5-rc1
MySQL: 4.1

[Michael S.]

Moved into general support as it isn't a bug. It's a limitation by the server settings. You have 16 MB of memory for the script to run but it needs 500kb more than allowed.

For reference:
http://www.tech-recipes.com/php_programm...ps777.html

[koziolek]

I know the server-side solution. But it is not a proper advice. I do not seek advice but rather I am posting bug (about MyBB code which produces too much memory use) or a suggestion (make a pagination to threaded view)...

As I said and the page you have linked:

Keep in mind that a huge memory limit is a poor substitute for good coding.

[brianmn]

hmmm.....it seems to me that this issue should be solved....

The bigger a thread gets shouldn't make it so that it uses more memory....

No matter how many replies and such you have it should use about the same ammount of memory. I mean obviously if you have very large posts, compared to small ones, it will use more.

In threaded view you need to have it have pages for the listing of thread.

Here is the issue:
http://community.mybboard.net/showthread...1&pid=#pid
That shouldn't place that long of a list on the page.... you should have pages so that memory requirements are much lower.

[koziolek]

Memory usage grows after this code (showthread.php, about line 370):

		$query = $db->query("SELECT u.*, u.username AS userusername, p.*, i.path as iconpath, i.name as iconname FROM ".TABLE_PREFIX."posts p LEFT JOIN ".TABLE_PREFIX."users u ON (u.uid=p.uid) LEFT JOIN ".TABLE_PREFIX."icons i ON (i.iid=p.icon) WHERE p.tid='$tid' AND p.visible='1' ORDER BY p.dateline");
		while($post = $db->fetch_array($query)) {
			if(!$postsdone[$post['pid']]) {
				$tree[$post['replyto']][$post['pid']] = $post;
				if($post['pid'] == $mybb->input['pid'] || ($isfirst && !$mybb->input['pid'])) {
					$isfirst = 0;
				}
				$postsdone[$post['pid']] = 1;
			}
		}

Variable "$tree" contains all posts data so it is really huge. It can be shrinked only by adding pagination.

Another good solution is to unset() some variables. It reduces memory usage from 16 MB to 2 MB.

See DIFF file.

[hemi]

Just a thought so don't flame me over this

I'm sure you may know this but I want to make mention anyhow, To clear that up find the php.ini file

add the line

memory_limit = 16m (Or whatever you want)

to the php.ini file.

However, If you keep getting "memory size exhausted" errors on the backup? PUT THE PHP.INI file inside the Admin folder. Just putting it in the main directory file may not stop the errors. On occasion only when installing the php.ini file containing the extra memory limit into the Admin directory from where the backup.php file works you will get an error free backup. I suggest putting a php.ini into your admin and set it is what I'm trying to say

[koziolek]

Changing memory limit is not a good solution for this problem - it is a temporary fix. I think the code should be fixed (specially the enormous variable "$tree")...

[Peter]

It would be interesting to see if this problem would still occur on MyBB 1.2. When it is released, you should notice significant speedups on your board.

[Asad_Niazi]

Yes. The code needs to be fixed. I didn't bother with the pagination yet, but please try to replace the query with this one and later we can work on the pagination part .

		$query = $db->query("SELECT u.*, u.username AS userusername, p.pid, p.replyto, p.subject, p.dateline, IF(p.pid <> ".intval($mybb->input['pid']).", '', p.message) as message, i.path as iconpath, i.name as iconname FROM ".TABLE_PREFIX."posts p LEFT JOIN ".TABLE_PREFIX."users u ON (u.uid=p.uid) LEFT JOIN ".TABLE_PREFIX."icons i ON (i.iid=p.icon) WHERE p.tid='$tid' AND p.visible='1' ORDER BY p.dateline");

If that slows it down because of some work being given to MySQL, then try using this instead.. Find:

				if($post['pid'] == $mybb->input['pid'] || ($isfirst && !$mybb->input['pid'])) {
					$isfirst = 0;
				}

Replace with:

				if($post['pid'] == $mybb->input['pid'] || ($isfirst && !$mybb->input['pid'])) {
					$isfirst = 0;
				} else {
					$tree[$post['replyto']][$post['pid']]['message'] = '';
				}

Signatures also play a role here, so you can use something in similar fashion to fetch the required signature only. An addition of a similar if statement in query or a similar way to set it to an empty string.

Try using both ways and let me know if it helps.

[koziolek]

1. First modification (wipe out post['message']) reduced memory usage from 16 MB to 12,37 MB (memory usage counted before and after building $tree variable).
2. After deleting user['signature'] - reduced to 12,1 MB
3. After wiping all user data - reduced to 2,5 MB. And everything looks fine .

I split the query to 2 separate ones - one for fetching first post (visible) and second for fetching everything else. See DIFF file.