Current time: 09-09-2010, 07:13 AM Hello There, Guest! (LoginRegister)


[B] Rebuild Thread Counters - thread cache grows above memory limit
01-28-2009, 08:32 AM (This post was last modified: 01-28-2009 08:41 AM by koziolek.)
Post: #1
[B] Rebuild Thread Counters - thread cache grows above memory limit
MyBB 1.4.4
Rebuilding Thread Counters with large Entries per page (e.g > 5000; with overall threads above 100000) fails with "Allowed memory size of 64 MB exhausted".

Memory usage grows because of static $thread_cache in get_thread() which holds every thread. This is not necessary - we do not have to cache every thread... So maybe limit the size of $thread_cache?

Proof of concept with hard limit (solves the issue, but there can be more elegant solutions) in patch.


Attached File(s)
.diff  015_limit_size_of_static_thread_cache.mod.diff (Size: 1.31 KB / Downloads: 100)

www.kozik.net.pl
- So... Maybe you shouldn't have hacked it.
- And why don't you try not breathing. Hurts, dunnit. (userfriendly.org)
Visit this user's website Find all posts by this user
01-28-2009, 11:23 AM
Post: #2
RE: Rebuild Thread Counters - thread cache grows above memory limit
This also happens with attachment rebuilding.
Visit this user's website Find all posts by this user
01-28-2009, 03:00 PM
Post: #3
RE: Rebuild Thread Counters - thread cache grows above memory limit
How about just limit of the size of your per page? This seems bogus because your the one forcing it to process so many at one time, not us

[Image: ryangordon.png]
"Coming together is a beginning; keeping together is a process; working together is success."
-Henry Ford

To travel is better than to arrive.
Visit this user's website Find all posts by this user
01-28-2009, 03:29 PM
Post: #4
RE: Rebuild Thread Counters - thread cache grows above memory limit
Good luck with recounting 130 000 threads with 500 per page... and 130 000 is not a big number - there are bigger boards...

www.kozik.net.pl
- So... Maybe you shouldn't have hacked it.
- And why don't you try not breathing. Hurts, dunnit. (userfriendly.org)
Visit this user's website Find all posts by this user
01-28-2009, 06:44 PM
Post: #5
RE: Rebuild Thread Counters - thread cache grows above memory limit
We have the defaults between 100 and 200 for a reason. That would be not a bug, but your server unable to handle that much at the same time.

Microsoft is like Umbrela Co., they create the virus and the vaccine, the zombie infestation is just a collateral effect.[/align]
Visit this user's website Find all posts by this user
01-28-2009, 09:44 PM (This post was last modified: 01-28-2009 09:47 PM by Ryan Gordon.)
Post: #6
RE: Rebuild Thread Counters - thread cache grows above memory limit
(01-28-2009 03:29 PM)koziolek Wrote:  Good luck with recounting 130 000 threads with 500 per page... and 130 000 is not a big number - there are bigger boards...

Yep. And those bigger boards have been correctly configured. I should know, since I help run one of them. In fact it is approximately 30 times bigger then yours and recount and rebuilds run smooth and fast.

You need to configure your server correctly if you want to do more then MyBB's default values for recounting.

[Image: ryangordon.png]
"Coming together is a beginning; keeping together is a process; working together is success."
-Henry Ford

To travel is better than to arrive.
Visit this user's website Find all posts by this user
01-28-2009, 11:56 PM
Post: #7
RE: [B] Rebuild Thread Counters - thread cache grows above memory limit
Can the recounting process at least be automated, rather than having to click on a button every time?
Visit this user's website Find all posts by this user
01-29-2009, 12:01 AM
Post: #8
RE: [B] Rebuild Thread Counters - thread cache grows above memory limit
(01-28-2009 11:56 PM)bobbit Wrote:  Can the recounting process at least be automated, rather than having to click on a button every time?

That's a good idea. Can you post it on the suggestions forum or the ideas site?

[Image: ryangordon.png]
"Coming together is a beginning; keeping together is a process; working together is success."
-Henry Ford

To travel is better than to arrive.
Visit this user's website Find all posts by this user
01-29-2009, 12:07 AM
Post: #9
RE: [B] Rebuild Thread Counters - thread cache grows above memory limit
Done.

Also added a little something else in but it's not necessary.
Visit this user's website Find all posts by this user
01-29-2009, 10:21 AM (This post was last modified: 01-29-2009 10:24 AM by koziolek.)
Post: #10
RE: Rebuild Thread Counters - thread cache grows above memory limit
(01-28-2009 09:44 PM)Ryan Gordon Wrote:  You need to configure your server correctly if you want to do more then MyBB's default values for recounting.
First: By configuring server correctly you mean raising up memory_limit? Heh... The memory usage of any script should not raise above some level. No matter what value the user passes to a script. App should still behave correctly with proper memory footprint. User can set "10000 entries per page" and the application can timeout (max execution time) but it cannot eat MB of memory. As I said once here - lazy and bad programming cannot be an excuse for buying more memory...

Second: Anyway, it is a hosting.

www.kozik.net.pl
- So... Maybe you shouldn't have hacked it.
- And why don't you try not breathing. Hurts, dunnit. (userfriendly.org)
Visit this user's website Find all posts by this user


Forum Jump:


User(s) browsing this thread: 2 Guest(s)