MyBB Community Forums - VPS server configuration

Pages: 1 2

I know there are a lot of moving parts that make up the entire system. But we've been experiencing some increasing lag times the last few weeks that are becoming serious problems. Looking for your experience to try to fix this. Or at least, what you consider to be the most important configuration parameters and what you run with successfully.

Hosting on a GoDaddy Centos7 / Apache2.4 / PHP 7.4.16 / MySQL 5.7.33 server since 11/2020, earlier on similar Centos6 since 11/2019, shared servers before that. MyBB 1.8.26 with plugins: Absolute-Relative Time, Board Announcements Manager, Courtesy Edit Time, Haz Spoiler, PHP and Template Conditionals 2.1, and Posts per Page

Curiously, phpMyAdmin server details reports using PHP 7.3.27

The problem: 90 active members, 31K threads, 492K posts, 405K attachments / 135GB. Picture posts may contain 100 to 150 images, displaying full images, not thumbnails, downloading 30 to 40MB max. Performance lagging since late February and increasing. If members stay out of the picture threads, performance does not suffer, sometimes picture threads deliver satisfactorily, sometimes delivery drags out to several minutes, and that's when every access suffers.

Server is self-managed VPS. GoDaddy's previous model was partially managed, but was changed to Self- or Fully-Managed. Paid assistance is available on a per-case basis. I had their support team tweak configuration settings very recently, and that seems to have made things slightly better, then it got worse.

At the beginning of February and again the beginning of March, GoDaddy sent emails that they were upgrading their systems and our server would experience some short outage followed by a reboot. Neither of the two instances resulted in performance falling off a cliff, but I can't help but think that something has changed. The forum is quite stable in terms of members and usage.

I have root access. I'm not a dummy, but I'm not an expert either. This performance degradation has me stumped.

Your thoughts?

The currently installed packages on the server.
Apache 2.4

config
config-runtime
mod_bwlimited
mod_cgid
mod_deflate
mod_expires
mod_headers
mod_http2
mod_mpm_event
mod_proxy
mod_proxy_fcgi
mod_proxy_http
mod_proxy_wstunnel
mod_ssl
mod_unique_id
mod_version
tools

PHP 7.3

libc-client
pear
php-bcmath
php-calendar
php-cli
php-common
php-curl
php-devel
php-fpm
php-ftp
php-gd
php-iconv
php-imap
php-litespeed
php-mbstring
php-mysqlnd
php-pdo
php-posix
php-sockets
php-xml
runtime

PHP 7.4

libc-client
pear
php-bcmath
php-calendar
php-cli
php-common
php-curl
php-devel
php-fpm
php-ftp
php-gd
php-iconv
php-imap
php-litespeed
php-mbstring
php-mysqlnd
php-opcache
php-pdo
php-posix
php-sockets
php-xml
runtime

Additional Packages

apr
apr-util
brotli
cpanel-tools
documentroot
libargon2
libcurl
libnghttp2
libxml2
nghttp2
oniguruma
oniguruma-devel
openssl11
php-cli
php-cli-lsphp
profiles-cpanel

Quote:I had their support team tweak configuration settings very recently, and that seems to have made things slightly better, then it got worse.

At the beginning of February and again the beginning of March, GoDaddy sent emails that they were upgrading their systems

Won't it be possible that their tweaks got lost when upgrading their systems?

you can use debug information of mybb https://community.mybb.com/thread-91870.html ...
and check what query is slow...

if slow query is plugin related you can disable or find another solution...

you can also analyze, repair and optimize your database. This can improve performance.

of course better check too memory usage, cpu usage etc...
if memory usage is very hight (and hight usage of swap), set database to use 50 to 75 percent of system memory or switch to a higher capacity server

(2021-04-15, 01:32 AM)Omar G. Wrote: [ -> ]Won't it be possible that their tweaks got lost when upgrading their systems?

Their tweaks were made over last 2 or 3 days ago.

(2021-04-15, 01:36 AM)martec Wrote: [ -> ]you can use debug information of mybb https://community.mybb.com/thread-91870.html ...
and check what query is slow... if slow query is plugin related you can disable or find another solution...

you can also analyze, repair and optimize your database. This can improve performance.

of course better check too memory usage, cpu usage etc...
if memory usage is very hight (and hight usage of swap), set database to use 50 to 75 percent of system memory or switch to a higher capacity server

Debug info is enabled and shows some queries in the millisecond range, otherwise all are in microseconds.

97ms

SELECT COUNT(aid) AS unapprovedattachments FROM mybb_attachments WHERE visible=0

23ms

SELECT COUNT(tid) as threads FROM mybb_threads WHERE fid = '2' AND (lastpost >= '1618455212' OR sticky=1) AND visible IN (-1,0,1) ORDER BY lastpost DESC

I have used phpmyadmin / status / monitor to watch when working well. As in this instance:
[attachment=44013]

When same request, but not working well. Notice the gap in monitor timing.
[attachment=44014]

Analyze, repair and optimise the database? Is there community discussion of those details? Previously

Memory usage in phpmyadmin monitor graphs is up to 3.4GB used after GoDaddy's tweaks, but still long delays.

I'm not expecting someone with a magic wand and a simple fix. But wanted to get other people's experience to point me in the right direction.

Thanks for any help in that regard.

(2021-04-15, 03:25 AM)HLFadmin Wrote: [ -> ]Debug info is enabled and shows some queries in the millisecond range, otherwise all are in microseconds.

97ms
SELECT COUNT(aid) AS unapprovedattachments FROM mybb_attachments WHERE visible=0
23ms
SELECT COUNT(tid) as threads FROM mybb_threads WHERE fid = '2' AND (lastpost >= '1618455212' OR sticky=1) AND visible IN (-1,0,1) ORDER BY lastpost DESC
I have used phpmyadmin / status / monitor to watch when working well. As in this instance:

When same request, but not working well. Notice the gap in monitor timing.

Analyze, repair and optimise the database? Is there community discussion of those details? Previously

Memory usage in phpmyadmin monitor graphs is up to 3.4GB used after GoDaddy's tweaks, but still long delays.

I'm not expecting someone with a magic wand and a simple fix. But wanted to get other people's experience to point me in the right direction.

Thanks for any help in that regard.

for me I would prefer the top command screen, but it doesn't seem to be the problem on the server unless the debug appears high Page Generation Time.
I already had several problems with my VPS in terms of delays.
I always looked to see if the server was not in trouble. If the server has no problem and the problem still persisted, I contacted support from the network area and sent the traceroute screen from my PC to the server and from the server to my PC. And the problem was usually solved that way.

about analyze, repair and optimize command

repair database

mysqlcheck --all-databases  -p[password] -u [username] -r

optimize database

mysqlcheck --all-databases  -p[password] -u [username] -o

analyze database

mysqlcheck --all-databases  -p[password] -u [username] -a

How come yo load 100-150 images? Remember the attachment system loads the whole MyBB global.php file. it would be nice to _fix_ these 97ms queries but I doubt that has anything to do. Even so read the following, maybe you get creative and fine a solution:
https://community.mybb.com/thread-231400.html

The following might also be useful:
https://ougc.network/entry?tutorial-redu...rces-usage

Although you said it used to work before so it might not be related to the attachments.php file itself.

debug for text thread load

Page Generation Statistics
Page Generation Time: 	103 ms 	No. DB Queries: 	40
PHP Processing Time: 	84 ms (81.65%) 	DB Processing Time: 	19 ms (18.35%)
Extensions Used: 	mysqli, xml 	Global.php Processing Time: 	35 ms
PHP Version: 	7.4.16 	Server Load: 	0.05
GZip Encoding Status: 	Enabled 	No. Templates Used: 	209 (208 Cached / 1 Manually Loaded)
Memory Usage: 	6 MB (6291456 bytes) 	Memory Limit: 	256M

debug for picture thread load with browser cached images

Page Generation Statistics
Page Generation Time: 183 ms No. DB Queries: 35
PHP Processing Time: 38 ms (20.79%) DB Processing Time: 145 ms (79.21%)
Extensions Used: mysqli, xml Global.php Processing Time: 125 ms
PHP Version: 7.4.16 Server Load: 0.08
GZip Encoding Status: Enabled No. Templates Used: 209 (208 Cached / 1 Manually Loaded)
Memory Usage: 6 MB (6291456 bytes) Memory Limit: 256M

debug for same picture thread load with cleared browser cache, did not lag

Page Generation Statistics
Page Generation Time: 199 ms No. DB Queries: 35
PHP Processing Time: 41 ms (20.63%) DB Processing Time: 158 ms (79.37%)
Extensions Used: mysqli, xml Global.php Processing Time: 140 ms
PHP Version: 7.4.16 Server Load: 0.37
GZip Encoding Status: Enabled No. Templates Used: 209 (208 Cached / 1 Manually Loaded)
Memory Usage: 4 MB (4194304 bytes) Memory Limit: 256M

Apache Status

Current Time: Thursday, 15-Apr-2021 05:09:53 MST
Restart Time: Wednesday, 14-Apr-2021 00:56:42 MST
Parent Server Config. Generation: 2
Parent Server MPM Generation: 1
Server uptime: 1 day 4 hours 13 minutes 10 seconds
Server load: 0.08 0.10 0.07
Total accesses: 39096 - Total Traffic: 2.4 GB - Total Duration: 226522802
CPU Usage: u40.93 s19.39 cu50.11 cs20.32 - .129% CPU load
.385 requests/sec - 25.1 kB/second - 65.3 kB/request - 5794.01 ms/request
2 requests currently being processed, 98 idle workers

Slot PID Stopping Connections Threads Async connections
total accepting busy idle writing keep-alive closing
0 27928 no 0 yes 1 24 0 0 0
1 27929 no 0 yes 1 24 0 0 0
4 28069 no 1 yes 0 25 1 0 0
5 28070 no 0 yes 0 25 0 0 0
Sum 4 0 1 2 98 1 0 0

____________________L___________________W_________..............
....................................____________________________
______________________..........................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
............................................................

mysqlcheck shows all tables OK after
-c for check,
-a for analyze,
-o for optimize (Note: table does not support optimize, doing recreate + analyze instead. Status OK)
-r for repair (Note: the storage engine for the table does not support repair).

(2021-04-15, 06:22 AM)Omar G. Wrote: [ -> ]How come yo load 100-150 images? Remember the attachment system loads the whole MyBB global.php file.

Because that's how it was always done since 2013?
2016 is when I became involved. User base is mostly older folks, and you know how hard it is to get senior citizens to change. Big Grin

With each jump to a new server, I have successfully migrated the database using mysqldump to backup old, and mysql to import new. One thing that is new on this server is the primary database engine appears to be InnoDB. In this thread, https://community.mybb.com/thread-227431.html, Darth Apple says he increased his buffer_pool_size, although did not get specific. Here's a link that helps: https://scalegrid.io/blog/calculating-in...ql-server/ although there is a caveat: for Linux systems that are dedicated for MySQL

Continuing to evaluate.
Thanks for looking.

Note to add:
Just now, the server stalled and lagged on the picture thread. I let it finish loading then looked at the debug info.
Very similar to performance posted earlier.
If I was going to express it in noob terms, it seems there is some process or partial disconnect between mysql and apache that is the problem.

The only thing I can see wrong there is the "5794.01 ms/request" in the Apache Status output. That's almost 6 seconds per request if reading that correctly. The rest of the stats look fine. Also it wouldn't be a disconnect between MySQL and Apache, Apache doesn't call MySQL, the MySQL calls are initiated by PHP, but the MySQL and PHP generation times are fine. It looks like Apache is just taking ages to serve the response to the browser.

I think you'd probably get more advanced help asking somewhere like Server Fault as this sounds like an Apache setup issue more than anything.

Something else to try:

If you right click in Chrome, go to Network, and you'll see all network requests. Keep this panel open as you browse, and the next time it's super slow, click the request for the page load, and then click the Timing tab. See how long the "Waiting (TTFB)" and "Content Download" times are. If either of these are slow (especially TTFB), but the MyBB debug stats times are really quick like they are above, then it suggests the issue is Apache.

Question: If a page is loaded, then I click on Advanced Details link, that pulls the same page again, but just shows the details?
A difference in page display times is a reflection of the Apache problem?

And, Matt, thanks for the explanation, because it makes sense to me, considering all the parts of the puzzle.

And, Godaddy says there *might* be a problem with their internal network. :/
That's as far as they could go.

Answered my own question by clicking advanced Details 3 times in succession. Each one varies. Interestingly, the first is longer and the subsequent ones are much quicker because cache is queried.

Pages: 1 2