I have virtual hosting. I created the OS and mybb from the start and it had about 4GB when i created a backup of everything (mybb, database, OS...everything). I have been creating backups along the way and it has been increasing dramatically. Today it is almost at 10GB. Im sure a large chunk is plugins, but it seems like it increases regardless of not adding plugins.
attachment space used 4.3MB
no sql database backups stored (as i use host snapshots)
Im not sure on what is eating all the space and why it seems to increase 1GB every month or so.
What makes you think the forum is anything to do with it? The forum is a tiny, tiny part of a full server backup. If you're backing up the entire server, the space will almost certainly be things like Apache access logs if you haven't got log rotation set up.
I agree with Matt. I'm guessing your logs are taking up a lot of space.
if you have server access and you're running *nix, try using something like logrotate to handle log rotations.
I would highly doubt your entire forum, with plugins, is more than 50MB. The base MyBB install is a whopping 2.09MB zipped, and you're unlikely to push that above 10MB even once you add a custom theme. Your database could be taking up a fair bit of space, but only if you're getting a LOT of new users and posts (storing all that information).
As matt said, it's probably your access logs that are causing the issue - a new line every time someone opens a URI on your site adds up fast. Look at logrotation or prune them. Its also worth noting - do you really need to backup the ENTIRE OS monthly?
I changed rotate to lower
from 52 to 5
metulburr ~ $ cat /etc/logrotate.d/apache2
/var/log/apache2/*.log {
weekly
missingok
rotate 5
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
if /etc/init.d/apache2 status > /dev/null ; then \
/etc/init.d/apache2 reload > /dev/null; \
fi;
endscript
prerotate
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi; \
endscript
}
and removed the numerous *.log.gz that existed beforehand. What is here is within the past day or so
metulburr /var/log $ ls
alternatives.log boot.log dpkg.log lastlog mysql.log.1.gz
alternatives.log.1 bootstrap.log dpkg.log.1 mail.err syslog
apache2 btmp faillog mail.err.1 syslog.1
apport.log btmp.1 fontconfig.log mail.log syslog.2.gz
apport.log.1 cloud-init.log fsck mail.log.1 udev
apport.log.2.gz cloud-init-output.log installer msmtp.log unattended-upgrades
apt dist-upgrade kern.log mysql upstart
auth.log dmesg kern.log.1 mysql.err wtmp
auth.log.1 dmesg.0 landscape mysql.log wtmp.1
but even after that the storage used is 9.1GB and doesnt seem to clear any space
metulburr /var/log $ df -h
Filesystem Size Used Avail Use% Mounted on
udev 487M 4.0K 487M 1% /dev
tmpfs 100M 360K 100M 1% /run
/dev/disk/by-label/DOROOT 30G 9.1G 19G 33% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 498M 0 498M 0% /run/shm
none 100M 0 100M 0% /run/user
Quote:Its also worth noting - do you really need to backup the ENTIRE OS monthly?
and yes i would require an entire OS backup since our forum was hacked. In fact we have it on a cron job to make a snapshot every 6 hours. If we didnt backup the OS we would of had to restart from scratch a month ago. It is worth the cost of hourly snapshots to ensure the least amount of data loss or the least amount of downtime.
I don't think this is really the best place to ask, I can guarantee the issue is nothing to do with MyBB, you'd be better off asking on
http://serverfault.com/ or something.
From that it would be best to do:
du -h /
Then from there find the largest folder (which I'm certain would be /var and then /var/log) and perform a du command on that folder
du -h /var
But I'm fairly certain all of it will be in /var/log/httpd
You could even go as far as turning access logs off if you really don't need them
metulburr /var/log $ sudo du -a / | sort -n -r | head -n 45
sudo: unable to resolve host python-forum
du: cannot access ‘/proc/16427/task/16427/fd/4’: No such file or directory
du: cannot access ‘/proc/16427/task/16427/fdinfo/4’: No such file or directory
du: cannot access ‘/proc/16427/fd/4’: No such file or directory
du: cannot access ‘/proc/16427/fdinfo/4’: No such file or directory
7803800 /
2556936 /var
2097156 /swapfile
1850008 /usr
1396464 /var/www
1102820 /lib
943808 /lib/modules
727672 /var/www/html
616784 /usr/lib
606432 /var/www/html/pygame
598152 /var/www/html/pygame/ftp
553884 /usr/src
550624 /var/www/ftp
549000 /var/cache
433300 /var/cache/apt
406260 /usr/share
345140 /var/cache/apt/archives
322976 /var/lib
295088 /usr/lib/x86_64-linux-gnu
264120 /var/log
248892 /var/www/html/pygame/ftp/ftp.tar.gz
248892 /var/www/ftp/ftp.tar.gz
189076 /lib/modules/3.13.0-105-generic
189076 /lib/modules/3.13.0-103-generic
188940 /lib/modules/3.13.0-101-generic
188376 /lib/modules/3.13.0-61-generic
188336 /lib/modules/3.13.0-57-generic
185012 /lib/modules/3.13.0-105-generic/kernel
185012 /lib/modules/3.13.0-103-generic/kernel
184876 /lib/modules/3.13.0-101-generic/kernel
184328 /lib/modules/3.13.0-61-generic/kernel
184288 /lib/modules/3.13.0-57-generic/kernel
174360 /var/lib/apt
174288 /var/lib/apt/lists
163984 /var/log/apache2
159428 /usr/bin
153992 /boot
143848 /usr/lib/python2.7
140564 /lib/modules/3.13.0-105-generic/kernel/drivers
140564 /lib/modules/3.13.0-103-generic/kernel/drivers
140428 /lib/modules/3.13.0-101-generic/kernel/drivers
140364 /lib/modules/3.13.0-61-generic/kernel/drivers
140340 /lib/modules/3.13.0-57-generic/kernel/drivers
129168 /lib/firmware
118164 /var/www/metulburr.com
I can account for about 3GB of data, 2GB for swap, and 1GB of random data in /var/www that i found. /var/log only has 258MB, which i guess is large, but not the majority
$ sudo du -hs /var/log
258M /var/log
[quote]
You could even go as far as turning access logs off if you really don't need them
[/qoute]
Wouldnt logrotate handle the ability to give at least some log files instead of completely turn it off?
Indeed, logrotate would be fine.
/var/www has 1.3GB, 720MB of which is in /var/www/html, 600MB of which is /var/www/html/pygame
Once you account for your "3GB of data, 2GB for swap, and 1GB of random data" that's 6GB, your backup is 9GB but that includes all the system files, APT caches, configs etc...
I'd say that's pretty normal. Metulburr.com seems to be taking up about 110MB but unless "pygame" is the directory for your forum, your forum is assumably taking up less than that. Your database doesn't appear to be large as mysql's data dir doesnt appear there either.
Every time i make a snapshot every 6 hours it increases for example from 9.68GB to 9.7GB or something similar to that. Thats not a lot, but when you think thats only in .02GB every 6 hours getting added, it can quickly add up. I would assume that logs are the reason for this increasing like this? And the fact that i turned down the logs from 52 weeks to 5 weeks it will no longer happen....i believe. I guess i will have to keep checking and see if logrotate clears it at this point.
This server is serving 2 websites. /var/www/html/* is python-gaming.com (which is more just documents and installers....just static files sitting there) while /var/www/metulburr.com/* and its sym link of /var/www/python-forum.io/* are python-forum.io....the actual MYBB forum that gets fluctuating data. And there are some exe's, zips, and MSI installers, shared libraries .so in /var/www/html/pygame which is apart of the 1GB of random data i was counting.
Quote:Once you account for your "3GB of data, 2GB for swap, and 1GB of random data" that's 6GB
the 3GB of data i mentioned are the 2GB of swap and 1GB random data combined. Ubuntu 16.04 system i believe takes up 3GB of space by itself....maybe 4GB?. Plus the 2GB swap and 2GB of random stuff and the actual. MAybe another 100MB for MYBB and its plugins Thats 7.1GB. I cant account for why its 9.1GB and keeps counting up every snapshot.
But i wasnt sure exactly how big MYBB could get...with vanilla mybb, and maybe like 15-20 plugins, attachments, uploads, images, etc. But i guess the forum and its associated data is no larger than 116MB
metulburr ~ $ du -hs /var/www/metulburr.com/
du: cannot read directory ‘/var/www/metulburr.com/inc/plugins/ProStats/inc’: Permission denied
du: cannot read directory ‘/var/www/metulburr.com/inc/plugins/ProStats/images’: Permission denied
116M /var/www/metulburr.com/
metulburr ~ $ sudo !!
sudo du -hs /var/www/metulburr.com/
sudo: unable to resolve host python-forum
[sudo] password for metulburr:
116M /var/www/metulburr.com/
But i was more concerned on why it keeps increasing every 6 hours. Rather than a large static file that i know takes up space.