How to See What Files Take HDD Space
Question: We need to see what files are taking up a lot of disk space in Web Safety appliance. How to check that from the command line?
Answer: First check how much free space is available on disk by running the following command df. The output will look something like this.
root@node12:~# df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 1988468 0 1988468 0% /dev
tmpfs 403976 1136 402840 1% /run
/dev/sda2 102684600 15401168 82024324 16% /
tmpfs 2019880 2060 2017820 1% /dev/shm
tmpfs 5120 0 5120 0% /run/lock
tmpfs 2019880 0 2019880 0% /sys/fs/cgroup
/dev/loop0 93568 93568 0 100% /snap/core/8689
/dev/loop2 96128 96128 0 100% /snap/core/8935
tmpfs 403976 0 403976 0% /run/user/0
We see the / partion is used for 16%. Now let's find out the top directories that use most disk space by running the following command du -hs * | sort -h | tail -n 7. It will show top 7 directories at / folder that occupies most space.
root@node12:/# du -hs * | sort -h | tail -n 7
209M boot
540M snap
1.1G lib
1.5G usr
2.0G var
3.9G swap.img
6.0G opt
It is now obvious that /opt directory where Web Safety is installed used most space. Let us now see what files in that folder exactly are responsible for large disk usage by running the following command. Please be patient as it takes some time to calculate disk usage.
The output will show that folders /opt/websafety-ui/var/stats and /opt/websafety-ui/var/reports take most space. These folders are used in reporting and it might mean the amount of information that reporting generates is too much for your deployment so you might need to limit the log retention settings or ignore some filtering modules in statistics as explained in this article.
5525600 ./websafety-ui
5524008 ./websafety-ui/var
2700996 ./websafety-ui/var/reports
1400880 ./websafety-ui/var/stats.temp
1390404 ./websafety-ui/var/stats
704432 ./websafety
678144 ./websafety-ui/var/reports/2019-11-26
642904 ./websafety/var
563128 ./websafety-ui/var/reports/2019-11-27
Note the file system can be filled not by large files but by a lot of small files too. In this case there will be free space available on the system but the amount of used inodes will be close to 100%. This can be checked by running the df -i command. The output will typically look like the following. Here we see the inode usage of the root partition is 10% only so it is fine.
root@node12:~# df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
udev 497111 405 496706 1% /dev
tmpfs 504970 667 504303 1% /run
/dev/sda2 3276800 302488 2974312 10% /
tmpfs 504970 5 504965 1% /dev/shm
tmpfs 504970 4 504966 1% /run/lock
tmpfs 504970 18 504952 1% /sys/fs/cgroup
/dev/loop0 12827 12827 0 100% /snap/core/7713
/dev/loop1 12847 12847 0 100% /snap/core/8689
tmpfs 504970 11 504959 1% /run/user/0