Collecting Statistics

As explained in the previous article, once per day using the cron entry file in /etc/cron.daily/websafety_report the system runs the following statistics generation program.

# the collector
/opt/websafety-ui/bin/collectord \
    --product=ws \
    --action=history \
    --config=/opt/websafety-ui/etc/traffic/collector.json \
        >> "$LOGFILE" 2>&1

This program parses each entry in the Squid access log files and generates the daily statistics as SQLite database files in /opt/websafety-ui/var/stats folder.

Sample SQLite file storage is shown below.

root@node12:~# ll /opt/websafety-ui/var/stats/

-rw-r--r-- 1 websafety websafety    221184 Jan  5 06:29 2023-12-30.sqlite
-rw-r--r-- 1 websafety websafety   4423680 Jan  5 06:29 2023-12-31.sqlite
-rw-r--r-- 1 websafety websafety   3657728 Jan  5 06:29 2024-01-01.sqlite
-rw-r--r-- 1 websafety websafety   7626752 Jan  5 06:29 2024-01-02.sqlite
-rw-r--r-- 1 websafety websafety   4153344 Jan  5 06:29 2024-01-03.sqlite
-rw-r--r-- 1 websafety websafety   5378048 Jan  5 08:20 2024-01-04.sqlite
-rw-r--r-- 1 websafety websafety     49152 Jan  5 06:27 2024-01-05.sqlite
-rw-r--r-- 1 websafety websafety     19713 Jan  5 06:27 history.state
-rw-r--r-- 1 websafety websafety     73728 Jan  5 09:52 realtime.sqlite