Evebox events.sqlite growing large

[domiflichi]

My events.sqlite is at a little over 100GB now, which is quite a chunk of the machine's hard disk.

Is there a proper way of keeping this under control? I would like it to not get any larger than say 10GB. It's running on the same physical box that Suricata is running and I just learned about logrotate for Suricata so I've got that working and under control now. So now I'd like to get Evebox under control. Do I use logrotate for Evebox too? I have a feeling that won't work because I know that events.sqlite is not just a flat, log file, but an actual SQLite database so not sure how to properly handle this kind of thing.

Any examples would be awesome. Maybe something for the documentation too?

Jamie

[jasonish]

The best way is to enable the retention setting. This can be done with a minimal configuration file of:

database:
  retention-period: 7

This will do its best to delete events older than 7 days.

One note is that your database file may not shrink. It should stop growing tho.

I've been using SQLite much more in my own usage and the following changes will land in a release soon:

• Retention period of 7 days by default to prevent the situation you are in now. Can be configured tho.
• Some reports for SQLite
• Auto-vacuum on fresh databases. It can't be enabled on existing databases though (SQLite limitation). This is what allows a database to shrink
• Some better indexes for better performance.

[domiflichi]

Hi Jason, thanks for the info!

I created an:
/etc/evebox/evebox.yaml
file with the contents of what you suggested. Then I did a:

sudo systemctl stop evebox
sudo systemctl restart evebox

Is that all I need to do then?

On a similar note, I was wondering though - could I stop evebox, then delete the entire contents of my /etc/evebox directory:

total 109G
drwxr-xr-x  2 evebox evebox 4.0K Jan 30 05:18 .
drwxr-xr-x 74 root   root   4.0K Aug 31 13:09 ..
-rw-rw-rw-  1 evebox evebox   98 Feb 16 05:12 b264daf6271f51125d20d5a7715e8947.bookmark
-rw-rw-rw-  1 evebox evebox  40K Aug 24 12:51 config.sqlite
-rw-rw-rw-  1 evebox evebox  32K Feb 16 05:11 config.sqlite-shm
-rw-rw-rw-  1 evebox evebox    0 Jan 30 05:18 config.sqlite-wal
-rw-rw-rw-  1 evebox evebox 2.4K Aug  2  2022 EveboxQuickLog
-rw-rw-rw-  1 evebox evebox 107G Feb 16 05:12 events.sqlite
-rw-rw-rw-  1 evebox evebox  32K Feb 16 05:12 events.sqlite-shm
-rw-rw-rw-  1 evebox evebox 2.1G Feb 16 05:12 events.sqlite-wal

I don't care if I lose all of the old data. I don't mind starting over. But I don't know if it's safe to delete everything so that evebox starts over or if it would 'break' evebox.

Thank you!

[jasonish]

You'll want to make sure that EveBox is reading that configuration file by adding -c /etc/evebox/evebox.yaml to your unit file. It doesn't look like you are using the defaults if your data directory is /etc/evebox.

But anyways, yes. You can stop EveBox, delete events.sqlite* and restart just fine. Adding that retention period in should make it maintenance free moving forward though.

[domiflichi]

Oops, I misspoke about the location. All the sqlite files are actually in /var/lib/evebox. Given that correction, I don't have to do anything to my 'unit' file, right? Still a Linux newbie, I don't even know what a unit file is. But I have a guess - would that happen to be the /etc/default/evebox file?

Thank you for the tip on what to delete (events.sqlite*). I went ahead and did that, and now have my disk space back (and an empty alerts list, for now), so all is good.

Before I forget, I want to thank you so much for Evebox! I've experimented with Wazuh and Elasticsearch a bit, but they seem so complicated, bloated, and have no colors. I love the simplicity, speed, and usability of Evebox. So thank you for all you do Jason!

Have a great weekend!

[jasonish]

The unit file would be in /lib/systemd/system... Some people modify them, some don't. But it looks like you stuck with the default Debian package install which is ideal.

The sqlite mode has been a little neglected, even though we use it all the time in Suricata training. I'm working on making it more of a first class option, as I think it makes sense for small/home deployments.

Thanks for the kind words.