Does Log4j2 compress rolled log files concurrently with writing to the new log file?

[stoyanpetrov1920-spec]

In Log4j 2, when using a DirectWriteRolloverStrategy with compression enabled (e.g., filePattern="app-%d{yyyy-MM-dd}.log.xz"), does the framework perform compression of the rolled file and writing to the new active log file concurrently?

In other words, after a rollover trigger (time or size-based), does Log4j start writing new log events to the fresh file immediately while compressing the old file in a background thread — or does the compression need to finish before logging resumes?

I couldn’t find a clear statement about this behavior in the official Log4j 2 documentation or API reference. Any pointers or clarification on whether compression and new file writes happen in parallel would be appreciated.

The reason I’m asking is that our team is evaluating switching from gzip to xz compression for rolled files. However, xz compression time for ~100 MB logs is quite high. If rollover and writing to the new file occur sequentially, that delay might block logging and impact performance, so we may need to consider other strategies.

[ppkarwasz]

Hi @stoyanpetrov1920-spec,

Yes, the compression of the rolled log file is performed asynchronously and does not block the creation of the new active log file. Logging continues immediately after the rollover as long as the compression finishes before the next rollover occurs.

In Log4j Core, the synchronous (blocking) part of a rollover is intentionally minimized and normally includes only the operations required to free up the current log file name (such as renaming or moving the active file) so a new file can be opened. However, with DirectWriteRolloverStrategy, this step is effectively eliminated because each rolled file is written directly with a unique name. As a result, no file move or rename is needed during rollover, and the synchronous phase is essentially empty.

Note

If you are rolling by size as well as by time, make sure your filePattern includes %i, for example:

filePattern="app-%d{yyyy-MM-dd}-%i.log.xz"

Without %i, multiple size-based rollovers within the same day would try to use the same file name, leading to collisions or overwritten logs.

Because compression happens asynchronously, it will not add latency to your logging calls. However, it will still consume some system resources (CPU and disk I/O) in the background. If you are concerned about that, or if you want to avoid large compression jobs at rollover time, you might consider appenders that compress data incrementally per event rather than at rollover time, for example, the Y-Scope CLP appenders.