Levelized Files

[ssoelvsten]

Motivation

1. A lot of our files are of the temporary read-once-and-produce-output type (specifically arc_file and node_file with move-semantics). Hence, to be able to squeeze more BDDs / ZDDs into a smaller disk, we should continually deallocate the input file.
2. There is an upper limit to the file size for each file system, e.g. ext4 does not support files larger than 16 TiB.

Solution

The meta_file is turned into a list shared_ptr to files (or a file of files?). When a stream hooks into a read-once file, it removes the file for each level, when it has finished reading it.

[ssoelvsten]

Alternative: Incrementally Deallocate Temporary Files

A better alternative to solve motivation 1 and would also make everyone using TPIE love us is to delete the part of a file that already was read. Specifically, this was described by Mathias Rav in thomasmoelhave/tpie#227 .