As I mentioned when I suggested to use mimalloc, this allocator in its version 2 has a drawback in that (if I describe it correctly) thread-local allocation regions are only cleaned up on subsequent allocations on the same thread, and a pattern of starting a new thread and having the original thread sleep for a long time can 'leak' memory during that time. It appears that people are running into this more often than one would think, here[1] is another one (it does not describe the problem in detail, though, so I can't tell for sure that this was the problem as I described it). But the interesting bit is that they moved to mimalloc v3 (they make it sound like it's a pre-release version but according to https://github.com/microsoft/mimalloc it seems stable) which fixed their leak. You may want to update to that at some point. Also, if running into more leak problems, they describe how to temporarily use jemalloc to use its tooling for memory usage visibility, you may want to check those out. Also, they mention that mimalloc doesn't work well on Windows (they do not use it there), you may want to keep that in mind too in case anyone needs to run SILO on Windows.
[1] https://blog.kerollmops.com/the-good-the-bad-and-the-leaky-jemalloc-bumpalo-and-mimalloc-in-meilisearch
As I mentioned when I suggested to use mimalloc, this allocator in its version 2 has a drawback in that (if I describe it correctly) thread-local allocation regions are only cleaned up on subsequent allocations on the same thread, and a pattern of starting a new thread and having the original thread sleep for a long time can 'leak' memory during that time. It appears that people are running into this more often than one would think, here[1] is another one (it does not describe the problem in detail, though, so I can't tell for sure that this was the problem as I described it). But the interesting bit is that they moved to mimalloc v3 (they make it sound like it's a pre-release version but according to https://github.com/microsoft/mimalloc it seems stable) which fixed their leak. You may want to update to that at some point. Also, if running into more leak problems, they describe how to temporarily use jemalloc to use its tooling for memory usage visibility, you may want to check those out. Also, they mention that mimalloc doesn't work well on Windows (they do not use it there), you may want to keep that in mind too in case anyone needs to run SILO on Windows.
[1] https://blog.kerollmops.com/the-good-the-bad-and-the-leaky-jemalloc-bumpalo-and-mimalloc-in-meilisearch