Try to avoid reboots when hugepages cannot be reserved

[idanyani]

The current implementation of Mosalloc requires that hugepages be reserved prior to allocating them in the pools. If the hugepages cannot be reserved -- because the memory is fragmented, for example -- the scripts fail and ask the user to reboot the machine. We suggest several mechanisms to avoid this inconvenience:

1. De-fragment the memory through khugepaged (as used by Linux Transparent Hugepages). This may not work because kernel objects are not movable if they are mapped in the direct map area. Clearing kernel objects like the dentry cache or inode cache may help.
2. It seems like Linux allows to defer the hugepage reservation to the allocation time. According to the docs:

/proc/sys/vm/nr_overcommit_hugepages specifies how large the pool of
huge pages can grow, if more huge pages than /proc/sys/vm/nr_hugepages are
requested by applications. Writing any non-zero value into this file
indicates that the hugetlb subsystem is allowed to try to obtain that
number of "surplus" huge pages from the kernel's normal page pool, when the
persistent huge page pool is exhausted. As these surplus huge pages become
unused, they are freed back to the kernel's normal page pool.

• Remember that we always need to reserve a little bit more than what the application needs because Mosalloc operates on the application and its child processes. If the application is called from a bash script (which is usually the case), then this bash process will allocate some of the previously reserved hugepages for itself. This allocation may be large if, for example, the Mosalloc pool begins with a 1GB hugepage. Note that the bash script won't allocate all reserved hugepages for itself because Mosalloc allocates the pools on-demand.