Skip to content

🐞 Bug run_command - Ruby IO.popen memory leak #336

Open
@Rikj000

Description

@Rikj000

Issue Description

There appears to be a memory leak in svn2git's run_command function,
I assume in IO.popen.

This memory leak leads to eating up all the RAM + Swap memory (128Gb + 64Gb on my system) during large conversions,
which will eventually crash the svn2git process once RAM is full.

In below btop screenshots you can see that:

  • This issue occurs while svn2git is running git svn fetch (which is called through run_command)
  • git svn fetch does not experience a memory leak, since it's memory usage stays low
  • svn2git does experience a memory leak, since it's memory usage keeps on growing
Click to Expand/Collapse the "Screenshots"

0h40min - 33.4Gb

33.4Gb RAM + 0Gb Swap
svn2git-0h-40min

1h01min - 51.6Gb

56.1Gb RAM + 0Gb Swap
svn2git-1h-01min

2h11min - 120Gb

56.1Gb RAM + 63.9Gb Swap
svn2git-2h-11min

3h00min - 159.6Gb

95.7Gb RAM + 63.9Gb Swap
svn2git-3h-00min
svn2git-git-svn-fetch-3h-02min

Work Around

  1. Either wait for svn2git to crash when out of memory,
    or manually force stop it with x2 CTRL + C
    (and also docker container ls + docker stop <container-id> when using Rikj000/Ruby-SVN2Git-Docker)
  2. Process continues upon restart, however:
    • In case of crash upon restart, fix by:
      • Open the .git/config file in the output Git repo
      • Delete all branches + tags entries within
      • Delete the .git/svn/refs/remotes/svn/trunk/index.lock file in the output Git repo, if it exists
    • Disk space cleanup before restart:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions