Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copying node is slow #831

Open
elibarzilay opened this issue Feb 6, 2025 · 3 comments
Open

Copying node is slow #831

elibarzilay opened this issue Feb 6, 2025 · 3 comments

Comments

@elibarzilay
Copy link
Contributor

Copying a big directory is slow.
Using symlinks is much lighter, but obviously there's too many issues.

I have two suggestions:

First, cp -l creates a hard link, which is of course much faster, and the result is very close to a plain cp so there shouldn't be any issues. For example, a very lightweight change would be to set CP="cp -l" and use that instead of all cps. (CP_OPTIONS is better, but maybe that's too big of a change.)


Second, having a symlink based thing is nice not only in being fast, but also in keeping things more organized, since the downloaded stuff is kept in a single place. But of course adding that properly is expensive + risky so unlikely to be desirable.

A possible way to get that with minimal changes could be:

  • Add one single symlink to the current version, something like /usr/local/n/versions/node/current -> 22.13.1. That should be just a one-line thing to create/set it before copying. This effectively adds a proper place that remembers the current version directly rather than node --version.
  • Add some --no-copy flag (better: envvar (better: both)) to completely skip copying (so just the above symlink is changed). This is probably also a very small change.

This means that I can maintain my own bin/* symlinks to current/bin/*, and others if I want to.

Now, this looks like a weird and overly specific feature, but a good way to think about it is that n has functionality to (A) get some node version, and more functionality to (B) install it. This suggestion makes it possible to use (A) without (B). (IOW: use installl() without activate().)

This is very close to using --download, the only difference is that it's still doing the symlink change. A tiny change would be to rename $DOWNLOAD to a $MODE internally, so it can have three values for downloading only, or also doing the symlink, or also doing the whole thing.


Again, I can do PRs for both of these if it sounds reasonable. I'm guessing that they will be very small overall.

@shadowspawn
Copy link
Collaborator

shadowspawn commented Feb 6, 2025

cp -l is an interesting idea, but significantly changes the result. The cached and installed hierarchies would no longer be independent. I don't think many operations would modify the files, as opposed to overwriting, so might not notice in normal use.

Also, hard links do not work across partitions, and the cache destination and install destination may be on different partitions.


Now, this looks like a weird and overly specific feature

I appreciate that you recognise switching to symlinks would be major and unlikely, but adding a partial feature to make it possible to opt-in is... a bit weird and overly specific. 😆

For interest, my recollection is this is similar to how fnm is implemented. There is a current-ish folder that fnm modifies symlinks in.

@shadowspawn
Copy link
Collaborator

The two supported ways of avoiding or lowering the copy cost are:

  1. use exec or run or which to use a cached node without installing
  2. skip copying npm. This makes a big difference! This fits particularly nicely into workflows where you usually run latest npm anyway, or are using a different package manager.

@shadowspawn
Copy link
Collaborator

For my own interest I did a quick check, and --preserve makes less of a difference than I remembered. Significant improvement, but rest of the install does still take real time.

% time n 22.13.1
     copying : node/22.13.1
   installed : v22.13.1 (with npm 10.9.2)
n 22.13.1  0.18s user 1.09s system 45% cpu 2.797 total
% time n 22.13.1
     copying : node/22.13.1
   installed : v22.13.1 (with npm 10.9.2)
n 22.13.1  0.18s user 1.07s system 46% cpu 2.676 total

% time n --preserve 22.13.1 
     copying : node/22.13.1
   installed : v22.13.1
n --preserve 22.13.1  0.09s user 0.44s system 29% cpu 1.815 total
% time n --preserve 22.13.1
     copying : node/22.13.1
   installed : v22.13.1
n --preserve 22.13.1  0.09s user 0.44s system 28% cpu 1.853 total

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants