Skip to content

Add enableAutoDedupe option - #344

Merged
arcanis merged 3 commits into
yarnpkg:mainfrom
latin-1:feat/enable-auto-dedupe
Oct 14, 2026
Merged

arcanis merged 3 commits into
yarnpkg:mainfrom
latin-1:feat/enable-auto-dedupe

Conversation

@latin-1

@latin-1 latin-1 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Disclosure: This PR uses Codex.

Automatically deduplicate dependencies with the highest strategy during explicit installs, before linking and building. Preserve lazy installs, focused installs, and explicit dedupe patterns.

Fixes #343

@netlify

netlify Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for yarn-v6 ready!

Name Link
🔨 Latest commit f789524
🔍 Latest deploy log https://app.netlify.com/projects/yarn-v6/deploys/6acf9260d8762770b6e134a2
😎 Deploy Preview https://deploy-preview-344--yarn-v6.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@latin-1

latin-1 commented Sep 25, 2026

Copy link
Copy Markdown
Contributor Author

@arcanis pnpm skips autoDedupe during frozen installs, so CI installs the locked versions even if duplicates exist. This implementation runs deduplication during immutable installs and fails if it would change the lockfile, effectively enforcing dedupe --check. Should we match pnpm and skip deduplication, or enforce a deduplicated lockfile on CI?

@arcanis

arcanis commented Sep 25, 2026

Copy link
Copy Markdown
Member

I believe we should run the dedupe as usual. Immutable installs guarantee that your colleagues won't get spurious changes in the lockfiles their PRs after you merge your work, skipping dedupe would break that promise.

@latin-1

latin-1 commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

Should we add --auto-dedupe and --no-auto-dedupe like pnpm? Since we don’t have --constraints and --no-constraints either, maybe it’s fine not to add them.

@latin-1
latin-1 force-pushed the feat/enable-auto-dedupe branch 3 times, most recently from 13d5d55 to 77a27c5 Compare October 6, 2026 21:53
Comment thread packages/zpm/src/commands/install.rs Outdated
// fast path so that a manually damaged project (say, a deleted
// package folder) heals when the user reaches for `yarn install`.
if !self.force
&& !project.config.settings.enable_auto_dedupe.value

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be an unfortunate limitation - I think if is_install_up_to_date returns true we should rather assume that auto-dedupe already ran, rather than always run the full install, no?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The design Codex originally suggested was to skip auto-dedupe in lazy mode. As a result, in that case, is_install_up_to_date doesn't necessarily mean that dedupe has already been performed.

I've updated the PR so that dedupe now runs normally in lazy mode as well. I'm not entirely sure which behavior is preferable, but given that pnpm appears to run dedupe during automatic installs too, I don't think this should be a major issue.

The only complication is that lazy installs with yarn dedupe need to be handled as a special case. It looks a little odd, but, well, there doesn't really seem to be a better way around it.

@latin-1
latin-1 force-pushed the feat/enable-auto-dedupe branch from 77a27c5 to 96eeaca Compare October 6, 2026 22:26
Automatically deduplicate dependencies with the highest strategy during
full-project installs, including lazy installs, before linking and
building. Preserve focused installs and explicit dedupe checks and patterns.
Skip automatic deduplication for temporary dlx installs. Keep the up-to-date
install fast path when no deduplication remains.

Fixes yarnpkg#343

Co-authored-by: Codex <noreply@openai.com>
@latin-1
latin-1 force-pushed the feat/enable-auto-dedupe branch from 96eeaca to a3cae15 Compare October 6, 2026 22:40
@latin-1

latin-1 commented Oct 6, 2026

Copy link
Copy Markdown
Contributor Author

Rebased. Like in #348, I also made yarn dlx skip dedupe.

@github-actions

Copy link
Copy Markdown

⏱️ Benchmark Results

gatsby install-full-cold

Metric Base Head Difference
Mean 4.715s 4.743s +0.60% ⚠️
Median 4.709s 4.738s +0.61% ⚠️
Min 4.565s 4.578s
Max 4.846s 4.884s
Std Dev 0.071s 0.076s
📊 Raw benchmark data (gatsby install-full-cold)

Base times: 4.798s, 4.709s, 4.769s, 4.755s, 4.661s, 4.709s, 4.710s, 4.715s, 4.679s, 4.680s, 4.787s, 4.700s, 4.767s, 4.846s, 4.759s, 4.802s, 4.820s, 4.752s, 4.754s, 4.707s, 4.799s, 4.677s, 4.708s, 4.565s, 4.717s, 4.628s, 4.670s, 4.571s, 4.618s, 4.618s

Head times: 4.752s, 4.811s, 4.881s, 4.720s, 4.770s, 4.818s, 4.861s, 4.784s, 4.722s, 4.745s, 4.749s, 4.663s, 4.689s, 4.631s, 4.654s, 4.676s, 4.671s, 4.578s, 4.764s, 4.728s, 4.658s, 4.732s, 4.691s, 4.724s, 4.884s, 4.691s, 4.847s, 4.785s, 4.822s, 4.802s


gatsby install-cache-only

Metric Base Head Difference
Mean 1.483s 1.468s -0.99% ✅
Median 1.474s 1.461s -0.93% ✅
Min 1.444s 1.429s
Max 1.699s 1.617s
Std Dev 0.046s 0.035s
📊 Raw benchmark data (gatsby install-cache-only)

Base times: 1.444s, 1.445s, 1.460s, 1.445s, 1.447s, 1.465s, 1.473s, 1.515s, 1.482s, 1.483s, 1.531s, 1.503s, 1.471s, 1.459s, 1.467s, 1.496s, 1.699s, 1.483s, 1.486s, 1.467s, 1.460s, 1.479s, 1.482s, 1.491s, 1.473s, 1.491s, 1.476s, 1.457s, 1.467s, 1.492s

Head times: 1.469s, 1.444s, 1.463s, 1.454s, 1.473s, 1.447s, 1.476s, 1.472s, 1.455s, 1.466s, 1.459s, 1.461s, 1.532s, 1.476s, 1.453s, 1.450s, 1.617s, 1.461s, 1.482s, 1.494s, 1.504s, 1.472s, 1.442s, 1.459s, 1.448s, 1.446s, 1.466s, 1.440s, 1.429s, 1.442s


gatsby install-cache-and-lock (warm, with lockfile)

Metric Base Head Difference
Mean 0.439s 0.438s -0.38% ✅
Median 0.440s 0.431s -2.01% ✅
Min 0.422s 0.414s
Max 0.466s 0.536s
Std Dev 0.011s 0.024s
📊 Raw benchmark data (gatsby install-cache-and-lock (warm, with lockfile))

Base times: 0.437s, 0.422s, 0.430s, 0.432s, 0.434s, 0.444s, 0.428s, 0.444s, 0.443s, 0.441s, 0.425s, 0.428s, 0.439s, 0.431s, 0.441s, 0.437s, 0.442s, 0.435s, 0.422s, 0.445s, 0.445s, 0.434s, 0.440s, 0.442s, 0.440s, 0.440s, 0.450s, 0.459s, 0.466s, 0.464s

Head times: 0.501s, 0.451s, 0.432s, 0.443s, 0.536s, 0.421s, 0.430s, 0.438s, 0.424s, 0.441s, 0.433s, 0.448s, 0.447s, 0.449s, 0.438s, 0.419s, 0.433s, 0.424s, 0.421s, 0.429s, 0.426s, 0.426s, 0.421s, 0.437s, 0.427s, 0.429s, 0.414s, 0.432s, 0.429s, 0.430s

@arcanis
arcanis merged commit 4222c3e into yarnpkg:main Oct 14, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an option to automatically dedupe packages after yarn install

2 participants