Skip to content

[rush-lib] Add override support for the PNPM virtual store directory #4987

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

[rush-lib] Add override support for the PNPM virtual store directory #4987

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions common/reviews/api/rush-lib.api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,7 @@ export class EnvironmentConfiguration {
static parseBooleanEnvironmentVariable(name: string, value: string | undefined): boolean | undefined;
static get pnpmStorePathOverride(): string | undefined;
static get pnpmVerifyStoreIntegrity(): boolean | undefined;
static get pnpmVirtualStorePathOverride(): string | undefined;
static reset(): void;
static get rushGlobalFolderOverride(): string | undefined;
static get rushTempFolderOverride(): string | undefined;
Expand All @@ -265,6 +266,7 @@ export const EnvironmentVariableNames: {
readonly RUSH_PARALLELISM: "RUSH_PARALLELISM";
readonly RUSH_ABSOLUTE_SYMLINKS: "RUSH_ABSOLUTE_SYMLINKS";
readonly RUSH_PNPM_STORE_PATH: "RUSH_PNPM_STORE_PATH";
readonly RUSH_PNPM_VIRTUAL_STORE_PATH: "RUSH_PNPM_VIRTUAL_STORE_PATH";
readonly RUSH_PNPM_VERIFY_STORE_INTEGRITY: "RUSH_PNPM_VERIFY_STORE_INTEGRITY";
readonly RUSH_DEPLOY_TARGET_FOLDER: "RUSH_DEPLOY_TARGET_FOLDER";
readonly RUSH_GLOBAL_FOLDER: "RUSH_GLOBAL_FOLDER";
Expand Down Expand Up @@ -1103,6 +1105,7 @@ export class PnpmOptionsConfiguration extends PackageManagerOptionsConfiguration
readonly pnpmLockfilePolicies: IPnpmLockfilePolicies | undefined;
readonly pnpmStore: PnpmStoreLocation;
readonly pnpmStorePath: string;
readonly pnpmVirtualStorePath: string;
readonly preventManualShrinkwrapChanges: boolean;
readonly resolutionMode: PnpmResolutionMode | undefined;
readonly strictPeerDependencies: boolean;
Expand Down
33 changes: 31 additions & 2 deletions libraries/rush-lib/src/api/EnvironmentConfiguration.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ export const EnvironmentVariableNames = {
* The default value is "common/temp" under the repository root.
*
* @remarks This environment variable is not compatible with workspace installs. If attempting
* to move the PNPM store path, see the `RUSH_PNPM_STORE_PATH` environment variable.
* to move the PNPM store or virtual store paths, see the `RUSH_PNPM_STORE_PATH` and
* `RUSH_PNPM_VIRTUAL_STORE_PATH` environment variables, respectively.
*/
RUSH_TEMP_FOLDER: 'RUSH_TEMP_FOLDER',

Expand Down Expand Up @@ -80,6 +81,15 @@ export const EnvironmentVariableNames = {
*/
RUSH_PNPM_STORE_PATH: 'RUSH_PNPM_STORE_PATH',

/**
* When using PNPM as the package manager, this variable can be used to configure the path that
* PNPM will use as the virtual store directory.
*
* If a relative path is used, then the virtual store path will be resolved relative to the process's
* current working directory. An absolute path is recommended.
*/
RUSH_PNPM_VIRTUAL_STORE_PATH: 'RUSH_PNPM_VIRTUAL_STORE_PATH',

/**
* When using PNPM as the package manager, this variable can be used to control whether or not PNPM
* validates the integrity of the PNPM store during installation. The value of this environment variable must be
Expand Down Expand Up @@ -240,6 +250,8 @@ export class EnvironmentConfiguration {

private static _pnpmStorePathOverride: string | undefined;

private static _pnpmVirtualStorePathOverride: string | undefined;

private static _pnpmVerifyStoreIntegrity: boolean | undefined;

private static _rushGlobalFolderOverride: string | undefined;
Expand Down Expand Up @@ -300,14 +312,23 @@ export class EnvironmentConfiguration {
}

/**
* An override for the PNPM store path, if `pnpmStore` configuration is set to 'path'
* An override for the PNPM store path.
* See {@link EnvironmentVariableNames.RUSH_PNPM_STORE_PATH}
*/
public static get pnpmStorePathOverride(): string | undefined {
EnvironmentConfiguration._ensureValidated();
return EnvironmentConfiguration._pnpmStorePathOverride;
}

/**
* An override for the PNPM virtual store path.
* See {@link EnvironmentVariableNames.RUSH_PNPM_VIRTUAL_STORE_PATH}
*/
public static get pnpmVirtualStorePathOverride(): string | undefined {
EnvironmentConfiguration._ensureValidated();
return EnvironmentConfiguration._pnpmVirtualStorePathOverride;
}

/**
* If specified, enables or disables integrity verification of the pnpm store during install.
* See {@link EnvironmentVariableNames.RUSH_PNPM_VERIFY_STORE_INTEGRITY}
Expand Down Expand Up @@ -476,6 +497,14 @@ export class EnvironmentConfiguration {
break;
}

case EnvironmentVariableNames.RUSH_PNPM_VIRTUAL_STORE_PATH: {
EnvironmentConfiguration._pnpmVirtualStorePathOverride =
value && !options.doNotNormalizePaths
? EnvironmentConfiguration._normalizeDeepestParentFolderPath(value) || value
: value;
break;
}

case EnvironmentVariableNames.RUSH_PNPM_VERIFY_STORE_INTEGRITY: {
EnvironmentConfiguration._pnpmVerifyStoreIntegrity =
value === '1' ? true : value === '0' ? false : undefined;
Expand Down
5 changes: 5 additions & 0 deletions libraries/rush-lib/src/api/LastInstallFlag.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ export interface ILastInstallFlagJson {
* Same with pnpmOptions.pnpmStorePath in rush.json
*/
storePath?: string;
/**
* Same with pnpmOptions.pnpmVirtualStorePath in rush.json
*/
virtualStorePath?: string;
/**
* An experimental flag used by cleanInstallAfterNpmrcChanges
*/
Expand Down Expand Up @@ -205,6 +209,7 @@ export function getCommonTempFlag(

if (currentState.packageManager === 'pnpm' && rushConfiguration.pnpmOptions) {
currentState.storePath = rushConfiguration.pnpmOptions.pnpmStorePath;
currentState.virtualStorePath = rushConfiguration.pnpmOptions.pnpmVirtualStorePath;
if (rushConfiguration.pnpmOptions.useWorkspaces) {
currentState.workspaces = rushConfiguration.pnpmOptions.useWorkspaces;
}
Expand Down
28 changes: 28 additions & 0 deletions libraries/rush-lib/src/api/test/EnvironmentConfiguration.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,4 +108,32 @@ describe(EnvironmentConfiguration.name, () => {
expect(EnvironmentConfiguration.pnpmStorePathOverride).toEqual(expectedValue);
});
});

describe('pnpmVirtualStorePathOverride', () => {
const ENV_VAR: string = 'RUSH_PNPM_VIRTUAL_STORE_PATH';

it('returns undefined for unset environment variable', () => {
EnvironmentConfiguration.validate();

expect(EnvironmentConfiguration.pnpmVirtualStorePathOverride).not.toBeDefined();
});

it('returns the expected path from environment variable without normalization', () => {
const expectedValue: string = '/var/temp';
process.env[ENV_VAR] = expectedValue;
EnvironmentConfiguration.validate({ doNotNormalizePaths: true });

expect(EnvironmentConfiguration.pnpmVirtualStorePathOverride).toEqual(expectedValue);
});

it('returns expected path from environment variable with normalization', () => {
const expectedValue: string = path.resolve(process.cwd(), 'temp');
const envVar: string = './temp';
process.env[ENV_VAR] = envVar;

EnvironmentConfiguration.validate();

expect(EnvironmentConfiguration.pnpmVirtualStorePathOverride).toEqual(expectedValue);
});
});
});
49 changes: 48 additions & 1 deletion libraries/rush-lib/src/api/test/RushConfiguration.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,11 @@ describe(RushConfiguration.name, () => {
rushConfiguration.pnpmOptions.pnpmStorePath,
'./repo/common/temp/pnpm-store'
);
assertPathProperty(
'pnpmVirtualStorePath',
rushConfiguration.pnpmOptions.pnpmVirtualStorePath,
'./repo/common/temp/.pnpm'
);
assertPathProperty(
'packageManagerToolFilename',
rushConfiguration.packageManagerToolFilename,
Expand Down Expand Up @@ -136,6 +141,11 @@ describe(RushConfiguration.name, () => {
rushConfiguration.pnpmOptions.pnpmStorePath,
'./repo/common/temp/pnpm-store'
);
assertPathProperty(
'pnpmVirtualStorePath',
rushConfiguration.pnpmOptions.pnpmVirtualStorePath,
'./repo/common/temp/.pnpm'
);
assertPathProperty(
'packageManagerToolFilename',
rushConfiguration.packageManagerToolFilename,
Expand Down Expand Up @@ -212,6 +222,11 @@ describe(RushConfiguration.name, () => {
rushConfiguration.pnpmOptions.pnpmStorePath,
path.join(expectedValue, 'pnpm-store')
);
assertPathProperty(
'pnpmVirtualStorePath',
rushConfiguration.pnpmOptions.pnpmVirtualStorePath,
path.join(expectedValue, '.pnpm')
);
assertPathProperty(
'packageManagerToolFilename',
rushConfiguration.packageManagerToolFilename,
Expand All @@ -227,15 +242,18 @@ describe(RushConfiguration.name, () => {
describe('PNPM Store Paths', () => {
afterEach(() => {
EnvironmentConfiguration['_pnpmStorePathOverride'] = undefined;
EnvironmentConfiguration['_pnpmVirtualStorePathOverride'] = undefined;
});

const PNPM_STORE_PATH_ENV: string = 'RUSH_PNPM_STORE_PATH';
const PNPM_VIRTUAL_STORE_PATH_ENV: string = 'RUSH_PNPM_VIRTUAL_STORE_PATH';

describe('Loading repo/rush-pnpm-local.json', () => {
const RUSH_JSON_FILENAME: string = path.resolve(__dirname, 'repo', 'rush-pnpm-local.json');

it(`loads the correct path when pnpmStore = "local"`, () => {
const EXPECT_STORE_PATH: string = path.resolve(__dirname, 'repo', 'common', 'temp', 'pnpm-store');
const EXPECT_VIRTUAL_STORE_PATH: string = path.resolve(__dirname, 'repo', 'common', 'temp', '.pnpm');
const rushConfiguration: RushConfiguration =
RushConfiguration.loadFromConfigurationFile(RUSH_JSON_FILENAME);

Expand All @@ -245,9 +263,13 @@ describe(RushConfiguration.name, () => {
Path.convertToSlashes(EXPECT_STORE_PATH)
);
expect(path.isAbsolute(rushConfiguration.pnpmOptions.pnpmStorePath)).toEqual(true);
expect(Path.convertToSlashes(rushConfiguration.pnpmOptions.pnpmVirtualStorePath)).toEqual(
Path.convertToSlashes(EXPECT_VIRTUAL_STORE_PATH)
);
expect(path.isAbsolute(rushConfiguration.pnpmOptions.pnpmVirtualStorePath)).toEqual(true);
});

it('loads the correct path when environment variable is defined', () => {
it('loads the correct store path when environment variable is defined', () => {
const EXPECT_STORE_PATH: string = path.resolve('/var/temp');
process.env[PNPM_STORE_PATH_ENV] = EXPECT_STORE_PATH;

Expand All @@ -259,6 +281,19 @@ describe(RushConfiguration.name, () => {
expect(rushConfiguration.pnpmOptions.pnpmStorePath).toEqual(EXPECT_STORE_PATH);
expect(path.isAbsolute(rushConfiguration.pnpmOptions.pnpmStorePath)).toEqual(true);
});

it('loads the correct virtual store path when environment variable is defined', () => {
const EXPECT_VIRTUAL_STORE_PATH: string = path.resolve('/var/temp');
process.env[PNPM_VIRTUAL_STORE_PATH_ENV] = EXPECT_VIRTUAL_STORE_PATH;

const rushConfiguration: RushConfiguration =
RushConfiguration.loadFromConfigurationFile(RUSH_JSON_FILENAME);

expect(rushConfiguration.packageManager).toEqual('pnpm');
expect(rushConfiguration.pnpmOptions.pnpmStore).toEqual('local');
expect(rushConfiguration.pnpmOptions.pnpmVirtualStorePath).toEqual(EXPECT_VIRTUAL_STORE_PATH);
expect(path.isAbsolute(rushConfiguration.pnpmOptions.pnpmVirtualStorePath)).toEqual(true);
});
});

describe('Loading repo/rush-pnpm-global.json', () => {
Expand All @@ -285,6 +320,18 @@ describe(RushConfiguration.name, () => {
expect(rushConfiguration.pnpmOptions.pnpmStore).toEqual('global');
expect(rushConfiguration.pnpmOptions.pnpmStorePath).toEqual(EXPECT_STORE_PATH);
});

it('loads the correct virtual store path when environment variable is defined', () => {
const EXPECT_STORE_PATH: string = path.resolve('/var/temp');
process.env[PNPM_VIRTUAL_STORE_PATH_ENV] = EXPECT_STORE_PATH;

const rushConfiguration: RushConfiguration =
RushConfiguration.loadFromConfigurationFile(RUSH_JSON_FILENAME);

expect(rushConfiguration.packageManager).toEqual('pnpm');
expect(rushConfiguration.pnpmOptions.pnpmStore).toEqual('global');
expect(rushConfiguration.pnpmOptions.pnpmVirtualStorePath).toEqual(EXPECT_STORE_PATH);
});
});

it(`throws an error when invalid pnpmStore is defined`, () => {
Expand Down
7 changes: 7 additions & 0 deletions libraries/rush-lib/src/cli/RushPnpmCommandLineParser.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -402,6 +402,13 @@ export class RushPnpmCommandLineParser {
pnpmEnvironmentMap.set('NPM_CONFIG_STATE_DIR', rushConfiguration.pnpmOptions.pnpmStorePath);
}

if (rushConfiguration.pnpmOptions.pnpmVirtualStorePath) {
pnpmEnvironmentMap.set(
'NPM_CONFIG_VIRTUAL_STORE_DIR',
rushConfiguration.pnpmOptions.pnpmVirtualStorePath
);
}

if (rushConfiguration.pnpmOptions.environmentVariables) {
for (const [envKey, { value: envValue, override }] of Object.entries(
rushConfiguration.pnpmOptions.environmentVariables
Expand Down
5 changes: 5 additions & 0 deletions libraries/rush-lib/src/logic/base/BaseInstallManager.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -854,6 +854,11 @@ ${gitLfsHookHandling}
}
}

// This will be overridden by RUSH_PNPM_VIRTUAL_STORE_PATH
if (EnvironmentConfiguration.pnpmVirtualStorePathOverride) {
args.push('--virtual-store-dir', this.rushConfiguration.pnpmOptions.pnpmVirtualStorePath);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a different approach than in RushPnpmCommandLineParser?

}

const { pnpmVerifyStoreIntegrity } = EnvironmentConfiguration;
if (pnpmVerifyStoreIntegrity !== undefined) {
args.push(`--verify-store-integrity`, `${pnpmVerifyStoreIntegrity}`);
Expand Down
3 changes: 2 additions & 1 deletion libraries/rush-lib/src/logic/installManager/WorkspaceInstallManager.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@ export class WorkspaceInstallManager extends BaseInstallManager {
if (EnvironmentConfiguration.rushTempFolderOverride !== undefined) {
throw new Error(
'The RUSH_TEMP_FOLDER environment variable is not compatible with workspace installs. If attempting ' +
'to move the PNPM store path, see the `RUSH_PNPM_STORE_PATH` environment variable.'
'to move the PNPM store path or virtual store path, see the `RUSH_PNPM_STORE_PATH` and ' +
'`RUSH_PNPM_VIRTUAL_STORE_PATH` environment variables, respectively.'
);
}

Expand Down
12 changes: 12 additions & 0 deletions libraries/rush-lib/src/logic/pnpm/PnpmOptionsConfiguration.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,13 @@ export class PnpmOptionsConfiguration extends PackageManagerOptionsConfiguration
*/
public readonly pnpmStorePath: string;

/**
* The path for PNPM to use as the virtual store directory.
*
* Will be overridden by environment variable RUSH_PNPM_VIRTUAL_STORE_PATH
*/
public readonly pnpmVirtualStorePath: string;

/**
* If true, then Rush will add the "--strict-peer-dependencies" option when invoking PNPM.
*
Expand Down Expand Up @@ -408,6 +415,11 @@ export class PnpmOptionsConfiguration extends PackageManagerOptionsConfiguration
} else {
this.pnpmStorePath = `${commonTempFolder}/pnpm-store`;
}
if (EnvironmentConfiguration.pnpmVirtualStorePathOverride) {
this.pnpmVirtualStorePath = EnvironmentConfiguration.pnpmVirtualStorePathOverride;
} else {
this.pnpmVirtualStorePath = `${commonTempFolder}/.pnpm`;
}
this.strictPeerDependencies = !!json.strictPeerDependencies;
this.preventManualShrinkwrapChanges = !!json.preventManualShrinkwrapChanges;
this.useWorkspaces = !!json.useWorkspaces;
Expand Down
2 changes: 1 addition & 1 deletion libraries/rush-lib/src/schemas/pnpm-config.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
},

"pnpmStore": {
"description": "Specifies the location of the PNPM store. There are two possible values:\n\n\"local\" - use the \"pnpm-store\" folder in the current configured temp folder: \"common/temp/pnpm-store\" by default.\n\"global\" - use PNPM's global store, which has the benefit of being shared across multiple repo folders, but the disadvantage of less isolation for builds (e.g. bugs or incompatibilities when two repos use different releases of PNPM)\n\nIn both cases, the store path can be overridden by the environment variable RUSH_PNPM_STORE_PATH.\n\nThe default value is \"local\".",
"description": "Specifies the location of the PNPM store. There are two possible values:\n\n\"local\" - use the \"pnpm-store\" folder in the current configured temp folder: \"common/temp/pnpm-store\" by default.\n\"global\" - use PNPM's global store, which has the benefit of being shared across multiple repo folders, but the disadvantage of less isolation for builds (e.g. bugs or incompatibilities when two repos use different releases of PNPM)\n\nIn both cases, the store path and the virtual store path can be overridden by the environment variables RUSH_PNPM_STORE_PATH and RUSH_PNPM_VIRTUAL_STORE_PATH, respectively.\n\nThe default value is \"local\".",
"type": "string",
"enum": ["local", "global"]
},
Expand Down
2 changes: 1 addition & 1 deletion libraries/rush-lib/src/schemas/rush.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@
"type": "object",
"properties": {
"pnpmStore": {
"description": "Specifies the location of the PNPM store. There are two possible values:\n\n\"local\" - use the \"pnpm-store\" folder in the current configured temp folder: \"common/temp/pnpm-store\" by default.\n\"global\" - use PNPM's global store, which has the benefit of being shared across multiple repo folders, but the disadvantage of less isolation for builds (e.g. bugs or incompatibilities when two repos use different releases of PNPM)\n\nIn all cases, the store path will be overridden by the environment variable RUSH_PNPM_STORE_PATH.\n\nThe default value is \"local\".",
"description": "Specifies the location of the PNPM store. There are two possible values:\n\n\"local\" - use the \"pnpm-store\" folder in the current configured temp folder: \"common/temp/pnpm-store\" by default.\n\"global\" - use PNPM's global store, which has the benefit of being shared across multiple repo folders, but the disadvantage of less isolation for builds (e.g. bugs or incompatibilities when two repos use different releases of PNPM)\n\nIn all cases, the store path and the virtual store path will be overridden by the environment variables RUSH_PNPM_STORE_PATH and RUSH_PNPM_VIRTUAL_STORE_PATH, respectively.\n\nThe default value is \"local\".",
"type": "string",
"enum": ["local", "global"]
},
Expand Down
Loading