Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose an option to customise schema combination #313

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .changeset/violet-dryers-cross.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
"@t3-oss/env-nextjs": minor
"@t3-oss/env-core": minor
"@t3-oss/env-nuxt": minor
---

feat!: added ability to customise schema combination

Combination of schemas can now be customised using the `createFinalSchema` option. This allows further refinement or transformation of the environment variables.

This comes with a type-only breaking change:

- `CreateEnv` now has the signature `CreateEnv<TFinalSchema, TExtends>`, instead of the previous `CreateEnv<TServer, TClient, TShared, TExtends>`.
- Previous behaviour can be achieved by using `DefaultCombinedSchema<TServer, TClient, TShared>` as the type for `TFinalSchema`.
Comment on lines +13 to +14
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a user-breaking change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CreateEnv is exported, so technically yes - in practicality I doubt anyone was relying on it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's exported cause the other packages needs it. I should add some @internal jsdoc annotations to things

Binary file removed bun.lockb
Binary file not shown.
33 changes: 33 additions & 0 deletions docs/src/app/docs/customization/page.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -126,3 +126,36 @@ export const env = createEnv({
extends: [authEnv],
});
```

## Further refinement or transformation

You can use the `createFinalSchema` option to further refine or transform the environment variables.

```ts title="src/env.ts"
import { createEnv } from "@t3-oss/env-core";
import { z } from "zod";

export const env = createEnv({
server: {
SKIP_AUTH: z.boolean().optional(),
EMAIL: z.string().email().optional(),
PASSWORD: z.string().min(1).optional(),
},
// ...
createFinalSchema: (shape, isServer) =>
z.object(shape).transform((env, ctx) => {
if (env.SKIP_AUTH || !isServer) return { SKIP_AUTH: true } as const;
if (!env.EMAIL || !env.PASSWORD) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: "EMAIL and PASSWORD are required if SKIP_AUTH is false",
});
return z.NEVER;
}
return {
EMAIL: env.EMAIL,
PASSWORD: env.PASSWORD,
};
}),
});
```
66 changes: 55 additions & 11 deletions packages/core/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import type { StandardSchemaDictionary, StandardSchemaV1 } from "./standard";
import { parseWithDictionary } from "./standard";
import { ensureSynchronous, parseWithDictionary } from "./standard";

export type { StandardSchemaV1, StandardSchemaDictionary };

Expand All @@ -8,6 +8,13 @@ export type Simplify<T> = {
[P in keyof T]: T[P];
} & {};

type PossiblyUndefinedKeys<T> = {
[K in keyof T]: undefined extends T[K] ? K : never;
}[keyof T];

type UndefinedOptional<T> = Partial<Pick<T, PossiblyUndefinedKeys<T>>> &
Omit<T, PossiblyUndefinedKeys<T>>;

// biome-ignore lint/suspicious/noExplicitAny: <explanation>
type Impossible<T extends Record<string, any>> = Partial<
Record<keyof T, never>
Expand Down Expand Up @@ -173,6 +180,22 @@ export interface ServerOptions<
}>;
}

export interface CreateSchemaOptions<
TServer extends Record<string, StandardSchemaV1>,
TClient extends Record<string, StandardSchemaV1>,
TShared extends Record<string, StandardSchemaV1>,
TFinalSchema extends StandardSchemaV1<{}, {}>,
> {
/**
* A custom function to combine the schemas.
* Can be used to add further refinement or transformation.
*/
createFinalSchema?: (
shape: TServer & TClient & TShared,
isServer: boolean,
) => TFinalSchema;
}

export type ServerClientOptions<
TPrefix extends string | undefined,
TServer extends Record<string, StandardSchemaV1>,
Expand All @@ -188,28 +211,38 @@ export type EnvOptions<
TClient extends Record<string, StandardSchemaV1>,
TShared extends Record<string, StandardSchemaV1>,
TExtends extends Array<Record<string, unknown>>,
> =
TFinalSchema extends StandardSchemaV1<{}, {}>,
> = (
| (LooseOptions<TShared, TExtends> &
ServerClientOptions<TPrefix, TServer, TClient>)
| (StrictOptions<TPrefix, TServer, TClient, TShared, TExtends> &
ServerClientOptions<TPrefix, TServer, TClient>);
ServerClientOptions<TPrefix, TServer, TClient>)
) &
CreateSchemaOptions<TServer, TClient, TShared, TFinalSchema>;

type TPrefixFormat = string | undefined;
type TServerFormat = Record<string, StandardSchemaV1>;
type TClientFormat = Record<string, StandardSchemaV1>;
type TSharedFormat = Record<string, StandardSchemaV1>;
type TExtendsFormat = Array<Record<string, unknown>>;

export type CreateEnv<
export type DefaultCombinedSchema<
TServer extends TServerFormat,
TClient extends TClientFormat,
TShared extends TSharedFormat,
> = StandardSchemaV1<
{},
UndefinedOptional<
StandardSchemaDictionary.InferOutput<TServer & TClient & TShared>
>
>;

export type CreateEnv<
TFinalSchema extends StandardSchemaV1<{}, {}>,
TExtends extends TExtendsFormat,
> = Readonly<
Simplify<
StandardSchemaDictionary.InferOutput<TServer> &
StandardSchemaDictionary.InferOutput<TClient> &
StandardSchemaDictionary.InferOutput<TShared> &
StandardSchemaV1.InferOutput<TFinalSchema> &
UnReadonlyObject<Reduce<TExtends>>
>
>;
Expand All @@ -220,9 +253,14 @@ export function createEnv<
TClient extends TClientFormat = NonNullable<unknown>,
TShared extends TSharedFormat = NonNullable<unknown>,
const TExtends extends TExtendsFormat = [],
TFinalSchema extends StandardSchemaV1<{}, {}> = DefaultCombinedSchema<
TServer,
TClient,
TShared
>,
>(
opts: EnvOptions<TPrefix, TServer, TClient, TShared, TExtends>,
): CreateEnv<TServer, TClient, TShared, TExtends> {
opts: EnvOptions<TPrefix, TServer, TClient, TShared, TExtends, TFinalSchema>,
): CreateEnv<TFinalSchema, TExtends> {
const runtimeEnv = opts.runtimeEnvStrict ?? opts.runtimeEnv ?? process.env;

const emptyStringAsUndefined = opts.emptyStringAsUndefined ?? false;
Expand All @@ -244,7 +282,7 @@ export function createEnv<
const isServer =
opts.isServer ?? (typeof window === "undefined" || "Deno" in window);

const finalSchema = isServer
const finalSchemaShape = isServer
? {
..._server,
..._shared,
Expand All @@ -255,7 +293,13 @@ export function createEnv<
..._shared,
};

const parsed = parseWithDictionary(finalSchema, runtimeEnv);
const parsed =
opts
.createFinalSchema?.(finalSchemaShape as never, isServer)
["~standard"].validate(runtimeEnv) ??
parseWithDictionary(finalSchemaShape, runtimeEnv);

ensureSynchronous(parsed, "Validation must be synchronous");

const onValidationError =
opts.onValidationError ??
Expand Down
24 changes: 16 additions & 8 deletions packages/core/src/standard.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,21 +88,29 @@ export namespace StandardSchemaDictionary {
};
}

export function ensureSynchronous<T>(
value: T | Promise<T>,
message: string,
): asserts value is T {
if (value instanceof Promise) {
throw new Error(message);
}
}

export function parseWithDictionary<TDict extends StandardSchemaDictionary>(
dictionary: TDict,
value: Record<string, unknown>,
): StandardSchemaV1.Result<StandardSchemaDictionary.InferOutput<TDict>> {
const result: Record<string, unknown> = {};
const issues: StandardSchemaV1.Issue[] = [];
for (const key in dictionary) {
const schema = dictionary[key];
const prop = value[key];
const propResult = schema["~standard"].validate(prop);
if (propResult instanceof Promise) {
throw new Error(
`Validation must be synchronous, but ${key} returned a Promise.`,
);
}
const propResult = dictionary[key]["~standard"].validate(value[key]);

ensureSynchronous(
propResult,
`Validation must be synchronous, but ${key} returned a Promise.`,
);

if (propResult.issues) {
issues.push(
...propResult.issues.map((issue) => ({
Expand Down
112 changes: 112 additions & 0 deletions packages/core/test/smoke-valibot.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -629,3 +629,115 @@ describe("extending presets", () => {
});
});
});

describe("createFinalSchema", () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does this work with extends? can you add some tests for that

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can do - extends just assigns all of the properties to the final result from the schema, like it did before

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what I was thinking of was more like what if you do some combinator that accesses server stuff, will that cause an "invalid server access" or not? I'd think not since you're not giving the proxy as the argument there right? also what are the types in that callback like, server variables will be undefined when the combinator runs on the client etc etc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see #240, the whole Proxy thing doesn't really work properly with extends - but that's irrelevant to createFinalSchema, because the proxy isn't applied until the schema is finished validating.

As for types, you're correct - it's currently typed to what it'd be on the server, like the rest of the library (client + server + shared). I don't know how much it would complicate things to mark the server variables as optional.

Copy link
Contributor Author

@EskiMojo14 EskiMojo14 Jan 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

had a brainwave and opened #316 to fix #240

coming back to this - schemas have always been allowed to access any key they want, though that'll only include the ones that have already been checked with the current schema (not any extended ones), since the schema ditches unknown keys.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

schemas have always been allowed to access any key they want

No since accessing a server var on the client would throw an error - so e.g if you'd chain on a toLower() you'd get the invalid access before. It seems here we'd get a "Cannot access property toLower of undefined" which feels suboptimal - but maybe I'm overthinking it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

schemas have never been run on the output of presets, they were always run before it. they have always been passed the raw runtimeEnv option, and any presets were merged into the parsed result.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the concern, but I don't think there's a good way to expose type-wise that the server variables might not be defined to the createFinalSchema callback. Best case, the schema result also reflects the server variables as optional (whereas they're currently always defined), worst case the inference breaks completely. We also can't wrap the intermediate parsed value in a Proxy since we don't have access to it.

We could possibly pass some sort of makeSafe function to the user that would make a Proxy, but that would need to be opt in.

createObjectSchema: (shape, { isServer, makeSafe }) => z.object(shape).refine((_env) => {
  const env = makeSafe(_env) // wraps in Proxy
  if (!isServer) env.SERVER_ENV // throws
  // ...
})

test("custom schema combiner", () => {
let receivedIsServer = false;
const env = createEnv({
server: {
SERVER_ENV: v.string(),
},
shared: {
SHARED_ENV: v.string(),
},
clientPrefix: "CLIENT_",
client: {
CLIENT_ENV: v.string(),
},
runtimeEnv: {
SERVER_ENV: "server",
SHARED_ENV: "shared",
CLIENT_ENV: "client",
},
createFinalSchema: (shape, isServer) => {
expectTypeOf(isServer).toEqualTypeOf<boolean>();
if (typeof isServer === "boolean") receivedIsServer = true;
return v.object(shape);
},
});
expectTypeOf(env).toEqualTypeOf<
Readonly<{
SERVER_ENV: string;
SHARED_ENV: string;
CLIENT_ENV: string;
}>
>();
expect(env).toMatchObject({
SERVER_ENV: "server",
SHARED_ENV: "shared",
CLIENT_ENV: "client",
});
expect(receivedIsServer).toBe(true);
});
test("schema combiner with further refinement", () => {
const env = createEnv({
server: {
SKIP_AUTH: v.optional(v.boolean()),
EMAIL: v.optional(v.pipe(v.string(), v.email())),
PASSWORD: v.optional(v.pipe(v.string(), v.minLength(1))),
},
runtimeEnv: {
SKIP_AUTH: true,
},
createFinalSchema: (shape) =>
v.pipe(
v.object(shape),
v.check((env) => env.SKIP_AUTH || !!(env.EMAIL && env.PASSWORD)),
),
});
expectTypeOf(env).toEqualTypeOf<
Readonly<{
SKIP_AUTH?: boolean;
EMAIL?: string;
PASSWORD?: string;
}>
>();
expect(env).toMatchObject({ SKIP_AUTH: true });
});
test("schema combiner that changes the type", () => {
const env = createEnv({
server: {
SKIP_AUTH: v.optional(v.boolean()),
EMAIL: v.optional(v.pipe(v.string(), v.email())),
PASSWORD: v.optional(v.pipe(v.string(), v.minLength(1))),
},
runtimeEnv: {
SKIP_AUTH: true,
},
createFinalSchema: (shape) =>
v.pipe(
v.object(shape),
v.rawTransform(({ addIssue, dataset, NEVER }) => {
const env = dataset.value;
if (env.SKIP_AUTH) return { SKIP_AUTH: true } as const;
if (!env.EMAIL || !env.PASSWORD) {
addIssue({
message:
"EMAIL and PASSWORD are required if SKIP_AUTH is false",
});
return NEVER;
}
return {
EMAIL: env.EMAIL,
PASSWORD: env.PASSWORD,
};
}),
),
});
expectTypeOf(env).toEqualTypeOf<
Readonly<
| {
readonly SKIP_AUTH: true;
EMAIL?: undefined;
PASSWORD?: undefined;
}
| {
readonly SKIP_AUTH?: undefined;
EMAIL: string;
PASSWORD: string;
}
>
>();
expect(env).toMatchObject({ SKIP_AUTH: true });
});
});
Loading