Skip to content

Commit 25e204a

Browse files
GeoffreyBoothGeoffreyBooth
committed
Initial commit
0 parents  commit 25e204a

File tree

7 files changed

+425
-0
lines changed

7 files changed

+425
-0
lines changed

‎CODE_OF_CONDUCT.md

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Code of Conduct
2+
3+
The [Node.js Code of Conduct][] applies to this team.
4+
5+
# Moderation Policy
6+
7+
The [Node.js Moderation Policy][] applies to this team.
8+
9+
[Node.js Code of Conduct]: https://github.com/nodejs/admin/blob/master/CODE_OF_CONDUCT.md
10+
[Node.js Moderation Policy]: https://github.com/nodejs/admin/blob/master/Moderation-Policy.md

‎GOVERNANCE.md

+174
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
# Loaders Team
2+
3+
For the current list of Team members, see the project
4+
[README.md](./README.md).
5+
6+
## Members
7+
8+
The [nodejs/loaders](https://github.com/nodejs/loaders) GitHub
9+
repository is maintained by the Team and additional Members who are
10+
added on an ongoing basis.
11+
12+
There are two types of Members:
13+
14+
### Active Members
15+
16+
* Invited to all meetings
17+
* Can participate in [consensus seeking process](#consensus-seeking-process)
18+
* Counted towards quorum in [Team Meetings](#team-meetings)
19+
* Participates in voting
20+
21+
### Observers
22+
23+
* Invited to all meetings
24+
* Can participate in [consensus seeking process](#consensus-seeking-process)
25+
* Not counted towards quorum in [Team Meetings](#team-meetings)
26+
* Cannot participate in voting
27+
28+
## Team Membership
29+
30+
Team Membership is not time-limited. There is no fixed size of the Team.
31+
32+
There is no specific set of requirements or qualifications for Team Membership beyond these rules.
33+
34+
Changes to Team membership should be proposed with an issue and labelled `loaders-agenda`
35+
to be included in the next [team meeting](#team-meetings). Decisions are made via the
36+
[Consensus Seeking Process](#consensus-seeking-process). If there are not objections in the
37+
issue new members may attend the meeting in which their membership is officially accepted.
38+
39+
If a Team member is unable to attend a meeting where a planned membership decision is being made,
40+
then their consent is assumed.
41+
42+
New Members to the team are initially accepted as Observers.
43+
44+
Observers can request to be made Active Members following the above process.
45+
46+
Active Members requesting to be made Observers following the above process are automatically approved
47+
by the Team and do not require consensus.
48+
49+
Any Member requesting to be removed from the group following the above process are automatically approved
50+
by the Team and do not require consensus.
51+
52+
## Team Meetings
53+
54+
The Team meets bi-weekly on Zoom.us. A designated moderator
55+
approved by the Team runs the meeting. Each meeting should be
56+
published to YouTube.
57+
58+
Items are added to the Team agenda that are considered contentious or
59+
are modifications of governance, contribution policy, Team membership,
60+
or release process.
61+
62+
The intention of the agenda is not to approve or review all patches;
63+
that should happen continuously on GitHub and be handled by the larger
64+
group of Collaborators.
65+
66+
Any community member or contributor can ask that something be added to
67+
the next meeting's agenda by logging a GitHub Issue. Any Collaborator,
68+
Team member or the moderator can add the item to the agenda by adding
69+
the ***loaders-agenda*** tag to the issue.
70+
71+
Prior to each Team meeting the moderator will share the agenda with
72+
members of the Team. Team members can add any items they like to the
73+
agenda at the beginning of each meeting. The moderator and the Team
74+
cannot veto or remove items.
75+
76+
The Team may invite persons or representatives from certain projects to
77+
participate in a non-voting capacity.
78+
79+
Decisions in meetings are made via the [Consensus Seeking Process](#Consensus-Seeking-Process)
80+
and require a quorum of 50% of Active Members.
81+
82+
The moderator is responsible for summarizing the discussion of each
83+
agenda item and sends it as a pull request after the meeting.
84+
85+
## Consensus Seeking Process
86+
87+
The Team follows a
88+
[Consensus Seeking](http://en.wikipedia.org/wiki/Consensus-seeking_decision-making)
89+
decision-making model.
90+
91+
When an agenda item has appeared to reach a consensus, the moderator
92+
will ask "Does anyone object?" as a final call for dissent from the
93+
consensus.
94+
95+
If an agenda item cannot reach a consensus, a Team member can call for
96+
the item to be decided by a vote or to table the issue to the next
97+
meeting. In both cases the decision must be seconded by a majority of the Team
98+
or else the discussion will continue. Simple majority wins. Only Active
99+
Members participate in a vote.
100+
101+
## Pull Requests
102+
103+
This section details expectations for team members involved in any pull
104+
requests that relate to the group's [scope][loaders-team-purpose] and
105+
[implementation work][loaders-team-plan].
106+
107+
The following expectations apply to all team members involved in a related pull
108+
requests in [this Repository][nodejs-loaders]. This section does
109+
not apply to [the Node.js core repository](https://github.com/nodejs/node).
110+
111+
These expectations are intended to ensure that all concurrent efforts align
112+
with the overall direction of the group for delivering a cohesive and predictable
113+
user experience for ECMAScript, CommonJS, and other loaders supported by Node.js.
114+
115+
Pull requests not included under the _special exemptions_ section below must
116+
reach consensus in a meeting in order to be merged into this repository. A pull
117+
request that is is unable to reach consensus cannot be merged into this
118+
repository.
119+
120+
### Special Exemptions to the PR landing process
121+
122+
Special exception is made for pull requests seeking to make any of the following
123+
changes to this repository:
124+
125+
- Errata fixes.
126+
- Editorial changes.
127+
- Meeting minutes.
128+
- Updates to the team lists via the `ncu-sync` tool.
129+
- Doc Fixes
130+
- Tests
131+
- Fixing Conflicts with a rebase
132+
133+
These pull requests may be merged without being presented at a meeting if a
134+
reasonable time is given for review and there no dissent. The time period seen
135+
as reasonable for review varies on a case by case basis as determined by the
136+
author. A member may request a specific time period for review of such a pull
137+
request not to exceed the next meeting date. If a time for review is requested,
138+
members must wait for that time period to pass or review be completed prior to
139+
that time.
140+
141+
<a id="developers-certificate-of-origin"></a>
142+
## Developer's Certificate of Origin 1.1
143+
144+
By making a contribution to this project, I certify that:
145+
146+
* (a) The contribution was created in whole or in part by me and I
147+
have the right to submit it under the open source license
148+
indicated in the file; or
149+
150+
* (b) The contribution is based upon previous work that, to the best
151+
of my knowledge, is covered under an appropriate open source
152+
license and I have the right under that license to submit that
153+
work with modifications, whether created in whole or in part
154+
by me, under the same open source license (unless I am
155+
permitted to submit under a different license), as indicated
156+
in the file; or
157+
158+
* (c) The contribution was provided directly to me by some other
159+
person who certified (a), (b) or (c) and I have not modified
160+
it.
161+
162+
* (d) I understand and agree that this project and the contribution
163+
are public and that a record of the contribution (including all
164+
personal information I submit with it, including my sign-off) is
165+
maintained indefinitely and may be redistributed consistent with
166+
this project or the open source license(s) involved.
167+
168+
169+
<!-- Links -->
170+
171+
[nodejs-loaders]: https://github.com/nodejs/loaders
172+
[nodejs-core]: https://github.com/nodejs/node
173+
[loaders-team-purpose]: ./README.md#purpose
174+
[loaders-team-plan]: ./doc/design.md

‎LICENSE.md

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
MIT License (MIT)
2+
3+
Copyright (c) Node.js contributors. All rights reserved.
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
6+
7+
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
8+
9+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE
10+

‎README.md

+39
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Loaders Team
2+
3+
## Purpose
4+
5+
The Node.js Loaders Team maintains and actively develops the ECMAScript Modules Loaders implementation in Node.js core.
6+
7+
## History
8+
9+
This team is spun off from the [Modules team](https://github.com/nodejs/modules). We aim to implement the [use cases](https://github.com/nodejs/modules/blob/main/doc/use-cases.md) that went unfulfilled by the initial ES modules implementation that can be achieved via loaders.
10+
11+
## Project
12+
13+
- [Resources](doc/resources.md)
14+
15+
- [Use cases](./doc/use-cases.md)
16+
17+
- [Design](./doc/design.md)
18+
19+
- [Project board](https://github.com/nodejs/node/projects/17)
20+
21+
- [Meeting minutes](./doc/meetings)
22+
23+
## Status
24+
25+
### In Progress
26+
27+
1. Finish https://github.com/nodejs/node/pull/37468 / https://github.com/nodejs/node/pull/35524, simplifying the hooks to `resolve`, `load` and `globalPreloadCode`.
28+
29+
### Upcoming
30+
31+
1. Refactor the internal Node ESM loader’s hooks into `resolve` and `load`. Node’s internal loader already has no-ops for `transformSource` and `getGlobalPreloadCode`, so all this really entails is merging the internal `getFormat` and `getSource` into one function `load`.
32+
33+
1. Refactor Node’s internal ESM loader to move its exception on unknown file types from within `resolve` (on detection of unknown extensions) to within `load` (if the resolved extension has no defined translator).
34+
35+
1. Implement chaining as described in the [design](doc/design.md), where the `default<hookName>` becomes `next` and references the next registered hook in the chain.
36+
37+
1. Get a `load` return value of `format: 'commonjs'` to work, or at least error informatively. See https://github.com/nodejs/node/issues/34753#issuecomment-735921348.
38+
39+
After this, we should get user feedback regarding the developer experience; for example, is too much boilerplate required? Should we have a separate `transform` hook? And so on. We should also investigate and potentially implement the [technical improvements](doc/use-cases.md#improvements) on our to-do list, such as the loaders-application communication channel and moving loaders into a thread.

‎doc/design.md

+140
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# Loaders Design
2+
3+
## Hooks
4+
5+
The [current five loader hooks](https://nodejs.org/api/esm.html#esm_loaders) (`resolve`, `getFormat`, `getSource`, `transformSource`, `getGlobalPreloadCode`) would be reduced to three:
6+
7+
- `resolve`: Take a specifier (the string after `from` in an `import` statement) and convert it into an URL to be loaded.
8+
9+
- `load`: Take the resolved URL and return runnable code (JavaScript, Wasm, etc.) as well as the name of one of Node’s ESM loader’s [“translators”](https://github.com/nodejs/node/blob/master/lib/internal/modules/esm/translators.js): `commonjs`, `module`, `builtin` (a Node internal module like `fs`), `json` (with `--experimental-json-modules`) or `wasm` (with `--experimental-wasm-modules`).
10+
11+
- `globalPreloadCode`: Define a string of JavaScript to be injected into the application global scope. This is more or less the current `getGlobalPreloadCode` hook.
12+
13+
## Chaining `resolve` hooks
14+
15+
Say you had a chain of three loaders, `unpkg`, `http-to-https`, `cache-buster`:
16+
17+
1. The `unpkg` loader resolves a specifier `foo` to an URL `http://unpkg.com/foo`.
18+
19+
2. The `http-to-https` loader rewrites that URL to `https://unpkg.com/foo`.
20+
21+
3. The `cache-buster` that takes the URL and adds a timestamp to the end, so like `https://unpkg.com/foo?ts=1234567890`.
22+
23+
In the new loaders design, these three loaders could be implemented as follows:
24+
25+
### `unpkg` loader
26+
27+
```js
28+
export async function resolve(specifier, context, next) { // next is Node’s resolve
29+
if (isBareSpecifier(specifier)) {
30+
return `http://unpkg.com/${specifier}`;
31+
}
32+
return next(specifier, context);
33+
}
34+
```
35+
36+
### `http-to-https` loader
37+
38+
```js
39+
export async function resolve(specifier, context, next) { // next is the unpkg loader’s resolve
40+
const result = await next(specifier, context);
41+
if (result.url.startsWith('http://')) {
42+
result.url = `https${result.url.slice('http'.length)}`;
43+
}
44+
return result;
45+
}
46+
```
47+
48+
### `cache-buster` loader
49+
50+
```js
51+
export async function resolve(specifier, context, next) { // next is the http-to-https loader’s resolve
52+
const result = await next(specifier, context);
53+
if (supportsQueryString(result.url)) { // exclude data: & friends
54+
// TODO: do this properly in case the URL already has a query string
55+
result.url += `?ts=${Date.now()}`;
56+
}
57+
return result;
58+
}
59+
```
60+
61+
The hook functions nest: each one always just returns a string, like Node’s `resolve`, and the chaining happens as a result of calling `next`; and if a hook doesn’t call `next`, the chain short-circuits. The API would be `node --loader unpkg --loader http-to-https --loader cache-buster`, following the pattern set by `--require`.
62+
63+
## Chaining `load` hooks
64+
65+
Chaining `load` hooks would be similar to chaining `resolve` hooks, though slightly more complicated in that instead of returning a single string, each `load` hook returns an object `{ format, source }` where `source` is the loaded module’s source code/contents and `format` is the name of one of Node’s ESM loader’s [“translators”](https://github.com/nodejs/node/blob/master/lib/internal/modules/esm/translators.js): `commonjs`, `module`, `builtin` (a Node internal module like `fs`), `json` (with `--experimental-json-modules`) or `wasm` (with `--experimental-wasm-modules`).
66+
67+
Currently, Node’s internal ESM loader throws an error on unknown file types: `import('file.javascript')` throws, even if the contents of `file.javascript` are perfectly acceptable JavaScript. This error happens during Node’s internal `resolve` when it encounters a file extension it doesn’t recognize; hence the current [CoffeeScript loader example](https://nodejs.org/api/esm.html#esm_transpiler_loader) has lots of code to tell Node to allow CoffeeScript file extensions. We should move this validation check to be after the format is determined, which is one of the return values of `load`; so basically, it’s the responsibility of `load` to return a `format` that Node recognizes. Node’s internal `load` doesn’t know to resolve a URL ending in `.coffee` to `module`, so Node would continue to error like it does now; but the CoffeeScript loader under this new design no longer needs to hook into `resolve` at all, since it can determine the format of CoffeeScript files within `load`. In code:
68+
69+
### `coffeescript` loader
70+
71+
```js
72+
import CoffeeScript from 'coffeescript';
73+
74+
// CoffeeScript files end in .coffee, .litcoffee or .coffee.md
75+
const extensionsRegex = /\.coffee$|\.litcoffee$|\.coffee\.md$/;
76+
77+
export async function load(url, context, next) {
78+
const result = await next(url, context);
79+
80+
// The first check is technically not needed but ensures that
81+
// we don’t try to compile things that already _are_ compiled.
82+
if (result.format === undefined && extensionsRegex.test(url)) {
83+
// For simplicity, all CoffeeScript URLs are ES modules.
84+
const format = 'module';
85+
const source = CoffeeScript.compile(result.source, { bare: true });
86+
return {format, source};
87+
}
88+
return result;
89+
}
90+
```
91+
92+
And the other example loader in the docs, to allow `import` of `https://` URLs, would similarly only need a `load` hook:
93+
94+
### `https` loader
95+
96+
```js
97+
import { get } from 'https';
98+
99+
export async function load(url, context, next) {
100+
if (url.startsWith('https://')) {
101+
let format; // default: format is undefined
102+
const source = await new Promise((resolve, reject) => {
103+
get(url, (res) => {
104+
// Determine the format from the MIME type of the response
105+
switch (res.headers['content-type']) {
106+
case 'application/javascript':
107+
case 'text/javascript': // etc.
108+
format = 'module';
109+
break;
110+
case 'application/node':
111+
case 'application/vnd.node.node':
112+
format = 'commonjs';
113+
break;
114+
case 'application/json':
115+
format = 'json';
116+
break;
117+
// etc.
118+
}
119+
120+
let data = '';
121+
res.on('data', (chunk) => data += chunk);
122+
res.on('end', () => resolve({ source: data }));
123+
}).on('error', (err) => reject(err));
124+
});
125+
return {format, source};
126+
}
127+
128+
return next(url, context);
129+
}
130+
```
131+
132+
If these two loaders are used together, where the `coffeescript` loader’s `next` is the `https` loader’s hook and `https` loader’s `next` is Node’s native hook, then for a URL like `https://example.com/module.coffee`:
133+
134+
1. The `https` loader would load the source over the network, but return `format: undefined`, assuming the server supplied a correct `Content-Type` header like `application/vnd.coffeescript` which our `https` loader doesn’t recognize.
135+
136+
2. The `coffeescript` loader would get that `{ source, format: undefined }` early on from its call to `next`, and set `format: 'module'` based on the `.coffee` at the end of the URL. It would also transpile the source into JavaScript. It then returns `{ format: 'module', source }` where `source` is runnable JavaScript rather than the original CoffeeScript.
137+
138+
## Chaining `globalPreloadCode` hooks
139+
140+
For now, we think that this wouldn’t be chained the way `resolve` and `load` would be. This hook would just be called sequentially for each registered loader, in the same order as the loaders themselves are registered. If this is insufficient, for example for instrumentation use cases, we can discuss and potentially change this to follow the chaining style of `load`.

0 commit comments

Comments
 (0)