|
| 1 | +########################################################################## |
| 2 | +Compile TypeScript before publishing and adopt tgz-based local development |
| 3 | +########################################################################## |
| 4 | + |
| 5 | +Summary |
| 6 | +======= |
| 7 | + |
| 8 | +We compile ``@openedx/frontend-base`` and ``@openedx/frontend-app-*`` packages |
| 9 | +to JavaScript (with ``.d.ts`` declarations) before publishing to npm, and adopt |
| 10 | +a tarball-based (``npm pack``) workflow as the official mechanism for local |
| 11 | +development of said packages. As part of this, we remove support for |
| 12 | +``module.config.js``-based local aliases. |
| 13 | + |
| 14 | +Context |
| 15 | +======= |
| 16 | + |
| 17 | +The need for TypeScript compilation |
| 18 | +------------------------------------ |
| 19 | + |
| 20 | +Previously, ``@openedx/frontend-base`` shipped raw TypeScript source and relied |
| 21 | +on consumers' bundlers (webpack via ``ts-loader``) to transpile it at build |
| 22 | +time. This worked, but prevented the use of TypeScript path aliases (e.g., |
| 23 | +``@src/*``) in consuming applications. Path aliases are resolved by ``tsc`` |
| 24 | +during compilation, and when a dependency ships raw TypeScript, its own aliases |
| 25 | +leak into the consumer's build, where they cannot be resolved. By compiling to |
| 26 | +JavaScript before publishing and using ``tsc-alias`` to resolve path aliases at |
| 27 | +that stage, consumers are free to define and use their own aliases |
| 28 | +independently. |
| 29 | + |
| 30 | +The problem with module.config.js |
| 31 | +----------------------------------- |
| 32 | + |
| 33 | +``module.config.js`` was the previous mechanism for local development of |
| 34 | +frontend dependencies. It worked by reading a configuration file at webpack |
| 35 | +build time and creating ``resolve.alias`` entries to redirect imports to local |
| 36 | +directories. While functional in simple cases, it had several problems: |
| 37 | + |
| 38 | +1. **Webpack-only**: It only affected webpack's module resolution, so features |
| 39 | + that depend on Node.js resolution (such as TypeScript path aliases or |
| 40 | + ``tsc``-based compilation) were not supported. |
| 41 | +2. **Brittle dependency resolution**: It attempted to second-guess npm's |
| 42 | + dependency resolution by manually resolving peer dependencies to the |
| 43 | + consumer's ``node_modules``. This not only side-stepped the deduplication |
| 44 | + that happens in production, but often broke things entirely with modern |
| 45 | + ``exports`` maps. |
| 46 | +3. **Divergence from production**: The aliased resolution paths differed |
| 47 | + fundamentally from how dependencies resolve in a published package, violating |
| 48 | + the principle that development should mirror production as closely as |
| 49 | + possible. |
| 50 | + |
| 51 | +Alternatives considered |
| 52 | +----------------------- |
| 53 | + |
| 54 | +- **``npm link``**: Does not work with the project's TypeScript configuration |
| 55 | + (``moduleResolution: "bundler"`` is incompatible with symlinked packages). |
| 56 | +- **``yarn``, ``pnpm``, ``bun``**: Switching package managers would entail too |
| 57 | + many changes across the Open edX ecosystem for the benefit gained. |
| 58 | +- **``yalc``**: A promising tool that wraps ``npm pack`` with push/watch |
| 59 | + semantics. It does what we need, but has not been actively maintained (last |
| 60 | + merge in 2023) and is missing features we would want (e.g., ``--peer`` flag). |
| 61 | + It remains a potential future option if it becomes actively maintained again or |
| 62 | + if we fork it. |
| 63 | + |
| 64 | +Design principles |
| 65 | +----------------- |
| 66 | + |
| 67 | +The following principles guided this decision: |
| 68 | + |
| 69 | +- **Closeness to production**: The development workflow should mirror production |
| 70 | + as closely as possible to minimize environment-specific bugs. |
| 71 | +- **Resource optimization**: The workflow should not add unnecessary hoops. |
| 72 | + Developer time, system resources, and cognitive load should be minimized. |
| 73 | +- **Consistency**: The same development workflow should work uniformly across all |
| 74 | + packages: ``frontend-base`` itself, consuming applications like |
| 75 | + ``frontend-app-authn``, third-party packages, and site configuration |
| 76 | + repositories. |
| 77 | + |
| 78 | +Decision |
| 79 | +======== |
| 80 | + |
| 81 | +1. **Compile TypeScript before publishing**: ``@openedx/frontend-base`` and all |
| 82 | + frontend-app-* repositories ship compiled JavaScript with ``.d.ts`` |
| 83 | + declaration files, using export maps to define its public API. This enables |
| 84 | + consuming applications to also pre-compile their TypeScript and use path |
| 85 | + aliases. |
| 86 | + |
| 87 | +2. **Remove ``module.config.js`` support**: The ``getLocalAliases()`` function |
| 88 | + and all references to ``module.config.js`` are removed from the webpack |
| 89 | + configurations and the codebase. |
| 90 | + |
| 91 | +3. **Adopt tarball-based local development**: The official mechanism for local |
| 92 | + development of Open edX frontend dependencies is the ``npm pack`` / |
| 93 | + ``.tgz``-based workflow. The developer builds the dependency, packs it into a |
| 94 | + tarball, and the consuming project installs from that tarball. Tooling to aid |
| 95 | + with this workflow will also be provided (details in the `Implementation`_ |
| 96 | + section). |
| 97 | + |
| 98 | +Implementation |
| 99 | +============== |
| 100 | + |
| 101 | +TypeScript compilation |
| 102 | +---------------------- |
| 103 | + |
| 104 | +The package is compiled with ``tsc`` using ``tsconfig.build.json``, producing |
| 105 | +JavaScript output and declaration files under ``/dist``. Path aliases are |
| 106 | +resolved post-compilation with ``tsc-alias``. The npm ``exports`` map in |
| 107 | +``package.json`` maps public entry points to their compiled locations. |
| 108 | + |
| 109 | +At the bundler level, we add ``tsconfig-paths-webpack-plugin`` to the default |
| 110 | +webpack configurations so that TypeScript path aliases are also respected by |
| 111 | +webpack builds (including during ``npm run dev``) without duplicating their |
| 112 | +definitions. |
| 113 | + |
| 114 | +As part of this change, we unify the TypeScript build outputs under ``/dist`` |
| 115 | +and use npm export maps to decouple the internal file structure from the |
| 116 | +package's public API. |
| 117 | + |
| 118 | +Local development workflow |
| 119 | +-------------------------- |
| 120 | + |
| 121 | +To develop a local dependency (e.g., ``@openedx/frontend-base``) against a |
| 122 | +consuming project: |
| 123 | + |
| 124 | +1. In the dependency: ``npm run build`` (or use a watcher like ``nodemon`` with |
| 125 | + ``npm run pack:local``) |
| 126 | +2. In the consumer: install from the tarball and run the dev server (or use the |
| 127 | + `autoinstall tool`_ from the ``frontend-dev-utils`` package) |
| 128 | + |
| 129 | +This approach is consistent across all package types and faithfully reproduces |
| 130 | +production resolution semantics, since ``npm pack`` produces the same artifact |
| 131 | +that ``npm publish`` would. |
| 132 | + |
| 133 | +The ``test-site`` project contains reference tooling that automates detecting |
| 134 | +new tarballs and reinstalling them during development. |
| 135 | + |
| 136 | +.. _autoinstall tool: https://github.com/openedx/frontend-dev-utils/blob/main/tools/autoinstall/README.md |
0 commit comments