Fix language files for Node+ESM #1377

sequba · 2024-02-15T14:04:22Z

Context

changed babel configuration to produce the ESM build with .mjs extensions and adjust the import and export statements in the output (custom babel plugin)
added exports property to package.json that lists all valid ways of importing HyperFormula
added the migration guide

Useful reading about similar solutions:

Related issues:

New import paths for the language files

Module system	valid import paths in HF 2.x	valid import paths in HF 3.0.0
ES modules	hyperformula/es/i18n/languages/ hyperformula/es/i18n/languages/frFR hyperformula/es/i18n/languages/frFR,js	hyperformula/i18n/languages/ hyperformula/i18n/languages/frFR hyperformula/es/i18n/languages/ (legacy) hyperformula/es/i18n/languages/frFR (legacy) hyperformula/es/i18n/languages/frFR,js (legacy)
CommonJS	hyperformula/commonjs/i18n/languages/ hyperformula/commonjs/i18n/languages/frFR hyperformula/commonjs/i18n/languages/frFR.js	hyperformula/i18n/languages/ hyperformula/i18n/languages/frFR hyperformula/commonjs/i18n/languages/ (legacy) hyperformula/commonjs/i18n/languages/frFR (legacy) hyperformula/commonjs/i18n/languages/frFR.js (legacy)
UMD	hyperformula/dist/languages/	hyperformula/dist/languages/

Breaking change for projects that use Typescript, Angular <=17, or Webpack <=4

Angular

Angular version	supported by Angular	legacy import paths	new import paths
18	yes	✅	✅
17	yes	⚠️works after configuring `moduleResolution`	⚠️works after configuring `moduleResolution`
16 with Typescript 5	until 2024-11-08	⚠️works after configuring `moduleResolution`	⚠️works after configuring `moduleResolution`
16 with Typescript 4	until 2024-11-08	⚠️works after upgrading to typescript 5 and configuring `moduleResolution`	⚠️works after upgrading to typescript 5 and configuring `moduleResolution`
15	no	❌	❌

Typescript

Typescript version	legacy import paths	new import paths
5	⚠️works after configuring `moduleResolution`	⚠️works after configuring `moduleResolution`
4	⚠️works after configuring `moduleResolution`	⚠️works after configuring `moduleResolution`

Further reading: https://www.typescriptlang.org/docs/handbook/modules/theory.html#module-resolution-for-libraries

Webpack

Webpack version	legacy import paths	new import paths
5	✅	✅
5 + babel	✅	✅
4	⚠️works after configuring `resolver-default`	❌
4 + babel	⚠️works after configuring `resolver-default`	❌

Webpack 4 has 2 separate issues:

It doesn't know how to handle mjs files.
It doesn't support exports property to package.json.

For the former issue, there is a workaround. It's enough to configure a rule for mjs files. But it seems that there is no way to make it understand the exports property. For that reason, Webpack 4 can work with HyperFormula 3.0.0, but only with import paths that match the actual paths (ignoring the exports property that provides the artificial paths).

Parcel

Parcel version	legacy import paths	new import paths
<2.9.0	✅	❌
>=2.9.0	✅	⚠️works after configuring `resolver-default`

Testing

Test importing HF in various setups

Demo projects are available in the hyperformula-demos repo.

Tested by @budnix:

Vite ^5.1.4
Parcel2 ^2.0.0
Webpack5 ^5.8.0
Webpack4 ^4.42.0

Tested by @magierg

Webpack 5.9 + TS
React 18^ + TS
Angular 16.0.2 + TS 4.4.3
Angular 17.3.1 + TS 5.2
Svelte 3.5.4 + vite 4
Vue 3.3.4 + vite 4.4.9

Further verification

Types of changes

Breaking change (a fix or a feature because of which an existing functionality doesn't work as expected anymore)
New feature or improvement (a non-breaking change that adds functionality)
Bug fix (a non-breaking change that fixes an issue)
Additional language file, or a change to an existing language file (translations)
Change to the documentation

Checklist:

I have reviewed the guidelines about Contributing to HyperFormula and I confirm that my code follows the code style of this project.
I have signed the Contributor License Agreement.
My change is compliant with the OpenDocument standard.
My change is compatible with Microsoft Excel.
My change is compatible with Google Sheets.
I described my changes in the CHANGELOG.md file.
My changes require a documentation update.
My changes require a migration guide.

github-actions · 2024-02-15T14:21:02Z

Performance comparison of head (`fecd03a`) vs base (`5def4ef`)

                                     testName |   base |   head | change
------------------------------------------------------------------------
                                      Sheet A |  502.4 | 503.89 | +0.30%
                                      Sheet B | 161.16 | 161.44 | +0.17%
                                      Sheet T | 141.27 | 141.11 | -0.11%
                                Column ranges | 504.91 | 506.38 | +0.29%
Sheet A:  change value, add/remove row/column |  14.98 |  13.95 | -6.88%
 Sheet B: change value, add/remove row/column | 129.08 | 122.98 | -4.73%
                   Column ranges - add column | 151.87 | 147.88 | -2.63%
                Column ranges - without batch | 436.92 | 434.39 | -0.58%
                        Column ranges - batch | 110.77 | 111.05 | +0.25%

budnix

I tested the dist files in some more popular bundlers and here are the results:

Vite ^5.1.4 - OK
Parcel2 ^2.0.0 - Does not support "exports" so the language file has to be imported with /es/ or /commonjs/ prefixes;
Webpack5 ^5.8.0 - OK
Webpack4 ^4.42.0 - I getting an error. Seems the webpack has the problem with importing values from "chevrotain".

// /node_modules/hyperformula/es/parser/FormulaParser.mjs
import { EmbeddedActionsParser, EMPTY_ALT, Lexer, tokenMatcher } from 'chevrotain';
// EmbeddedActionsParser, EMPTY_ALT and others are missing

ERROR in ./node_modules/hyperformula/es/parser/FormulaParser.mjs 689:125-137
Can't import the named export 'tokenMatcher' from non EcmaScript module (only default export is available)
 @ ./node_modules/hyperformula/es/parser/index.mjs
 @ ./node_modules/hyperformula/es/ArraySize.mjs
 @ ./node_modules/hyperformula/es/index.mjs
 @ ./src/index.js

ERROR in ./node_modules/hyperformula/es/parser/FormulaParser.mjs 708:29-41
Can't import the named export 'tokenMatcher' from non EcmaScript module (only default export is available)
 @ ./node_modules/hyperformula/es/parser/index.mjs
 @ ./node_modules/hyperformula/es/ArraySize.mjs
 @ ./node_modules/hyperformula/es/index.mjs
 @ ./src/index.js

package.json

sequba · 2024-02-29T15:43:37Z

Webpack4 ^4.42.0 - I getting an error. Seems the webpack has the problem with importing values from "chevrotain".

// /node_modules/hyperformula/es/parser/FormulaParser.mjs
import { EmbeddedActionsParser, EMPTY_ALT, Lexer, tokenMatcher } from 'chevrotain';
// EmbeddedActionsParser, EMPTY_ALT and others are missing

ERROR in ./node_modules/hyperformula/es/parser/FormulaParser.mjs 689:125-137
Can't import the named export 'tokenMatcher' from non EcmaScript module (only default export is available)
 @ ./node_modules/hyperformula/es/parser/index.mjs
 @ ./node_modules/hyperformula/es/ArraySize.mjs
 @ ./node_modules/hyperformula/es/index.mjs
 @ ./src/index.js

ERROR in ./node_modules/hyperformula/es/parser/FormulaParser.mjs 708:29-41
Can't import the named export 'tokenMatcher' from non EcmaScript module (only default export is available)
 @ ./node_modules/hyperformula/es/parser/index.mjs
 @ ./node_modules/hyperformula/es/ArraySize.mjs
 @ ./node_modules/hyperformula/es/index.mjs
 @ ./src/index.js

I found out that webpack 4 does not support exports, and there seems to be no easy workaround for that. By tweaking the webpack configuration, I managed to make the project import HyperFormula correctly using the legacy paths. Working demo.

magierg · 2024-03-27T07:14:06Z

I have retested all the demos available on https://github.com/handsontable/hyperformula-demos/tree/import-demos

plus the bundlers mentioned by @budnix getting the same results.

On top of the above I have tested:

Webpack 5.9 + TS - tsconfig.js setting change required:

"module": "node16",
"moduleResolution": "node16",

React 18^ + TS - same tsconfig.js change required
Angular 16.0.2 + TS 4.4.3 - @sequba to investigate tsconfig changes required
Angular 17.3.1 + TS 5.2 - same issue with TS
Svelte 3.5.4 + vite 4 - getting this error - might be user error - @sequba to investigate

Error: Language already registered.
    at HyperFormula.registerLanguage (/Users/magierg/repos/hyperformula-demos/svelte-demo/node_modules/hyperformula/commonjs/HyperFormula.js:287:13)
    at Hyperformula.svelte:18:15
    at Object.$$render (/node_modules/svelte/internal/index.mjs:1892:22)
    at eval (/src/routes/+page.svelte:24:148)
    at Object.$$render (/node_modules/svelte/internal/index.mjs:1892:22)
    at Object.default (root.svelte:41:38)
    at eval (/node_modules/@sveltejs/kit/src/runtime/components/layout.svelte:8:41)
    at Object.$$render (/node_modules/svelte/internal/index.mjs:1892:22)
    at root.svelte:40:37
    at $$render (/node_modules/svelte/internal/index.mjs:1892:22)

Vue 3.3.4 + vite 4.4.9 - OK

sequba · 2024-04-12T12:06:52Z

TODO: verify it with the code example in issue #1143

sequba · 2024-04-25T19:46:58Z

Svelte 3.5.4 + vite 4 - getting this error - might be user error - @sequba to investigate

It works correctly. The issue you reported was caused by running the setup code twice. I created a demo for svelte: https://github.com/handsontable/hyperformula-demos/tree/import-demos/import-demo-esm-svelte

Angular case is still to be verified.

codecov · 2024-08-07T11:11:17Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.36%. Comparing base (e32e132) to head (0cd4e05).
Report is 33 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #1377   +/-   ##
========================================
  Coverage    97.36%   97.36%           
========================================
  Files          169      169           
  Lines        14421    14421           
  Branches      3021     3021           
========================================
  Hits         14041    14041           
  Misses         380      380

…eature/issue-1344

sequba · 2024-08-08T07:32:36Z

Angular 16.0.2 + TS 4.4.3 - @sequba to investigate tsconfig changes required
Angular 17.3.1 + TS 5.2 - same issue with TS

Angular issue can be resolved by setting "moduleResolution": "bundler" in tsconfig.json. Tested with Angular 16, 17 and 18 and Typescript 5. Demo

Unfortunately, I couldn't make combination Angular + Typescript 4 work. I'm afraid we need to ask our clients to upgrade to Typescript 5 if they want to use HyperFormula 3.x with Angular.

budnix

Looks good 👌

magierg · 2024-11-08T00:36:50Z

I have retested the previously mentioned configurations and performed the migration steps - found out that the @babel/core is not required - this has been already addressed by @sequba

sequba added 5 commits February 15, 2024 14:11

Add babel script add-import-extension

636ccaa

Adjust check-publish-package script to new way of building ES

c09c336

Make babel output ES build with mjs extensions

1ba6009

Add exports property to package.json

2dc845f

Fix the exports property

bdbc35e

sequba self-assigned this Feb 15, 2024

sequba linked an issue Feb 15, 2024 that may be closed by this pull request

Language files do not work with ES modules in node #1344

Open

sequba added 3 commits February 21, 2024 15:27

Tweaking package.json exports for cjs modules

fe9f96e

Adjust package.json exports for individual language files in CJS

d25db4e

Add info about the source of the add-import-extension babel script

1962234

sequba marked this pull request as ready for review February 21, 2024 14:59

Add changelog entry

4bf3f07

sequba requested a review from budnix February 21, 2024 15:08

sequba added 2 commits February 22, 2024 14:47

Move types key to be the first key in JSON object (exports property)

36b997e

Fix publint suggestions

5348bf7

budnix requested changes Feb 26, 2024

View reviewed changes

package.json Outdated Show resolved Hide resolved

sequba added 3 commits February 28, 2024 10:14

Add paths ./i18n/languages/xxXX to exports property in package.json

bccb945

Use path ./i18n/languages/xxXX to import languages in the docs

8a6d108

Add paths with .js extensions to exports in package.json

0cd4e05

sequba requested a review from budnix February 29, 2024 15:52

budnix approved these changes Mar 4, 2024

View reviewed changes

sequba added the Breaking Change Will need a major release label Mar 5, 2024

sequba changed the title ~~Fix language files for Node+ESM~~ [DO NOT MERGE] Fix language files for Node+ESM Mar 5, 2024

sequba changed the title ~~[DO NOT MERGE] Fix language files for Node+ESM~~ [DO NOT MERGE!] Fix language files for Node+ESM Mar 5, 2024

sequba added 2 commits August 7, 2024 13:13

Merge branch 'develop' of github.com:handsontable/hyperformula into f…

b309044

…eature/issue-1344

Add migration guide for 3.0

6f32832

sequba added 5 commits September 5, 2024 15:32

Working on the migration guide

89159a5

Unpolished vershion of the migration guide

3c29512

Upgrade migration instructions for Angular projects

4c3d108

Add more explenations to the migration guide

f0cea7b

Add a fallback option for parcel projects to the migration guide

51732b7

sequba requested a review from budnix October 31, 2024 14:34

Undo the version number change in package.json

5128ba3

sequba changed the title ~~[DO NOT MERGE!] Fix language files for Node+ESM~~ Fix language files for Node+ESM Nov 5, 2024

budnix approved these changes Nov 5, 2024

View reviewed changes

sequba and others added 2 commits November 5, 2024 11:11

Update migration guide

152b0dd

Remove unnsecessary step from the parcel section of the migration guide

44e9d56

sequba added 2 commits November 12, 2024 11:20

Merge branch 'develop' into feature/issue-1344

a07d548

Rephrase CHANGELOG.md

fecd03a

sequba merged commit c27a9d5 into develop Nov 12, 2024
21 of 22 checks passed

sequba deleted the feature/issue-1344 branch November 12, 2024 11:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix language files for Node+ESM #1377

Fix language files for Node+ESM #1377

sequba commented Feb 15, 2024 •

edited

Loading

github-actions bot commented Feb 15, 2024 •

edited

Loading

budnix left a comment

sequba commented Feb 29, 2024 •

edited

Loading

magierg commented Mar 27, 2024

sequba commented Apr 12, 2024

sequba commented Apr 25, 2024

codecov bot commented Aug 7, 2024

sequba commented Aug 8, 2024 •

edited

Loading

budnix left a comment

magierg commented Nov 8, 2024

Fix language files for Node+ESM #1377

Fix language files for Node+ESM #1377

Conversation

sequba commented Feb 15, 2024 • edited Loading

Context

Related issues:

New import paths for the language files

Breaking change for projects that use Typescript, Angular <=17, or Webpack <=4

Angular

Typescript

Webpack

Parcel

Testing

Test importing HF in various setups

Tested by @budnix:

Tested by @magierg

Further verification

Types of changes

Checklist:

github-actions bot commented Feb 15, 2024 • edited Loading

Performance comparison of head (fecd03a) vs base (5def4ef)

budnix left a comment

Choose a reason for hiding this comment

sequba commented Feb 29, 2024 • edited Loading

magierg commented Mar 27, 2024

sequba commented Apr 12, 2024

sequba commented Apr 25, 2024

codecov bot commented Aug 7, 2024

Codecov Report

sequba commented Aug 8, 2024 • edited Loading

budnix left a comment

Choose a reason for hiding this comment

magierg commented Nov 8, 2024

sequba commented Feb 15, 2024 •

edited

Loading

github-actions bot commented Feb 15, 2024 •

edited

Loading

Performance comparison of head (`fecd03a`) vs base (`5def4ef`)

sequba commented Feb 29, 2024 •

edited

Loading

sequba commented Aug 8, 2024 •

edited

Loading