Building wheel package fails with JS dependency error

[bertptrs]

I'm trying to build the wheel from scratch, for packaging for Arch Linux, but am running into a dependency issue. I believe the lockfile needs to be updated. To reproduce:

$ git clone https://github.com/plotly/plotly.py.git
Cloning into 'plotly.py'...
remote: Enumerating objects: 137137, done.
remote: Counting objects: 100% (10730/10730), done.
remote: Compressing objects: 100% (2578/2578), done.
remote: Total 137137 (delta 8449), reused 10172 (delta 8086), pack-reused 126407 (from 1)
Receiving objects: 100% (137137/137137), 165.11 MiB | 3.77 MiB/s, done.
Resolving deltas: 100% (107263/107263), done.
$ cd plotly.py/packages/python/plotly
$ git checkout v5.24.0
$ python -m build --wheel --no-isolation
* Getting build dependencies for wheel...
running egg_info
running jsdeps
10.8.3
10.8.3
Installing build dependencies with npm.  This may take a while...
<SNIP lots of output>
> [email protected] build:labextension
> jupyter labextension build .

Building extension in .
/usr/lib/python3.12/site-packages/jupyterlab/debuglog.py:54: UserWarning: An error occurred.
  warnings.warn("An error occurred.")
/usr/lib/python3.12/site-packages/jupyterlab/debuglog.py:55: UserWarning: ValueError: Extensions require a devDependency on @jupyterlab/builder@^4.2.5, you have a dependency on 3.6.1
  warnings.warn(msg[-1].strip())
/usr/lib/python3.12/site-packages/jupyterlab/debuglog.py:56: UserWarning: See the log file for details: /tmp/jupyterlab-debug-vvcteyvy.log
  warnings.warn(f"See the log file for details: {log_path!s}")
rebuilding js and css failed
missing files: ['/home/bert/Downloads/plotly.py/packages/python/plotly/jupyterlab_plotly/labextension/package.json']
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in <module>
    main()
  File "/usr/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main
    json_out["return_val"] = hook(**hook_input["kwargs"])
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 134, in get_requires_for_build_wheel
    return hook(config_settings)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
    return self._get_build_requires(config_settings, requirements=['wheel'])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
    self.run_setup()
  File "/usr/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
    exec(code, locals())
  File "<string>", line 540, in <module>
  File "/usr/lib/python3.12/site-packages/setuptools/__init__.py", line 104, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup
    return run_commands(dist)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
    dist.run_commands()
  File "/usr/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.12/site-packages/setuptools/dist.py", line 967, in run_command
    super().run_command(command)
  File "/usr/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "<string>", line 106, in run
  File "<string>", line 99, in run
  File "/usr/lib/python3.12/site-packages/setuptools/dist.py", line 967, in run_command
    super().run_command(command)
  File "/usr/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "<string>", line 200, in run
  File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['npm', 'run', 'build:prod']' returned non-zero exit status 1.

ERROR Backend subprocess exited when trying to invoke get_requires_for_build_wheel

This problem can be worked around by causing the lockfile to be invalid, so dependencies are evaluated again, but that is not desirable in the context of reproducible builds.

[bertptrs]

Nice to see that this was picked up already. Is there anything I can do to help out here? I have tried regenerating the lockfile but that runs into different issues because certain libraries no longer type check. Nevertheless, I'd be happy to help solving this issue.

[archmoj]

@marthacryan @LiamConnors Do you have a similar issue?

[LiamConnors]

@bertptrs I've been trying to recreate the issue you encounter and have been able to when I have jupyterlab 4 installed. I was wondering is --no-isolation a requirement for you here? With --no-isolation, as far as I know, it's not going to install the build dependency versions specified in the project.toml file https://github.com/plotly/plotly.py/blob/master/packages/python/plotly/pyproject.toml#L2
In places in CI when doing the builds, we specifically install jupyterlab 3 first: https://github.com/plotly/plotly.py/blob/master/.circleci/config.yml for example, here https://github.com/plotly/plotly.py/blob/master/.circleci/config.yml#L350

[bertptrs]

Thanks for looking into this! I think you found the issue.

I am indeed building with jupyterlab (4.2.5) as I am packaging plotly for Arch Linux which as a rolling distro works with the latest version of its dependencies by default. Shipping an older version is tricky as (as far as I know) different installs of jupyterlab cannot coexist system-wide.

You gave me some ideas on how to make this work, so thank you!