Skip to content

Stop rebuilding C++ dependencies in pip devcontainers #293

Description

@vyasr

RAPIDS C++ wheels are depend on shared libraries from other RAPIDS C++ wheels. They are designed to find these libraries at build time via CMake. Each wheel exports the standard config files, but the key piece to make these discoverable is when a package uses scikit-build-core as its build backend (which all RAPIDS C++ wheels do), scikit-build-core will scan all installed packages in the build environment and if any of them expose a cmake.prefix entry point the paths in that entry point will be automatically appended to CMAKE_PREFIX_PATH during the build. Therefore, every RAPIDS C++ wheel exports this entry point to make libraries discoverable for downstream C++ wheel dependents.

Unfortunately, this mechanism is not in play during pip devcontainer builds because pip devcontainers build the C++ libraries directly with CMake, not as wheels. Those C++ libraries are then manually made available to downstream Python extension module builds by way of devcontainer scripts specifying CMake <packageName>_ROOT variables. Fortunately, we can bridge this gap. Since the cmake.prefix entry points are always available as package metadata, all we have to do is reproduce scikit-build-core's functionality for finding these entry points. rapidsai/rapids-cmake#951 already added this feature, and it is in use in rapidsmpf as of rapidsai/rapidsmpf#727. We should port that over to other RAPIDS C++ CMakeLists.txt files as well to reduce build times and the number of unique failure points in CI even further.

Metadata

Metadata

Assignees

No one assigned

    Labels

    improvementImproves an existing functionality

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions