Client Resource Handling.

[Deledrius]

It seems that we keep running into issues with the code to render and package the resource.dat (used by the client to contain pre-ResMgr assets such as cursors and loading-screen images), due to it depending on fragile, ill-supported libraries that are being used solely for this one tool. The issues caused by the module dependencies have been infrequent enough to avoid needing immediate attention, but frequent enough to feel like an issue that will keep arising. A few notions have been raised over the years about fixing this, but nothing concrete has emerged.

Is this enough of a problem that it needs to be addressed, or is it a low enough priority that it should continue as-is?

I see a few different ways to address the problem:

• When the client's resource.dat rendering and packaging code was added, we didn't have any other repositories for game assets. We could move the source image SVG assets and the scripts to render them to moul-assets. This would move the problem away from building the client code, but still require maintaining the tools. Since we don't update/change the assets frequently, the need to build them with each client build does feel superfluous at times.
• We could move away from the poorly-supported Python modules and make these scripts into a first-class Plasma tool written in C++, using the upstream cairo/librsvg libraries. Unfortunately, even these libraries are not well-documented and would only remove the dependency on the Python interpreter for Plasma, while shifting the required modules into C/C++ libraries. Their current vcpkg integration is... poor. I have already done some work on writing code for this option while exploring it as a potential solution. It seems as if this would be less fragile than the current solution, but I'm not sure how much less fragile (and it would also be a substantial amount of work).
• Add SVG rendering directly into the engine and render the images to plMipmaps or plPlates on-demand. This would still require cairo/librsvg but remove the need for custom tooling to render them at build time, reducing the build complexity. This seems like it might be an overkill solution, but could also have side benefits when it comes to improved future GUI elements.

I think it's a valuable part of the process that these assets can be built reliably and deterministically every time, and I'd rather not lose that aspect. The current Python script itself is relatively easy to maintain and flexible due to the inherent benefits of being written in Python (as shown by the modifications made to support our Gehn resources). At the same time, it's frustrating to have our build system depend on several libraries solely for such a small part of the client, especially when those libraries are prone to breakage due to lack of upstream maintenance and support.

Thoughts on whether this should change, and if so, how?

[dpogue]

One of the downsides of the current Python-based system is that we require both the Python development files (for the target platform of the engine) and the Python interpreter executable (for the host platform where the build is running). CMake isn't particularly adept at managing that distinction, which has caused issues for cross-compile builds for things like Apple Silicon and Arm Linux.

I believe the need for the Python interpreter is also what forces us to use dynamic linking of python.dll on Windows, rather than making it a static library (which IIRC is what we want to do for macOS because it makes codesigning a lot easier).

Those problems aside, there are clear upsides to having the resource generation happen as part of the build.

• Moving to moul-assets is probably the best option if we want to keep the current tooling but move it out of the client.

• Moving from Python to C++ directly would involve some changes to the build process to both build that new tool and invoke it as part of plClient's build. I share your concerns about fragility.

• Adding SVG rendering directly to the engine seems like something we'd want to avoid. It would add additional library requirements, and SVG (XML) parsing is inherently a source of bugs and potential security issues that we probably don't want to open up at runtime.

I think for the moment, my vote goes to option 1 (moving to moul-assets) based on being the lowest effort option, but I'm not opposed to option 2.

[Hoikas]

I'm somewhat in favor of option 2. It seems like a non-optimal design, IMO, to have something that the client basically requires to start not be buildable or available in the client repository. I appreciate that option 1 is easier, though. The downside to option 2 is that it will basically require building that tool for the host system where everything else might be being cross-compiled. IIRC this would require a second CMake build :/. If we rewrite the shaders in HLSL, though, and compile them as part of the build process (eg with plShaderAssembler), we'll run into this problem there as well. Another option (4) might be to retain the resource tools in this repository but remove them from the plClient CMake and version the resource.dat output.