Add JavaModule#runClasspathAsJars

[alexarchambault]

This adds a JavaModule#runClasspathAsJars task, that returns the same class path as JavaModule#runClasspath, except all entries are JAR files, that it contains no directories. This is useful when passing the class path to applications that expect it to be only JAR files.

[lefou]

So what happens to the non-jar entries? Are they converted or just filtered out? If the latter, the name needs some change. While it suggests being the run classpath as jars, it's just a partial classpath, filtered to contain only jars.

[lefou]

Can you give some examples for application that expect only jars which result in an incomplete classpath?

[lihaoyi]

Not sure this needs to be upstreamed into Mill, seems like it could be a small helper in the project that needs it?

[alexarchambault]

I don't recall the exact use cases, but I remember having needed that from time to time, in other projects. This is useful to interface with external tools in particular.

sbt has similar keys, for more tasks actually. These seem to be used in the wild (even though forks pollute those results).

[alexarchambault]

So what happens to the non-jar entries? Are they converted or just filtered out? If the latter, the name needs some change. While it suggests being the run classpath as jars, it's just a partial classpath, filtered to contain only jars.

This computes the same thing as runClasspath, but uses JavaModule#jar rather than JavaModule#localClasspath to build it. There's no filtering, both tasks are supposed return equivalent class paths.

[alexarchambault]

In Spark, the spark.jars config key only accepts JARs for example. That's what initially motivated adding an equivalent option in Scala CLI (VirtusLab/scala-cli#2028). That extra task is handy when interfacing with such tools, getting Mill tasks results from the command-line in a script and passing that to other tools, for example.

[lefou]

So what happens to the non-jar entries? Are they converted or just filtered out? If the latter, the name needs some change. While it suggests being the run classpath as jars, it's just a partial classpath, filtered to contain only jars.

This computes the same thing as runClasspath, but uses JavaModule#jar rather than JavaModule#localClasspath to build it. There's no filtering, both tasks are supposed return equivalent class paths.

Got it. This could indeed be useful sometimes. E.g. in mill-osgi I need to override localClasspath, transitiveLocalClasspath and compileClasspath to manage something equivalent, which turned out to be harder to maintain with every new Mill release. (Background: OSGi modules depend on the JARs (or experimentally their manifests) to calculate proper package imports and version constraints.)

One thing worth pointing out though: runClasspathAsJars is more expensive than just runClasspath as it triggers more computaions and chances are, users use it in cases where runClasspath would be enough. Also, it's likely not enough to have a runClasspathAsJars. What about its dual for compileClasspath.

I'm not sure, we should provide it without a concrete use case (e.g. suport for a specific tool or framework like OSGi) that we can assert with an integration test.

[lihaoyi]

Would an alternative be to add a flag to make compile output a jar file rather than a loose folder of class files? IIRC Zinc already has this functionality, and at my last job we turned it on to satisfy scenarios like what you described where some downstream workflow expects jar files

[lefou]

Would an alternative be to add a flag to make compile output a jar file rather than a loose folder of class files? IIRC Zinc already has this functionality, and at my last job we turned it on to satisfy scenarios like what you described where some downstream workflow expects jar files

Probably not. Even if zinc would include the resources in that jar, it's missing the runtime-resources, which might be produced by arbitrary complex build tasks.

[alexarchambault]

One thing worth pointing out though: runClasspathAsJars is more expensive than just runClasspath as it triggers more computaions and chances are, users use it in cases where runClasspath would be enough. Also, it's likely not enough to have a runClasspathAsJars. What about its dual for compileClasspath.

Extra *AsJars tasks could be added, yes.

I'm not sure, we should provide it without a concrete use case (e.g. suport for a specific tool or framework like OSGi) that we can assert with an integration test.

I already added unit tests, that assert that runClasspathAsJars works and consists only of JARs.

[alexarchambault]

Just added compileClasspathAsJars.

A few other *AsJars tasks were needed for that. I let some of them private[mill], because some care is needed when using them. In particular, they should not be used as the implementation of the non-AsJars task, as this would introduce a cycle.

So the following is fine (public *AsJars tasks):

def compileClasspath = compileClasspathAsJars
def runClasspath = runClasspathAsJars
def localCompileClasspath = localCompileClasspathAsJars // not sure why anyone would do this one though

(first two are actually used in the unit tests added in this PR)

But the following is not (private[mill] *AsJars tasks)

def unmanagedClasspath = unmanagedClasspathAsJars
def compileResources = compileResourcesAsJars

[lihaoyi]

It's still not clear to me why this needs to be in JavaModule and not a helper trait in the downstream build or plugin that needs it. It's also not clear who the usages are, or what the exact requirements of those usages are. There's no way to propert review this PR until those questions are answered