The slimfast-plugin can be used in place of the maven-assembly-plugin or maven-shade-plugin (which are often used to build
fat jars). In addition, if you configure the maven-jar-plugin in the right way, the resulting jar (although not a fat jar)
will still be runnable using plain old java -jar
(ie, without needing to manually construct the classpath).
This uses a feature of the JVM which is that if you run a jar which has a Class-Path
entry in its manifest, then those
paths are added to the classpath of the JVM. Using this feature, we can tell the maven-jar-plugin to build the classpath
for us at build time and add it as a manifest property. Then we can configure the slimfast-plugin to copy the dependency
jars to the right place and the resulting jar will start up fine when run with java -jar
.
The plugin has three goals: copy
, upload
, and download
.
copy
can be used to copy your dependencies to the target folder so they're available at runtime (example).
This is similar to the copy-dependencies
goal of the maven-dependency-plugin
, but we were unable to get that to work
with a repository layout combined with resolved snapshot versions (the useBaseVersion
flag seems to get ignored when the
useRepositoryLayout
flag is set). Using the copy
goal saves you the time of building a fat jar and eliminates the jar
merging complexities, but it doesn't reduce the size of your build artifacts.
Just using the copy
goal has a lot of advantages and is a big win in its own right, but there's still room for improvement.
At HubSpot, for example, we tar up the build directory and upload it to S3 at the end of the build. Then we download and
untar it on the application servers when someone wants to deploy. Using the copy
goal doesn't reduce the size of these
tarballs so we're still uploading the same amount to S3 on build and downloading the same amount on deploy. This adds
time to builds and deploys, uses lots of bandwidth, and costs money for storing these large artifacts in S3.
But fear not! This is what the upload
and download
goals are for. The upload
goal binds to the deploy phase by default
and will upload all of the project's dependencies to S3 (example). It only uploads a dependency if it doesn't
already exist in S3, so after the initial build this step should mostly be a no-op and go very fast. When it's done uploading
the files, it will write out a JSON file (target/slimfast.json
by default) containing information that can be used later to
download the dependencies to the correct paths.
The most straightforward way to use this JSON file is to run the download
goal on your application servers before startup.
This goal doesn't require a project so it can run in standalone mode without a pom.xml
. A minimal invocation would look
like this. It will download all of the project dependencies (determined by reading target/slimfast.json
)
to the correct paths so that the application will start up with java -jar
.
Another option is to integrate this into your deployment phase, which is what we've done at HubSpot. Before using SlimFast,
at build time we would generate a single S3 artifact and store its information in the database so that we can fetch it at deploy
time. Now, we just store an array of S3 artifacts produced by the build (the main artifact, combined with the SlimFast artifacts
read from target/slimfast.json
). At deploy time, Singularity downloads all these S3
artifacts for us so everything just works.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>${your-main-class-property}</mainClass>
<classpathPrefix>lib/</classpathPrefix>
<classpathLayoutType>repository</classpathLayoutType>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>com.hubspot.maven.plugins</groupId>
<artifactId>slimfast-plugin</artifactId>
<version>1.0.0</version>
<executions>
<execution>
<goals>
<goal>copy</goal>
</goals>
<phase>package</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>${your-main-class-property}</mainClass>
<classpathPrefix>lib/</classpathPrefix>
<classpathLayoutType>repository</classpathLayoutType>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>com.hubspot.maven.plugins</groupId>
<artifactId>slimfast-plugin</artifactId>
<version>1.0.0</version>
<executions>
<execution>
<goals>
<goal>upload</goal>
</goals>
<phase>deploy</phase>
<configuration>
<s3Bucket>my-bucket</s3Bucket>
<s3ArtifactRoot>jars</s3ArtifactRoot>
<s3AccessKey>abc</s3AccessKey>
<s3SecretKey>123</s3SecretKey>
<s3Region>us-east-1</s3Region>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
You probably don't want to hard-code these S3 credentials in your pom though, instead you can use the
properties-maven-plugin
to read them from a file that is managed by puppet or your configuration management
tool of choice. If you have a file located at /etc/slimfast.properties
with contents like:
s3.bucket=my-bucket
s3.artifact.root=jars
s3.access.key=abc
s3.secret.key=123
Then you could invoke SlimFast like this:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>${your-main-class-property}</mainClass>
<classpathPrefix>lib/</classpathPrefix>
<classpathLayoutType>repository</classpathLayoutType>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0.0</version>
<executions>
<execution>
<goals>
<goal>read-project-properties</goal>
</goals>
<phase>initialize</phase>
<configuration>
<files>
<file>/etc/slimfast.properties</file>
</files>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>com.hubspot.maven.plugins</groupId>
<artifactId>slimfast-plugin</artifactId>
<version>1.0.0</version>
<executions>
<execution>
<goals>
<goal>upload</goal>
</goals>
<phase>deploy</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
mvn com.hubspot.maven.plugins:slimfast-plugin:0.12:download -Dslimfast.s3.accessKey=abc -Dslimfast.s3.secretKey=123