Elastic infrastructure that scales up & down based on demand is not just a “serverless” fad but an operational model which reduces cost and waste. Yet there is a little devil lurking under the covers… When an application / microservice needs to spin up based on demand there can be some lag as the application needs to be downloaded to the node, potentially a VM needs to be started, the application itself needs to be started, and potentially local caches need to be hydrated. In traditional enterprise systems this “cold start” process can realistically take tens of minutes. But it’s near impossible to have demand-based scaling when things take that long to start.
GraalVM is a Java Virtual Machine implementation that addresses parts of the “cold start” problem by doing Ahead-Of-Time (AOT) compilation on JVM-based applications. GraalVM can create a “native image” of your application so that it no longer needs to run inside a JVM. This can reduce startup time and in some cases improve overall performance. The native images can also be much smaller than the usual OS + JVM + all dependency JARs. For example, a recent application I was working on went from a pretty trim 208MB docker image (OS + JVM + deps + app) that started in 2 seconds, down to 14MB and a 0.5s startup time. Sounds amazing! But there are some caveats, tips, and tricks I’d like to share with you.
Let’s start with a couple samples:
|Build Tool / Language / Framework||Source||Container Size||Demo on Cloud Run|
|Maven / Java / Quarkus||github.com/jamesward/hello-quarkus||28MB||Live Demo|
|Gradle / Kotlin / Micronaut||github.com/jamesward/hello-micronaut (graalvm branch)||55MB||Live Demo|
|sbt / Scala / ZIO||github.com/jamesward/hello-uzhttp (graalvm branch)||14MB||Live Demo|
Check out the README’s for instructions on how to build and run those. Now let’s dive into some tips & tricks.
Build Containers in Containers
One thing you might have noticed in those samples is that I run the builds and native-image tool inside of Docker because that is the most reliable way to have a reproducible build with GraalVM. Here is the basic structure for a GraalVM native image
FROM oracle/graalvm-ce:20.0.0-java11 as builder WORKDIR /app COPY . /app RUN gu install native-image # Build the app (via Maven, Gradle, etc) and create the native image FROM scratch COPY --from=builder /app/target/my-native-image /my-native-image ENTRYPOINT ["/my-native-image"]
Of course it’s time consuming to re-run a docker build like this as part of the development cycle so it’s nice to just run these apps on a JVM for local development and only create the native image as part of the CI/CD deployment process. Quarkus and sbt have a way to run the native-image tool inside a docker container without needing a
Dockerfile but those methods don’t seem to support some customizations that we will get into shortly. Quarkus and sbt also have build tasks that build the app and then run the
native-image tool for you. Gradle / Micronaut instead just use the regular packaging tasks and then you can run
native-image in your
RUN ./gradlew --no-daemon --console=plain distTar RUN tar -xvf build/distributions/hello-micronaut.tar -C build/distributions RUN native-image -cp build/distributions/hello-micronaut/lib/hello-micronaut.jar:build/distributions/hello-micronaut/lib/* hello.WebAppKt
This more explicit approach makes it clear that
native-image is just taking the jar files containing the app and the dependencies as well as a class with a
static void main and turning those into the native image.
GraalVM has a neat feature that allows us to build a statically linked native image using the
--static parameter. If you are calling
native-image directly then you just add that parameter. If you are using Maven / Quarkus or sbt then you’ll use a build config parameter to add it (i.e. pom.xml, build.sbt). By default this still requires your base container image to have
libstdc++ on it. But wouldn’t it be cool to build a statically linked native image and put it into an empty container image with no system dependencies giving us super small containers? Well you can use
FROM scratch in the
Dockerfile and copy the native image in! But problems arise when an execution path tries to call something in
libstdc++ which is the case with DNS lookups.
Luckily GraalVM has a way to also include the necessary system libraries in the static native image! There is a library called musl libc that you can include in your native image. In your
Dockerfile download the musl bundle for GraalVM:
RUN curl -L -o musl.tar.gz https://github.com/gradinac/musl-bundle-example/releases/download/v1.0/musl.tar.gz && \ tar -xvzf musl.tar.gz
And then add a
native-image parameter that points to the extracted location of the bundle, like:
Now your native image will include the standard library system calls that are needed!
Don’t Fall Back
GraalVM has a default feature where if the AOT thing fails, it will fallback to just running the app in the JVM. You likely don’t want that because why would you be creating a native image if it’s just going to end up running in the JVM? Right, so make sure you set a
Don’t Defer Problems to Runtime
Some AOT compilation errors can be deferred to runtime. You really don’t want this unless you like production crashes. So make sure
native-image is NOT being run with any of these params:
--report-unsupported-elements-at-runtime --allow-incomplete-classpath -H:+ReportUnsupportedElementsAtRuntime
Lots and lots and lots of very widely adopted Java libraries use reflection but it creates a problem for GraalVM native images because reflection happens at runtime, making it hard for an AOT complier to figure out the execution paths. To deal with this you can tell GraalVM about what needs reflection access. But this can quickly get a bit out-of-hand, hard to derive, and hard to maintain. Micronaut and Quarkus do a pretty good job generating the reflection configuration at compile time but you might need to augment the generated config. This can get a bit tricky when you have dependencies with shaded transitive dependencies.
To reliably generate a reflection config you need to exercise as many execution code paths as possible, ideally by running your unit and integration tests. GraalVM has a way to keep track of reflection and output the configuration. To use this you’ll need to run the app on GraalVM (remember it’s just a JVM implementation) and use a special Java agent that will be able to see the reflective calls. I didn’t do this inside Docker because then getting the generated config files out becomes more challenging. First, you need to grab a GraalVM Community Edition release, set your
PATH. Then also from the release assets grab the right
native-image-installable-svm-BLAH.jar file and extract it in the root of your GraalVM
JAVA_HOME directory. Ok, now run your tests with this parameter:
This will generate the reflection config (and possibly other configs for dynamic proxies, etc). Then you need to tell
native-image about those configs, like:
Or better, don’t use reflection or things that use reflection. More on that in a future blog. :)
I only have one small app in production that uses GraalVM native image, running on Cloud Run via a
FROM scratch container image. It starts up super fast and lives in a 44MB container image. GraalVM really helped address the cold start problem. But what about for more complex applications? I dunno. Let me know how it goes and what you’ve learned!