GraalVM Native Image Tips & Tricks

Elastic infrastructure that scales up & down based on demand is not just a “serverless” fad but an operational model which reduces cost and waste. Yet there is a little devil lurking under the covers… When an application / microservice needs to spin up based on demand there can be some lag as the application needs to be downloaded to the node, potentially a VM needs to be started, the application itself needs to be started, and potentially local caches need to be hydrated. In traditional enterprise systems this “cold start” process can realistically take tens of minutes. But it’s near impossible to have demand-based scaling when things take that long to start.

GraalVM is a Java Virtual Machine implementation that addresses parts of the “cold start” problem by doing Ahead-Of-Time (AOT) compilation on JVM-based applications. GraalVM can create a “native image” of your application so that it no longer needs to run inside a JVM. This can reduce startup time and in some cases improve overall performance. The native images can also be much smaller than the usual OS + JVM + all dependency JARs. For example, a recent application I was working on went from a pretty trim 208MB docker image (OS + JVM + deps + app) that started in 2 seconds, down to 14MB and a 0.5s startup time. Sounds amazing! But there are some caveats, tips, and tricks I’d like to share with you.

Let’s start with a couple samples:

Check out the README’s for instructions on how to build and run those. Now let’s dive into some tips & tricks.

Build Containers in Containers

One thing you might have noticed in those samples is that I run the builds and native-image tool inside of Docker because that is the most reliable way to have a reproducible build with GraalVM. Here is the basic structure for a GraalVM native image Dockerfile:

FROM oracle/graalvm-ce:20.0.0-java11 as builder

WORKDIR /app
COPY . /app

RUN gu install native-image

# Build the app (via Maven, Gradle, etc) and create the native image

FROM scratch

COPY --from=builder /app/target/my-native-image /my-native-image

ENTRYPOINT ["/my-native-image"]

Of course it’s time consuming to re-run a docker build like this as part of the development cycle so it’s nice to just run these apps on a JVM for local development and only create the native image as part of the CI/CD deployment process. Quarkus and sbt have a way to run the native-image tool inside a docker container without needing a Dockerfile but those methods don’t seem to support some customizations that we will get into shortly. Quarkus and sbt also have build tasks that build the app and then run the native-image tool for you. Gradle / Micronaut instead just use the regular packaging tasks and then you can run native-image in your Dockerfile, like:

RUN ./gradlew --no-daemon --console=plain distTar
RUN tar -xvf build/distributions/hello-micronaut.tar -C build/distributions
RUN native-image -cp build/distributions/hello-micronaut/lib/hello-micronaut.jar:build/distributions/hello-micronaut/lib/* hello.WebAppKt

This more explicit approach makes it clear that native-image is just taking the jar files containing the app and the dependencies as well as a class with a static void main and turning those into the native image.

From Scratch

GraalVM has a neat feature that allows us to build a statically linked native image using the --static parameter. If you are calling native-image directly then you just add that parameter. If you are using Maven / Quarkus or sbt then you’ll use a build config parameter to add it (i.e. pom.xml, build.sbt). By default this still requires your base container image to have libc and libstdc++ on it. But wouldn’t it be cool to build a statically linked native image and put it into an empty container image with no system dependencies giving us super small containers? Well you can use FROM scratch in the Dockerfile and copy the native image in! But problems arise when an execution path tries to call something in libc or libstdc++ which is the case with DNS lookups.

Luckily GraalVM has a way to also include the necessary system libraries in the static native image! There is a library called musl libc that you can include in your native image. In your Dockerfile download the musl bundle for GraalVM:

RUN curl -L -o musl.tar.gz https://github.com/gradinac/musl-bundle-example/releases/download/v1.0/musl.tar.gz && \
    tar -xvzf musl.tar.gz

And then add a native-image parameter that points to the extracted location of the bundle, like:

-H:UseMuslC=bundle/

Now your native image will include the standard library system calls that are needed!

Don’t Fall Back

GraalVM has a default feature where if the AOT thing fails, it will fallback to just running the app in the JVM. You likely don’t want that because why would you be creating a native image if it’s just going to end up running in the JVM? Right, so make sure you set a native-image param:

--no-fallback

Don’t Defer Problems to Runtime

Some AOT compilation errors can be deferred to runtime. You really don’t want this unless you like production crashes. So make sure native-image is NOT being run with any of these params:

--report-unsupported-elements-at-runtime
--allow-incomplete-classpath
-H:+ReportUnsupportedElementsAtRuntime

Reflection Woes

Lots and lots and lots of very widely adopted Java libraries use reflection but it creates a problem for GraalVM native images because reflection happens at runtime, making it hard for an AOT complier to figure out the execution paths. To deal with this you can tell GraalVM about what needs reflection access. But this can quickly get a bit out-of-hand, hard to derive, and hard to maintain. Micronaut and Quarkus do a pretty good job generating the reflection configuration at compile time but you might need to augment the generated config. This can get a bit tricky when you have dependencies with shaded transitive dependencies.

To reliably generate a reflection config you need to exercise as many execution code paths as possible, ideally by running your unit and integration tests. GraalVM has a way to keep track of reflection and output the configuration. To use this you’ll need to run the app on GraalVM (remember it’s just a JVM implementation) and use a special Java agent that will be able to see the reflective calls. I didn’t do this inside Docker because then getting the generated config files out becomes more challenging. First, you need to grab a GraalVM Community Edition release, set your JAVA_HOME and PATH. Then also from the release assets grab the right native-image-installable-svm-BLAH.jar file and extract it in the root of your GraalVM JAVA_HOME directory. Ok, now run your tests with this parameter:

-agentlib:native-image-agent=config-output-dir=src/graal"

This will generate the reflection config (and possibly other configs for dynamic proxies, etc). Then you need to tell native-image about those configs, like:

-H:ReflectionConfigurationFiles=src/graal/reflect-config.json

Note: For Quarkus & Micronaut see their docs (Quarkus / Micronaut) for details on how to add your own reflection config files.

Or better, don’t use reflection or things that use reflection. More on that in a future blog. :)

What Else?

I only have one small app in production that uses GraalVM native image, running on Cloud Run via a FROM scratch container image. It starts up super fast and lives in a 44MB container image. GraalVM really helped address the cold start problem. But what about for more complex applications? I dunno. Let me know how it goes and what you’ve learned!